System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting personal data of a user; means for generating a customized virtual character based on the personal data of the user; means for analyzing daily conversations with the user and reducing sense of alienation; means for monitoring health condition of the user and detecting abnormalities; means for determining risk of fraud and crime and alerting to suspicious contacts; means for automatically notifying local doctors and public agencies if dementias are suspected; and means for providing a user-friendly interface.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to a life support system primarily for elderly people living alone. In particular, it aims to provide technology to address the following issues faced by elderly people:

[0005] 1. Reducing the sense of social isolation and loneliness felt by elderly people living alone.

[0006] 2. Constant monitoring of the user's health status and prompt response when an abnormality occurs.

[0007] 3. Strengthened protection against fraud and crime.

[0008] 4. Early detection of dementia and appropriate treatment.

[0009] 5. Providing a user-friendly interface.

[0010] To solve these problems, it is necessary to devise a system using multifunctional virtual characters. [Means for solving the problem]

[0011] According to the present invention, a multifunctional virtual character system is provided to solve the above-mentioned problems.

[0012] The system includes the following means:

[0013] 1. Means of collecting user personal information: Information such as the user's hobbies, past life history, and personal preferences is collected during initial registration and sent to the server.

[0014] 2. Means for generating a customized virtual character based on the user's personal information: Analyze the collected information and generate and display a virtual character with customized appearance and personality.

[0015] 3. Analyzing everyday conversations with users to reduce feelings of alienation: A virtual character will converse with the user on everyday topics, analyze the content of the conversation, and reflect it in the next conversation.

[0016] 4. A means of monitoring the user's health condition and detecting abnormalities: Changes in voice and facial color are monitored in real time, and an alert is automatically issued if an abnormality occurs.

[0017] 5. A means of determining the risk of fraud and crime and issuing warnings about suspicious communications: Analyze the content of calls and messages in real time and display warnings about communications that pose a risk of fraud or crime.

[0018] 6. A means of automatically notifying local doctors and public institutions in the event of suspected dementia: Patterns indicating memory and cognitive decline will be analyzed, and relevant institutions will be automatically notified in the event of suspected dementia.

[0019] 7. Means of providing a user-friendly interface: Provide an interface that can be easily used by the elderly.

[0020] These measures will enable multifaceted support for elderly people living alone and improve their quality of life.

[0021] "User's personal information" refers to information such as the user's hobbies, past life history, and personal preferences.

[0022] A "virtual character" is a computer-generated imaginary entity with a particular appearance and personality.

[0023] "Daily conversation" refers to typical everyday interactions between a user and a virtual character.

[0024] "Monitoring" refers to the act of constantly observing the user's voice, changes in facial expression, etc., and acquiring data.

[0025] "Means for monitoring health status" refers to the function of monitoring the user's voice, complexion, etc. and checking their condition.

[0026] "Means for issuing a warning when an abnormality occurs" refers to a function that notifies the user of a warning when a health problem is determined as a result of monitoring.

[0027] "Means for determining the risk of fraud and crime" refers to the ability to analyze the content of calls and messages and identify risks related to fraud and crime.

[0028] "Means for automatically notifying if dementia is suspected" refers to a function that automatically sends information to relevant institutions if it is determined that there is a problem with the user's cognitive function.

[0029] A "user-friendly interface" refers to an easy-to-use operation screen and navigation that can be used intuitively by seniors.

[0030] "Server" refers to a computer system that collects, analyzes, and manages data, and communicates with each terminal.

[0031] "Terminal" refers to a device or equipment that a user directly operates to interact with virtual characters and use various functions. [Brief explanation of the drawings]

[0032] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0033] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0034] First, the terms used in the following description will be explained.

[0035] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0036] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0037] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0038] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0039] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0040] [First embodiment]

[0041] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0042] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0043] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0044] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0045] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0046] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0047] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0048] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0049] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0050] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0051] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0052] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0053] The present invention is a life support system primarily targeted at elderly people living alone, and is designed to solve various problems faced by elderly people. Specific embodiments for carrying out the present invention are described below.

[0054] 1. Collecting personal information and creating a customized avatar

[0055] User

[0056] During initial registration, users enter personal information into the interface, such as their hobbies, past life history, and personal preferences.

[0057] Terminal

[0058] The terminal collects the information entered by the user and sends it to the server.

[0059] server

[0060] The server analyzes the received personal information and uses databases and machine learning algorithms to customize the appearance and personality of the virtual character.

[0061] Data of a customized virtual character is generated and transmitted to a terminal.

[0062] Terminal

[0063] The terminal displays the received virtual character on the screen and starts an initial conversation with the user.

[0064] Specific examples

[0065] When a user enters information such as "I love dogs and used to be a nurse," the server generates an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on the screen and asks, "Hello. Is there anything I can help you with?"

[0066] 2. Everyday conversation reduces feelings of alienation

[0067] Terminal

[0068] The terminal periodically receives topics for talking to the user from the server.

[0069] Start the conversation with everyday topics such as "How was your day?"

[0070] User

[0071] The user can have everyday conversations with the virtual character, for example, by replying, "The weather was nice today, so I went for a walk in the park."

[0072] Terminal

[0073] The terminal converts the user's response into text data using voice recognition technology and sends it to the server.

[0074] server

[0075] The server analyzes the received conversation content, generates the next conversation topic and reply content, and sends them to the terminal.

[0076] Specific examples

[0077] When a user says, "I went shopping at the local supermarket today," the device converts the information into text and sends it to the server. The server then generates data to ask the next question, "What did you buy at the supermarket?", and sends it to the device.

[0078] 3. Health monitoring and abnormality detection

[0079] Terminal

[0080] The device uses a built-in camera and microphone to monitor the user's tone of voice, facial expression, and other information in real time.

[0081] The monitoring results are sent to the server.

[0082] server

[0083] The server analyzes the received data and detects any abnormalities in health status.

[0084] If an abnormality is detected, a warning message is generated and sent to the terminal.

[0085] Terminal

[0086] The device will notify the user of the warning message on the screen or via voice, and if necessary, will automatically notify public authorities.

[0087] Specific examples

[0088] If the device detects abnormalities such as "a trembling voice" or "a pale complexion," it sends that information to the server. The server suspects "abnormal blood pressure" and displays a warning on the device saying, "You seem unwell. Would you like to call a doctor?"

[0089] 4. Risk assessment to prevent fraud and crime

[0090] Terminal

[0091] The device monitors the content of calls and messages in real time and sends any content deemed risky to the server.

[0092] server

[0093] The server analyzes the content of incoming messages and applies algorithms to determine the risk of fraud or crime.

[0094] If a risk is identified, an alert message is generated and sent to the device.

[0095] Terminal

[0096] The device will notify the user of the alert via visual or audio notification, and if necessary, will automatically notify trusted contacts.

[0097] Specific examples

[0098] The device receives a message saying "You have received a large bill" and sends it to the server. The server determines that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[0099] 5. Detecting and responding to suspected dementia

[0100] Terminal

[0101] The device continuously records the conversation with the user and transmits it to the server.

[0102] server

[0103] The server analyzes the received data and detects patterns that may indicate dementia.

[0104] If there is any suspicion, doctors and public authorities will be automatically notified.

[0105] Specific examples

[0106] The device records behaviors such as "repeating the same story over and over again, even yesterday" and "forgetting where the house is," and sends the records to a server. The server then detects possible early symptoms of dementia and automatically notifies a doctor.

[0107] 6. Providing a user-friendly interface

[0108] server

[0109] The server designs a simple and intuitive interface for seniors and sends it to the device.

[0110] Terminal

[0111] The device uses large buttons and voice navigation to make it easy for users to use.

[0112] Provide necessary operating instructions via voice or text.

[0113] Specific examples

[0114] The device is designed so that users can use it simply by touching the large buttons that display functions such as "talk to avatar" and "health check." Voice guidance is provided to help users navigate the device without any problems.

[0115] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[0116] The processing flow will be explained below.

[0117] Collecting personal information and creating a customized avatar

[0118] Step 1:

[0119] User

[0120] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[0121] Step 2:

[0122] Terminal

[0123] The terminal collects personal information entered by the user and transmits the data to the server.

[0124] Step 3:

[0125] server

[0126] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character optimized for the user.

[0127] Step 4:

[0128] server

[0129] Data of the customized virtual character is generated and transmitted to the terminal.

[0130] Step 5:

[0131] Terminal

[0132] The terminal displays the received virtual character on the screen, and the user can begin the initial interaction.

[0133] Reducing feelings of alienation through everyday conversation

[0134] Step 1:

[0135] Terminal

[0136] The terminal periodically receives topics for talking to the user from the server.

[0137] Step 2:

[0138] Terminal

[0139] Start the conversation with everyday topics such as "How was your day?"

[0140] Step 3:

[0141] User

[0142] The user converses with the virtual character, replying, for example, "I went to the park today."

[0143] Step 4:

[0144] Terminal

[0145] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[0146] Step 5:

[0147] server

[0148] The server analyzes the received conversation content and creates information to generate the next conversation topic and reply content. The created information is sent to the device.

[0149] Step 6:

[0150] Terminal

[0151] The device uses the received information to prepare the next conversation topic.

[0152] Health monitoring and anomaly detection

[0153] Step 1:

[0154] Terminal

[0155] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[0156] Step 2:

[0157] Terminal

[0158] The collected monitoring data is sent to the server.

[0159] Step 3:

[0160] server

[0161] The server analyzes the received data and checks for any abnormalities in the person's health.

[0162] Step 4:

[0163] server

[0164] If an abnormality is detected, a warning message is generated and sent to the terminal.

[0165] Step 5:

[0166] Terminal

[0167] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[0168] Risk assessment for fraud and crime prevention

[0169] Step 1:

[0170] Terminal

[0171] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[0172] Step 2:

[0173] server

[0174] The server analyzes the content of the received messages and assesses the possibility of fraud or criminal activity.

[0175] Step 3:

[0176] server

[0177] If a risk is identified, an alert message is generated and sent to the device.

[0178] Step 4:

[0179] Terminal

[0180] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[0181] Detecting and responding to suspected dementia

[0182] Step 1:

[0183] Terminal

[0184] The device continuously records the conversation with the user and transmits the data to a server.

[0185] Step 2:

[0186] server

[0187] The server analyzes the received data and checks for signs of dementia.

[0188] Step 3:

[0189] server

[0190] If dementia is suspected, local doctors and public institutions will be automatically notified.

[0191] Providing a user-friendly interface

[0192] Step 1:

[0193] server

[0194] Design a simple and intuitive interface for the elderly and send the design data to the device.

[0195] Step 2:

[0196] Terminal

[0197] The device displays a user-friendly interface on the screen with simple buttons, large text, and voice navigation.

[0198] Step 3:

[0199] Terminal

[0200] The system provides voice or text guidance on how to operate the system, helping users to use it easily.

[0201] Example 1

[0202] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] Elderly people living alone face a sense of isolation due to a lack of daily conversation, undetected health problems, the risk of becoming involved in fraud or crime, and the difficulty of early detection of dementia. These issues significantly reduce the quality of life and safety of the elderly. Conventional technologies have the difficulty of solving these problems comprehensively and efficiently.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0205] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the personal information of the user using a machine learning algorithm, and means for analyzing daily conversations with the user and reducing the sense of alienation using a natural language processing algorithm. This makes it possible to provide a sense of psychological security through the generation of a virtual character optimized for the individual based on the personal information entered by the user and daily conversations.

[0206] The server also includes a means for monitoring the user's health condition in real time using a built-in camera and microphone and using a data analysis algorithm to detect abnormalities, and a means for monitoring the content of calls and messages in real time and applying a natural language processing algorithm to determine the risk of fraud or crime and issue a warning, thereby enabling early detection of the user's health abnormalities and the risk of fraud or crime and prompting appropriate measures.

[0207] Furthermore, the server includes a means for continuously recording the contents of the user's conversations, automatically notifying local doctors or public institutions if dementia is suspected, and a means for providing a simple and intuitive interface for the elderly, which will enable early detection of dementia, appropriate follow-up, and highly user-friendly system operation.

[0208] "Personal information" refers to information about a user in general, such as the user's hobbies, past life history, personal preferences, etc.

[0209] A "machine learning algorithm" refers to a computational method for learning patterns from data and making predictions or classifications.

[0210] "Virtual character" refers to a virtual person or animal generated on a computer based on a user's personal information.

[0211] "Natural language processing algorithms" refer to algorithms that analyze and understand human language.

[0212] "Built-in camera" refers to a device that is built into a device and is used to capture video.

[0213] A "microphone" refers to a device used to record sound.

[0214] "Data analysis algorithms" refer to methods used to analyze collected data and detect anomalies and patterns.

[0215] "Real-time" refers to processing or reaction occurring immediately in the current time.

[0216] "Risk of fraud or crime" refers to the possibility of being involved in fraudulent or criminal activity.

[0217] "Automatic notification" refers to the ability of the system to automatically notify pre-defined contacts when certain conditions are met.

[0218] The term "elderly" generally refers to people aged 65 and over.

[0219] "Interface" refers to the screen and operating means through which the user interacts with the system.

[0220] "Continuous recording" refers to the consistent collection and storage of data over a period of time.

[0221] "Alienation" refers to the psychological state of feeling isolated from society or community.

[0222] "What is typing?"

[0223] Refers to a device or method for inputting characters.

[0224] The present invention is a life support system for elderly people living alone, designed to solve various problems faced by elderly people. The system generates a customized virtual character using the user's personal information, and performs daily conversations, health management, fraud prevention, and early dementia detection. Specific embodiments are as follows.

[0225] 1. Collecting personal information and creating a customized avatar

[0226] User

[0227] The user uses the interface to input personal information such as their hobbies, past life history, and personal preferences.

[0228] Terminal

[0229] The device collects the information entered by the user and sends it to the server in JSON format or similar.

[0230] server

[0231] The server uses machine learning algorithms (e.g., TensorFlow) to analyze the received personal information and search a database to generate a profile of a virtual character that best suits the user.

[0232] The server transmits the generated avatar data to the terminal.

[0233] Terminal

[0234] The terminal displays the received avatar data on the screen and starts an initial dialogue with the user.

[0235] Specific examples

[0236] If a user enters information like "I love dogs and used to be a nurse," the device sends that information to the server. The server uses a machine learning algorithm to generate an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on its screen and asks, "Hello. Is there anything I can help you with?"

[0237] 2. Everyday conversation reduces feelings of alienation

[0238] Terminal

[0239] The device periodically receives conversation topics from the server, for example, "How was your day?"

[0240] User

[0241] The user can have everyday conversations with the virtual character, replying with things like, "The weather was nice today, so I went for a walk in the park."

[0242] Terminal

[0243] The device converts the user's response into text data using voice recognition technology (e.g., Google Speech-to-Text) and sends it to the server.

[0244] server

[0245] The server analyzes the received conversation content using a natural language processing algorithm (e.g., GPT-3), generates the next conversation topic and response content, and sends them to the device.

[0246] Specific examples

[0247] When a user says, "I went shopping at the local supermarket today," the device uses voice recognition technology to convert the content into text and send it to the server. The server then uses a natural language processing algorithm to generate data to ask the question, "What did you buy at the supermarket?" and sends it to the device.

[0248] 3. Health monitoring and abnormality detection

[0249] Terminal

[0250] The device uses a built-in camera (e.g., a general HD camera) and microphone (e.g., a high-sensitivity microphone) to monitor the user's voice tone, facial expression, etc. in real time.

[0251] Terminal

[0252] The monitoring results are sent to the server.

[0253] server

[0254] The server analyzes the received data and uses algorithms (e.g., TensorFlow) to detect abnormalities in health status.

[0255] server

[0256] If an abnormality is detected, the server generates a warning message and sends it to the terminal.

[0257] Terminal

[0258] The device will notify the user of the warning message on the screen and via voice, and if necessary, will automatically notify public authorities.

[0259] Specific examples

[0260] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to a server. The server then uses a data analysis algorithm to suspect an abnormality in blood pressure and displays a warning on the device saying, "You appear unwell. Would you like to call a doctor?"

[0261] 4. Risk assessment to prevent fraud and crime

[0262] Terminal

[0263] The device monitors the content of calls and messages in real time.

[0264] Terminal

[0265] Any content deemed to be risky is sent to the server.

[0266] server

[0267] The server analyzes incoming messages using natural language processing algorithms to determine the risk of fraud or crime.

[0268] server

[0269] If a risk is identified, an alert message is generated and sent to the device.

[0270] Terminal

[0271] The device will notify the user of alerts visually and audibly, and if necessary, will automatically notify trusted contacts.

[0272] Specific examples

[0273] The device receives a message saying "You have received a large bill" and sends it to the server. The server uses a natural language processing algorithm to determine that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[0274] 5. Detecting and responding to suspected dementia

[0275] Terminal

[0276] The device continuously records the conversation with the user and transmits it to the server.

[0277] server

[0278] The server analyzes the received data and detects patterns that may indicate dementia (e.g., memory loss, inappropriate behavior).

[0279] server

[0280] In case of suspicion, an automatic notification is generated and sent to doctors and public authorities.

[0281] Specific examples

[0282] The device records behaviors such as "repeating the same story over and over again, even yesterday" or "forgetting where home is," and sends the records to a server. The server then uses a machine learning model to detect possible early symptoms of dementia and automatically notify a doctor.

[0283] 6. Providing a user-friendly interface

[0284] server

[0285] The server designs a simple and intuitive interface for seniors and sends it to the device.

[0286] Terminal

[0287] The device uses large buttons and voice navigation to make it easy for users to use.

[0288] Terminal

[0289] Provide necessary operating instructions via voice or text.

[0290] Specific examples

[0291] The device is equipped with large buttons for functions such as "talk to avatar" and "health check," and is designed so that users can use it simply by touching them. Voice guidance is provided to help users navigate the device without any problems.

[0292] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[0293] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0294] Specific processing steps of the system

[0295] 1. Collecting personal information and creating a customized avatar

[0296] Step 1: Enter user personal information

[0297] The user uses the interface to input personal information such as hobbies, past life history, and personal preferences.

[0298] Input: User's hobbies, past life history, personal preferences.

[0299] Output: User's personal information data.

[0300] Step 2: Submit your information

[0301] The device organizes the personal information entered by the user and sends it to the server in JSON format or similar.

[0302] Input: User's personal information data.

[0303] Output: Personal information data in JSON format.

[0304] Step 3: Data analysis and avatar generation

[0305] The server inputs the received personal information into a machine learning algorithm (e.g., TensorFlow) to analyze it, and searches a database to generate a profile of a virtual character that best suits the user.

[0306] Input: Personal information data in JSON format.

[0307] Output: A profile of the generated virtual character.

[0308] Step 4: Send and display your avatar

[0309] The server transmits the generated avatar data to the terminal.

[0310] Input: The profile of the generated virtual character.

[0311] Output: Avatar data sent to the device.

[0312] The terminal displays the received avatar data on the screen and starts an initial dialogue with the user.

[0313] Input: Avatar data sent from the server.

[0314] Output: A virtual character displayed on the screen.

[0315] 2. Everyday conversation reduces feelings of alienation

[0316] Step 1: Receiving conversation topics

[0317] The terminal periodically receives conversation topics from the server.

[0318] Input: Conversation topic data from the server.

[0319] Output: Conversation topic data saved on your device.

[0320] Step 2: Start a conversation

[0321] The device talks to the user about everyday topics such as "How was your day?"

[0322] Input: Conversation topic data.

[0323] Output: Voice or text conversation start.

[0324] Step 3: User response

[0325] The user engages in everyday conversation with the virtual character, replying, "The weather was nice today, so I went for a walk in the park."

[0326] Input: Question from terminal.

[0327] Output: The user's response.

[0328] Step 4: Convert the audio data

[0329] The device converts the user's response into text data using voice recognition technology (e.g., Google Speech-to-Text) and sends it to the server.

[0330] Input: The user's spoken response.

[0331] Output: The response converted to text data.

[0332] Step 5: Analyzing conversation content and generating next topics

[0333] The server analyzes the received conversation content using a natural language processing algorithm (e.g., GPT-3) and generates the next conversation topic and response content.

[0334] Input: The user's response converted into text data.

[0335] Output: Next conversation topic and reply.

[0336] The server transmits the generated data to the terminal.

[0337] Input: Next conversation topic and reply content.

[0338] Output: The next conversation topic and reply sent to your device.

[0339] 3. Health monitoring and abnormality detection

[0340] Step 1: Monitoring your health data

[0341] The device uses a built-in camera (e.g., a general HD camera) and microphone (e.g., a high-sensitivity microphone) to monitor the user's voice tone, facial expression, etc. in real time.

[0342] Input: User's video and audio data.

[0343] Output: Monitored health data.

[0344] Step 2: Sending data

[0345] The terminal transmits the monitoring results to the server.

[0346] Input: Monitored health data.

[0347] Output: Health data sent to the server.

[0348] Step 3: Data analysis and anomaly detection

[0349] The server analyzes the received data and uses algorithms (e.g., TensorFlow) to detect abnormalities in health status.

[0350] Input: Health data received by the server.

[0351] Output: Anomaly detection results.

[0352] Step 4: Generate and send a warning message

[0353] If an abnormality is detected, the server generates a warning message and sends it to the terminal.

[0354] Input: Anomaly detection results.

[0355] Output: Generated warning message data.

[0356] Step 5: Notification of warning messages

[0357] The device will notify the user of the warning message on the screen and via voice, and if necessary, will automatically notify public authorities.

[0358] Input: The alert message data sent by the server.

[0359] Output: Warning notification to the user and automatic notification to public authorities.

[0360] 4. Risk assessment to prevent fraud and crime

[0361] Step 1: Monitor calls and messages

[0362] The device monitors the content of calls and messages in real time.

[0363] Input: User's call and message data.

[0364] Output: Monitoring result data.

[0365] Step 2: Submit risk data

[0366] The device sends any content that is determined to be risky to the server.

[0367] Input: Monitoring result data.

[0368] Output: The risk data sent to the server.

[0369] Step 3: Risk analysis and alert generation

[0370] The server analyzes incoming messages using natural language processing algorithms to determine the risk of fraud or crime.

[0371] Input: Risk data sent to the server.

[0372] Output: Risk assessment result and alert message.

[0373] Step 4: Alert Notification

[0374] The device will notify the user of alerts visually and audibly, and if necessary, will automatically notify trusted contacts.

[0375] Input: The alert message data sent from the server.

[0376] Output: Alert notification to user and automatic notification to trusted contacts.

[0377] 5. Detecting and responding to suspected dementia

[0378] Step 1: Record the conversation

[0379] The device continuously records the conversation with the user and transmits it to the server.

[0380] Input: User conversation data.

[0381] Output: The transcript data sent to the server.

[0382] Step 2: Data analysis and pattern detection

[0383] The server analyzes the received data and detects patterns that may indicate dementia (e.g., memory loss, inappropriate behavior).

[0384] Input: Conversation recording data sent to the server.

[0385] Output: Pattern detection results.

[0386] Step 3: Generate notifications

[0387] The server generates and sends automatic notifications to doctors and public authorities in case of suspicion.

[0388] Input: Pattern detection results.

[0389] Output: Automatic notification to doctors and public authorities.

[0390] 6. Providing a user-friendly interface

[0391] Step 1: Design the interface

[0392] The server designs a simple and intuitive interface for seniors.

[0393] Input: Usability design data.

[0394] Output: The designed interface data.

[0395] Step 2: Send Interface

[0396] The server sends the designed interface to the terminal.

[0397] Input: The designed interface data.

[0398] Output: Interface data sent to the terminal.

[0399] Step 3: View the interface

[0400] The device uses large buttons and voice navigation to make it easy for users to use.

[0401] Input: Interface data sent by the server.

[0402] Output: A user-friendly interface displayed on the screen.

[0403] Step 4: Providing operation guides

[0404] The device will provide necessary operating instructions via voice or text.

[0405] Input: User operation status and requests.

[0406] Output: Voice and text guide.

[0407] (Application example 1)

[0408] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] In modern society, elderly people living alone face various challenges in their daily lives. In particular, they need to deal with feelings of loneliness due to a lack of daily conversation, inadequate health monitoring, the risk of becoming a victim of fraud and crime, and the progression of dementia. Another major issue is the lack of dietary suggestions tailored to individual dietary preferences and health conditions. There is a need to develop a system that can solve these issues and provide an environment where elderly people living alone can live with peace of mind.

[0410] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0411] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the user's personal information, means for analyzing daily conversations with the user to reduce feelings of alienation, means for monitoring the user's health condition and detecting abnormalities, means for assessing the risk of fraud or crime and issuing a warning against suspicious communications, means for automatically notifying local doctors or public institutions if dementia is suspected, means for providing a user-friendly interface, means for suggesting customized dishes based on the user's personal information, and means for suggesting healthy dishes based on the user's health condition. This enables multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[0412] "User's personal information" refers to information necessary for the system to provide personalized services for each user, such as the user's hobbies, preferences, past lifestyle history, health status, and allergy information.

[0413] A "customized virtual character" is a virtual entity that is specially designed based on the user's personal information, is friendly to the user, and is used for daily conversations and providing services.

[0414] "Means for reducing feelings of alienation" refers to techniques and methods for reducing feelings of loneliness and alienation by having a virtual character regularly engage in everyday conversations with the user, giving the user a sense of social connection.

[0415] "Means for monitoring health status and detecting abnormalities" refers to a function that uses technologies such as voice recognition and facial color analysis to continuously monitor the user's health status and issue an alert if an abnormality is detected.

[0416] "Means for determining the risk of fraud or crime and issuing warnings for suspicious communications" refers to technology that monitors the content of calls and messages and issues a warning to users if it is determined that there is a risk of fraud or crime based on predefined keywords or patterns.

[0417] "Means for automatically notifying local doctors and public institutions if dementia is suspected" is a system that analyzes a user's conversation patterns and behavior, and if it detects suspicion of dementia, automatically notifies local doctors and public institutions.

[0418] "Means for providing a user-friendly interface" refers to a user interface design that provides large buttons, voice navigation, simple operation screens, and other features that are generally easy for seniors to use.

[0419] "Means for providing customized food suggestions" refers to technologies and algorithms that provide personalized food suggestions based on a user's personal information (such as preferences and allergy information).

[0420] The "means for suggesting healthy meals" is a mechanism for suggesting healthy meal menus based on the user's health condition and nutritional balance.

[0421] A "generative AI model" is an artificial intelligence algorithm or model used to interact with users and analyze data.

[0422] A "prompt" is a question or introductory text that a generative AI model uses when engaging in dialogue or making suggestions.

[0423] This invention provides a food delivery service as part of a lifestyle assistance system for elderly people living alone. The system generates a customized virtual character based on the user's personal information and reduces the user's sense of alienation through everyday conversations. It also monitors the user's health and provides a sense of security by detecting abnormalities. It also has a function to assess the risk of fraud and crime and automatically notify local doctors and public institutions if dementia is suspected. The system is designed to be easy for anyone to use, providing a user-friendly interface.

[0424] The system mainly consists of the following hardware and software:

[0425] Hardware: Smartphone, built-in camera, microphone

[0426] Software: Speech recognition technology (Google Cloud Speech-to-Text API), machine learning algorithms (Python's scikit-learn library), voice response technology (Google Text-to-Speech API), interface design (React Native)

[0427] 1. Collecting personal information and creating a customized avatar

[0428] The server collects personal information, such as hobbies, past life history, and personal preferences, entered by the user during initial registration. Based on this information, a virtual chef character is generated. For example, based on the information that "I like pasta and have no allergies," a friendly virtual chef is created and sent to the terminal. The terminal begins an initial dialogue with the user by displaying the virtual chef.

[0429] 2. Everyday conversation reduces feelings of alienation

[0430] The device periodically receives topics from the server to talk to the user. Conversations such as "What would you like to eat today?" are automatically generated. Through everyday conversations with the virtual chef, the user can reduce their sense of alienation. For example, if the user replies "I would like to eat salad today," the device converts this into text data and sends it to the server. The server then generates data to ask the question "What did you think of the salad?" in the next conversation.

[0431] 3. Health monitoring and abnormality detection

[0432] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color in real time. This data is sent to a server, and if an abnormality is detected based on the analysis results, a warning message is generated and sent to the device. For example, if a trembling voice or abnormal facial color is detected, a warning message will be sent saying, "You seem unwell. Would you like to call a doctor?"

[0433] 4. Risk assessment to prevent fraud and crime

[0434] The device monitors the content of calls and messages in real time, and if it determines there is a risk, it sends it to the server. The server applies an algorithm to determine the risk of fraud or crime, generates an alert message, and sends it to the device. For example, if a message containing a high bill is received, a warning will be displayed saying, "This may be a scam. Please ignore it."

[0435] 5. Detecting and responding to suspected dementia

[0436] The device continuously records the conversations with the user and sends them to a server. The server analyzes the received data and automatically notifies doctors and public institutions if dementia is suspected. For example, if the user repeats the same story multiple times or forgets where their home is, the server will automatically notify them that they may be experiencing early symptoms of dementia.

[0437] 6. Providing a user-friendly interface

[0438] The device is designed to be easy for users to operate, using large buttons and voice navigation. The server designs a simple and intuitive interface and sends it to the device. For example, it has large buttons such as "Talk to an avatar" and "Health check," which users can use simply by touching them.

[0439] Examples and prompts

[0440] As a specific example of a conversation, if a user says, "What should I eat today?", the virtual chef will suggest, "How about a fresh salad?". If a user also says, "I went shopping at the local supermarket today," the server will generate data to ask, "What did you buy at the supermarket?" in the next conversation.

[0441] Prompt Sentence Examples

[0442] "Is there anything in particular you'd like to eat today?"

[0443] "How are you feeling? Is there anything that's bothering you?"

[0444] "Want to try a new dish?"

[0445] This will enable multifaceted support for the lives of elderly people living alone, providing them with a safe and comfortable life.

[0446] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0447] Step 1:

[0448] The user inputs personal information such as their hobbies, lifestyle history, personal preferences, and allergy information into the device (smartphone). This information is sent to the server via the device. This inputs the user's personal information into the server.

[0449] Step 2:

[0450] After receiving the user's personal information, the server analyzes the data using machine learning algorithms to generate a virtual character suited to the user, whose appearance and personality are customized to suit the user's preferences.

[0451] Step 3:

[0452] The server transmits the data of the generated customized virtual character to the terminal, which receives the data and displays the virtual character on the screen, allowing the user to start interacting with the virtual character.

[0453] Step 4:

[0454] The terminal periodically receives conversation topics from the server. The server creates a prompt and sends it to the terminal. For example, a prompt such as "Is there anything in particular you'd like to eat today?"

[0455] Step 5:

[0456] The user converses with the virtual character, and the user's response (e.g., "I want to eat salad today") is converted into text data using voice recognition technology and sent to the server.

[0457] Step 6:

[0458] The server processes the received conversation content to generate the next conversation topic and suggestions. For example, if a user says they want to eat salad, the next topic generated will be "What did you think of the salad?"

[0459] Step 7:

[0460] The device uses a built-in camera and microphone to monitor the user's voice tone, facial expression, etc. in real time, and the results of this monitoring are sent to the server.

[0461] Step 8:

[0462] The server analyzes the monitoring data and detects abnormalities in health status. If an abnormality is detected, a warning message is generated and sent to the device. For example, if there is a trembling voice or a change in complexion, a warning message will be displayed saying, "You seem unwell. Would you like to contact a doctor?"

[0463] Step 9:

[0464] The device monitors the content of the user's calls and messages in real time, and if it determines that there is a risk of fraud or crime, it sends the content to a server, which then analyzes the data to determine the risk of fraud or crime.

[0465] Step 10:

[0466] If the server detects a fraud or criminal risk, it generates an alert message and sends it to the device, which then displays a warning such as "This may be fraud. Please ignore."

[0467] Step 11:

[0468] The device continuously records conversations with the user and sends them to a server, which analyzes the data and detects patterns that may indicate dementia.

[0469] Step 12:

[0470] If the server suspects dementia, it will automatically notify local doctors and public institutions, for example, saying, "There are suspicions of early symptoms of dementia."

[0471] Step 13:

[0472] The server designs a simple and intuitive interface for the elderly and sends it to the device. The device uses large buttons and voice navigation to make it easy for users to operate. Buttons such as "Talk to an avatar" and "Health check" are provided.

[0473] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0474] The present invention is a life support system for elderly people living alone, and aims to provide more personalized support by recognizing the user's emotions through the combination of an emotion engine. Specific embodiments for carrying out the present invention are described below.

[0475] 1. Collecting personal information and creating a customized avatar

[0476] User

[0477] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[0478] Terminal

[0479] The terminal collects the information entered by the user and sends it to the server.

[0480] server

[0481] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character.

[0482] Terminal

[0483] The customized virtual character data is displayed on the screen, and the user can start the initial interaction.

[0484] Specific examples

[0485] When a user enters information such as "I love cats and used to be a teacher," the server generates an avatar in the shape of a friendly cat and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[0486] 2. Everyday conversation reduces feelings of alienation

[0487] Terminal

[0488] The device periodically receives topics from the server to talk to the user about, starting a conversation with an everyday topic such as "How was your day?"

[0489] User

[0490] The user can have everyday conversations with the virtual character, for example, by replying, "I went shopping at the local supermarket today."

[0491] Terminal

[0492] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[0493] server

[0494] The server analyzes the received conversation content, creates information for generating the next conversation topic and reply content, and sends it to the terminal.

[0495] Specific examples

[0496] When a user says, "The weather was nice today, so I went for a walk in the park," the device converts the content into text and sends it to the server. The server then generates data to ask the question "How was your walk in the park?" in the next conversation and sends it to the device.

[0497] 3. Health monitoring and abnormality detection

[0498] Terminal

[0499] The built-in camera and microphone are used to monitor the user's tone of voice and facial expression in real time, and the collected monitoring data is sent to a server.

[0500] server

[0501] The server analyzes the received data and checks for any abnormalities in the health status. If an abnormality is detected, it generates a warning message and sends it to the device.

[0502] Terminal

[0503] The device will display a warning message to the user on screen or via voice, and if necessary, will automatically notify public authorities or emergency contacts.

[0504] Specific examples

[0505] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to the server. The server suspects the user may be in poor health and displays a warning on the device saying, "You appear to be unwell. Would you like to call a doctor?"

[0506] 4. Risk assessment to prevent fraud and crime

[0507] Terminal

[0508] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[0509] server

[0510] The server analyzes the content of the received message and evaluates the possibility of fraud or crime. If a risk is identified, an alert message is generated and sent to the device.

[0511] Terminal

[0512] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[0513] Specific examples

[0514] The device detects the message "I received a bill but I don't recognize it" and sends it to the server. The server determines that it is likely a scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[0515] 5. Detecting and responding to suspected dementia

[0516] Terminal

[0517] The device continuously records the conversation with the user and transmits the data to a server.

[0518] server

[0519] The server analyzes the received data and checks for signs of dementia. If dementia is suspected, it automatically notifies local doctors and public institutions.

[0520] Specific examples

[0521] The device records behaviors such as "repeats the same story" and "forgets where home is" and sends them to a server. The server then runs a system that suspects "early symptoms of dementia" and automatically notifies a doctor.

[0522] 6. Providing a user-friendly interface

[0523] server

[0524] Design a simple and intuitive interface for the elderly and send the design data to the device.

[0525] Terminal

[0526] The device displays a user-friendly interface with large buttons and voice navigation, and provides voice or text instructions to help users navigate easily.

[0527] Specific examples

[0528] The device's interface has large buttons for "talk to avatar" and "health check," and is designed to allow users to easily operate it using voice navigation.

[0529] 7. Introduction of Emotion Engine and Emotion Recognition

[0530] Terminal and Emotion Engine

[0531] The device uses a built-in camera and microphone to collect the user's voice and facial expression data, which is then sent to the emotion engine.

[0532] Emotion Engine

[0533] The emotion engine analyzes the received voice and facial expression data to recognize the user's emotions (joy, anger, sadness, happiness, etc.), and sends the recognized emotion data to the server.

[0534] server

[0535] The server generates optimal responses and actions based on the emotional data received from the emotion engine, depending on the user's emotional state.

[0536] Terminal

[0537] The terminal allows the virtual character to speak to the user based on the responses and actions received from the server.

[0538] Specific examples

[0539] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[0540] The processing flow will be explained below.

[0541] Collecting personal information and creating a customized avatar

[0542] Step 1:

[0543] User

[0544] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[0545] Step 2:

[0546] Terminal

[0547] The terminal collects the information entered by the user and sends it to the server.

[0548] Step 3:

[0549] server

[0550] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character.

[0551] Step 4:

[0552] server

[0553] Data of the customized virtual character is generated and transmitted to the terminal.

[0554] Step 5:

[0555] Terminal

[0556] A customized virtual character is displayed on the screen, and the user can initiate an initial interaction with this avatar.

[0557] Specific examples

[0558] When a user enters information such as "I love cats and used to be a teacher," the server generates an avatar in the shape of a friendly cat and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[0559] Reducing feelings of alienation through everyday conversation

[0560] Step 1:

[0561] Terminal

[0562] The terminal periodically receives topics for talking to the user from the server.

[0563] Step 2:

[0564] Terminal

[0565] Start the conversation with everyday topics such as "How was your day?"

[0566] Step 3:

[0567] User

[0568] The user can converse with the virtual character, for example, by saying, "I went shopping at the local supermarket today."

[0569] Step 4:

[0570] Terminal

[0571] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[0572] Step 5:

[0573] server

[0574] The server analyzes the received conversation content, creates information for generating the next conversation topic and reply content, and sends it to the terminal.

[0575] Step 6:

[0576] Terminal

[0577] The device uses the received information to prepare the next conversation topic.

[0578] Specific examples

[0579] When a user says, "The weather was nice today, so I went for a walk in the park," the device converts the content into text and sends it to the server. The server then generates data to ask the question "How was your walk in the park?" in the next conversation and sends it to the device.

[0580] Health monitoring and anomaly detection

[0581] Step 1:

[0582] Terminal

[0583] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[0584] Step 2:

[0585] Terminal

[0586] The collected monitoring data is sent to the server.

[0587] Step 3:

[0588] server

[0589] The server analyzes the received data and checks for any abnormalities in the person's health.

[0590] Step 4:

[0591] server

[0592] If an abnormality is detected, a warning message is generated and sent to the terminal.

[0593] Step 5:

[0594] Terminal

[0595] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[0596] Specific examples

[0597] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to the server. The server suspects the user may be in poor health and displays a warning on the device saying, "You appear to be unwell. Would you like to call a doctor?"

[0598] Risk assessment for fraud and crime prevention

[0599] Step 1:

[0600] Terminal

[0601] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[0602] Step 2:

[0603] server

[0604] The server analyzes the content of the received messages and applies algorithms to determine the risk of fraud or crime.

[0605] Step 3:

[0606] server

[0607] If a risk is identified, an alert message is generated and sent to the device.

[0608] Step 4:

[0609] Terminal

[0610] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[0611] Specific examples

[0612] The device receives a message saying, "I received a large bill but I don't recognize it," and sends it to the server. The server determines that it is likely a scam and displays a warning on the device saying, "This may be a scam. Please ignore it."

[0613] Detecting and responding to suspected dementia

[0614] Step 1:

[0615] Terminal

[0616] The device continuously records the conversation with the user and transmits the data to a server.

[0617] Step 2:

[0618] server

[0619] The server analyzes the received data and checks for any problems with memory or cognitive ability.

[0620] Step 3:

[0621] server

[0622] If dementia is suspected, doctors and public institutions will be automatically notified.

[0623] Specific examples

[0624] The device records such things as "Recently, the patient has been repeating the same things over and over again" and "Forgetting where home is" and sends the information to a server. The server then runs a system that suspects "early symptoms of dementia" and automatically notifies a doctor.

[0625] Providing a user-friendly interface

[0626] Step 1:

[0627] server

[0628] Design a simple and intuitive interface for the elderly and send the design data to the device.

[0629] Step 2:

[0630] Terminal

[0631] The device displays a user-friendly interface on the screen with large buttons and voice navigation.

[0632] Step 3:

[0633] Terminal

[0634] The system provides voice or text guidance on how to operate the system, helping users to use it easily.

[0635] Specific examples

[0636] The device's interface has large buttons for "talk to avatar" and "health check," and is designed to allow users to easily operate it using voice navigation.

[0637] Introducing an emotion engine and emotion recognition

[0638] Step 1:

[0639] Terminal and Emotion Engine

[0640] The device uses a built-in camera and microphone to collect the user's voice and facial expression data and send it to the emotion engine.

[0641] Step 2:

[0642] Emotion Engine

[0643] The emotion engine analyzes the received voice and facial expression data to recognize the user's emotions (joy, anger, sadness, happiness, etc.), and sends the recognized emotion data to the server.

[0644] Step 3:

[0645] server

[0646] The server generates optimal responses and actions based on the emotional data received from the emotion engine, depending on the user's emotional state.

[0647] Step 4:

[0648] Terminal

[0649] The terminal allows the virtual character to speak to the user based on the responses and actions received from the server.

[0650] Specific examples

[0651] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[0652] Example 2

[0653] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0654] In modern society, the number of elderly people living alone is increasing, resulting in a wide range of problems, including feelings of isolation and alienation, declining health, the risk of fraud and crime, and early detection of dementia. Conventional solutions to these problems are insufficient, and there is a need to provide an environment where elderly people can live with peace of mind. The objective of this invention is to provide a system that comprehensively solves these issues.

[0655] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting personal information of the user; means for generating a customized virtual character based on the personal information of the user; means for converting the user's daily conversation into text data and analyzing it using voice recognition technology; means for monitoring the user's health status and detecting abnormalities; means for determining the risk of fraud or crime and issuing a warning about suspicious communications; means for automatically notifying local doctors or public institutions if dementia is suspected; means for providing a user-friendly interface including large buttons and voice navigation; and means for recognizing the user's emotions using an emotion engine and generating responses and actions according to the user's emotional state. This makes it possible to provide comprehensive support for elderly people to live with peace of mind.

[0656] "Users" refer to the elderly people who use this system.

[0657] "Personal information" refers to data such as a user's hobbies, past life history, and personal preferences.

[0658] "Virtual character" refers to a virtual character generated based on the user's personal information.

[0659] "Speech recognition technology" refers to technology that converts a user's voice into text data.

[0660] "Monitoring" refers to observing a user's health condition and behavior and collecting data.

[0661] "Abnormal" refers to a state in which the user's health condition or behavior is different from normal and requires emergency response.

[0662] "Risk of fraud or crime" refers to the possibility of fraud or criminal activity against a user.

[0663] "Warning" refers to a warning issued to the user when a risk of fraud or crime is detected.

[0664] "Suspected dementia" refers to a state in which the user shows early symptoms or signs of dementia.

[0665] A "user-friendly interface" refers to an intuitive operating screen designed to be easy for seniors to use.

[0666] "Emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[0667] "Responses and actions" refer to responses and actions generated by the server depending on the user's emotional state.

[0668] This invention is a life support system for elderly people living alone, and aims to provide more personalized support by recognizing the user's emotions through the combination of an emotion engine. This system is composed of multiple hardware and software components.

[0669] Hardware and Software

[0670] The system consists of the following main components:

[0671] server

[0672] Database for collecting and analyzing users' personal information

[0673] Machine learning algorithms for generating virtual characters (e.g., Amazon Web Services' SageMaker or Google Cloud's AutoML)

[0674] Voice recognition technology (e.g., Google Cloud's Speech-to-Text API)

[0675] Models for analyzing health status, fraud risk, and dementia symptoms (e.g., TensorFlow and PyTorch)

[0676] Terminal

[0677] An interface for users to enter personal information and record conversations

[0678] Devices that collect audio and video data using built-in cameras and microphones (e.g., tablets and smartphones)

[0679] User-friendly interface with large buttons and voice navigation

[0680] Emotion Engine

[0681] An engine for analyzing the user's voice and facial expressions to recognize their emotional state (e.g., Emotion API)

[0682] Software that works in conjunction with a server to generate responses and actions according to the user's emotional state

[0683] Data processing and calculation

[0684] The system includes the following processes:

[0685] 1. Collection of personal information and avatar generation

[0686] Through the interface, users input personal information such as hobbies, past life history, and personal preferences. The device sends this information to a server, which then uses machine learning algorithms to analyze the personal information and generate a virtual character (avatar) tailored to the user. The generated avatar is then sent to the device, where the user can begin interacting with it.

[0687] 2. Assistance with everyday conversation

[0688] The device periodically receives conversation topics sent from the server and speaks to the user. The user's responses are converted into text data using voice recognition technology and sent to the server. The server then generates the next conversation topic and response content and sends them to the device.

[0689] 3. Health monitoring

[0690] The system uses a built-in camera and microphone to monitor the user's tone of voice and facial expression, and sends the data to a server. The server analyzes the data and, if an abnormality is detected, generates a warning message and sends it to the device.

[0691] 4. Preventing fraud and crime

[0692] The device monitors the user's calls and messages and sends any content deemed risky to the server. The server analyzes the message content and assesses the risk of fraud or crime. If necessary, it generates an alert message and sends it to the device.

[0693] 5. Detection of suspected dementia

[0694] The device continuously records conversations with the user and sends the data to a server, which analyzes the data and automatically notifies local doctors and public institutions if there are signs of dementia.

[0695] 6. User-friendly interface

[0696] The server designs a simple and intuitive interface for seniors and sends the design data to the device, which displays it and provides voice or text instructions on how to operate it.

[0697] 7. Use of Emotion Engines

[0698] The device uses a built-in camera and microphone to collect the user's voice and facial expression and transmits them to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state. The server generates the optimal response or action based on the recognized emotional data and transmits it to the device.

[0699] Specific examples

[0700] When a user enters information such as "I love cats and used to be a teacher," the server analyzes the information, generates a friendly cat avatar, and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[0701] Prompt Sentence Examples

[0702] "Please give me a detailed explanation of the assisted living system for elderly people living alone. Please include specific examples."

[0703] As described above, the present invention can solve each of these problems by providing multifaceted and personalized support to elderly people living alone.

[0704] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0705] Step 1:

[0706] User

[0707] Through the interface, the user inputs personal information such as hobbies, past life history, personal preferences, etc. This provides personal information as input data.

[0708] Step 2:

[0709] Terminal

[0710] The terminal temporarily stores the personal information entered by the user and sends the data to the server. The input is the personal information entered by the user, and the output is the data to be sent to the server.

[0711] Step 3:

[0712] server

[0713] The server analyzes the received personal information and generates a virtual character using a database and machine learning algorithms. The input is the personal information received from the device, and the output is customized virtual character data.

[0714] Step 4:

[0715] Terminal

[0716] It receives data of a customized virtual character and displays it on the screen. Specifically, the user can start interacting with the generated avatar. The input is character data, and the output is display output.

[0717] Step 5:

[0718] Terminal

[0719] The terminal periodically receives conversation topics sent from the server. The input is the topic data sent from the server, and the output is the data for starting the conversation.

[0720] Step 6:

[0721] Terminal

[0722] Based on the topics received from the server, everyday conversations such as "How was your day?" are generated for the user and displayed as conversations from an avatar. The input is topic data, and the output is the display to the user.

[0723] Step 7:

[0724] User

[0725] The user can have everyday conversations with the virtual character, for example, saying, "I went shopping at the local supermarket today." This allows the input to be the user's voice data.

[0726] Step 8:

[0727] Terminal

[0728] Using speech recognition technology, the user's response is converted into text data and sent to the server as a conversation log. Specifically, Google Cloud's Speech-to-Text API is used. The input is the user's voice, and the output is text data sent to the server.

[0729] Step 9:

[0730] server

[0731] The received conversation content is analyzed, and data for generating the next conversation topic and reply content is created and sent to the terminal. The input is the text data to be analyzed, and the output is the next conversation data.

[0732] Step 10:

[0733] Terminal

[0734] The built-in camera and microphone are used to monitor the user's voice tone and facial expression in real time, and the monitoring data is sent to the server. The input is real-time audio and video data, and the output is monitoring data sent to the server.

[0735] Step 11:

[0736] server

[0737] The received data is analyzed to check for any abnormalities in the health status. If an abnormality is detected, a warning message is generated and sent to the device. TensorFlow is used as a specific example of operation. The input is monitoring data, and the output is a warning message.

[0738] Step 12:

[0739] Terminal

[0740] The warning message is displayed on the screen or is output as an audio message to the user. If necessary, public institutions and emergency contacts are automatically notified. The input is the warning message, and the output is the user notification and emergency contact.

[0741] Step 13:

[0742] Terminal

[0743] The content of calls and messages is monitored in real time, and any content deemed to be risky is sent to the server. The input is message data, and the output is risk data sent to the server.

[0744] Step 14:

[0745] server

[0746] It analyzes the content of received messages and evaluates the risk of fraud or crime. If a risk is confirmed, it generates an alert message and sends it to the terminal. The input is the message data to be analyzed, and the output is the alert message.

[0747] Step 15:

[0748] Terminal

[0749] The alert is notified to the user via a visual or audio message, and if necessary, automatically notifies trusted contacts. The input is the alert message, and the output is the user notification and contact notification.

[0750] Step 16:

[0751] Terminal

[0752] It continuously records the conversation with the user and sends the data to the server. The input is the conversation data, and the output is the data sent to the server.

[0753] Step 17:

[0754] server

[0755] The received data is analyzed to check for signs of dementia. If dementia is suspected, local doctors and public institutions are automatically notified. The input is conversation data, and the output is notification data.

[0756] Step 18:

[0757] server

[0758] We design a simple and intuitive interface for the elderly and send the design data to the terminal. The input is the interface design data, and the output is the design data for the terminal.

[0759] Step 19:

[0760] Terminal

[0761] A user-friendly interface with large buttons and voice navigation is displayed on the screen, and operation instructions are provided via voice or text. Input is design data, and output is displayed on the screen.

[0762] Step 20:

[0763] Terminal

[0764] The built-in camera and microphone are used to collect the user's voice and facial expression data and send it to the emotion engine. The input is the voice and facial expression data, and the output is the data sent to the emotion engine.

[0765] Step 21:

[0766] Emotion Engine

[0767] The received voice and facial expression data is analyzed to recognize the user's emotional state. The recognized emotional data is sent to the server. The input is the data to be analyzed, and the output is emotional data.

[0768] Step 22:

[0769] server

[0770] Based on the emotional data received from the emotion engine, it generates the optimal response and action according to the user's emotional state. The input is emotional data, and the output is response and action data.

[0771] Step 23:

[0772] Terminal

[0773] The virtual character speaks to the user based on the responses and actions received from the server. The input is the response and action data, and the output is the display to the user.

[0774] Specific examples of operation

[0775] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[0776] Through these steps, this system provides multifaceted support to elderly people living alone, helping them live with peace of mind.

[0777] (Application example 2)

[0778] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0779] Existing support systems are insufficient to address the issues faced by elderly people living alone, such as feelings of isolation, lack of health management, risk of fraud and crime, and early detection of dementia. In addition to these issues, elderly people living alone often face difficulties in daily food selection and nutritional management. Existing systems do not offer personalized food recommendations based on emotional and health status, or systems that can arrange for food delivery. Therefore, there is a need for a comprehensive support system to help elderly people living alone continue to live safely and healthily.

[0780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0781] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the user's personal information, means for analyzing daily conversations with the user to reduce feelings of alienation, means for monitoring the user's health and detecting abnormalities, means for assessing the risk of fraud and crime and issuing a warning against suspicious communications, means for automatically notifying local doctors and public institutions if dementia is suspected, means for providing a user-friendly interface, means for evaluating the user's emotional state and health and recommending individual meals, and means for ordering the recommended meals from a delivery service. This comprehensively solves the various challenges faced by elderly people living alone in their daily lives, enabling them to live safe and healthy lives.

[0782] "Personal information" refers to information such as the user's hobbies, past life history, and personal preferences.

[0783] A "virtual character" is a virtual character that is customized based on a user's personal information.

[0784] "Daily conversation" is a dialogue between a user and a system on everyday topics.

[0785] "Health status" is information relating to the user's current physical condition and health.

[0786] "Risk of fraud and crime" is a risk assessment of contacts or situations that may involve fraud or crime.

[0787] Dementia is a condition in which abnormalities in memory, judgment, and thinking ability are observed.

[0788] A "user-friendly interface" is a simple and intuitive user interface that even elderly people can easily operate.

[0789] The "emotional state" refers to the user's emotional state, such as joy, anger, sadness, or pleasure.

[0790] "Meal recommendation" refers to suggesting suitable meals based on the user's emotional and health state.

[0791] "Delivery service" is a service that delivers meals selected by the user to their home.

[0792] The present invention is a life support system designed for elderly people living alone, which utilizes emotion recognition technology and health monitoring technology to provide personalized meal recommendations and delivery arrangements. Specific embodiments of the present invention are described below.

[0793] Collection of personal information

[0794] The device collects personal information such as the user's hobbies, past life history, personal preferences, etc. This personal information is input by the user through the device's interface.

[0795] Creating a customized virtual character

[0796] The server analyzes the collected personal information and uses machine learning algorithms to customize the virtual character. The customized virtual character data is sent to the device and displayed on the screen. For example, if a user enters information such as "I love cats and used to be a teacher," the server will generate an avatar in the shape of a gentle cat.

[0797] Reducing feelings of alienation through everyday conversation

[0798] The device periodically receives conversation topics from the server and engages in daily conversations with the user. The server analyzes the content of the user's conversation and generates the next conversation topic and response content. For example, if the user says, "The weather was nice today, so I went for a walk in the park," the server generates data to ask, "How was your walk in the park?"

[0799] Health monitoring and anomaly detection

[0800] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color, and sends the data to a server. The server analyzes the data and checks for any abnormalities in the user's health. If an abnormality is detected, the server generates a warning message and sends it to the device. For example, if the device detects abnormalities such as a trembling voice or pale complexion, the server will display a warning that the user may be in poor health.

[0801] Risk assessment for fraud and crime prevention

[0802] The device monitors the content of calls and messages in real time and sends any content deemed to be risky to the server. The server then uses a database to analyze the content of the message and assess the possibility of fraud or crime. If a risk is confirmed, the server generates an alert message and sends it to the device. For example, if the device detects a message saying, "I received a bill, but I don't recognize it," the server will display a warning saying, "This may be fraud."

[0803] Detecting and responding to suspected dementia

[0804] The device continuously records conversations with the user and sends the data to a server. The server analyzes the data and checks for signs of dementia. If dementia is suspected, a local doctor or public institution is automatically notified. For example, if the device records content such as "repeating the same story" or "forgetting where home is," the server will suspect early symptoms of dementia and automatically notify a doctor.

[0805] Providing a user-friendly interface

[0806] The server designs a simple and intuitive interface for seniors and sends the design data to the device. The device then displays the interface on the screen, featuring large buttons and voice navigation, and provides voice or text guidance on how to operate it. For example, the interface has large buttons for "talk to avatar" and "health check," making it easy for users to operate.

[0807] Introducing an emotion engine and emotion recognition

[0808] The device and emotion engine use the built-in camera and microphone to collect the user's voice and facial expression data. The emotion engine analyzes this data and recognizes the user's emotions (e.g., joy, anger, sadness, and happiness). The recognized emotion data is sent to the server, which then generates a response and appropriate action. For example, if the emotion engine senses that the user's voice tone sounds "lonely," the server generates a friendly response such as "How are you today? Tell me something," and the device's virtual character conveys this response to the user.

[0809] Meal recommendations and delivery arrangements

[0810] The server evaluates the user's emotional and health states and recommends individual meals based on them. When the user selects a recommended meal, the server arranges for the order to be placed with a delivery service. For example, if the user says, "I've been feeling sad lately," the server will recommend a nutritious meal that is good for both body and mind based on emotion recognition data and health data. The meal selected by the user is delivered to the user's home by a delivery service.

[0811] Example prompt sentence:

[0812] We are developing a food delivery service app that recommends and arranges delivery of optimal meals when the user is sad or in poor health. Specifically, we combine an emotion engine with health monitoring functionality to evaluate the user's emotions and health status in real time and recommend individual meals based on that. This also includes the ability to arrange meal delivery. Please suggest the best way to make meal recommendations and automate delivery arrangements.

[0813] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0814] Step 1:

[0815] The user inputs personal information using the device. The device collects input data such as hobbies, past life history, and personal preferences, and sends this data to the server. The server stores the received data in a database. It also generates a user profile based on this information.

[0816] Input: Personal information entered by the user

[0817] Output: A database containing the collected personal information

[0818] Step 2:

[0819] The server analyzes the collected personal information and customizes the virtual character using a machine learning algorithm. The generated virtual character data is sent to the device. The device displays the customized virtual character on the screen and starts the initial interaction with the user.

[0820] Input: Personal information collected

[0821] Output: Customized virtual character data

[0822] Step 3:

[0823] The device periodically receives conversation topics from the server and initiates daily conversations with the user. When the user converses with the virtual character, the content is converted into text data using voice recognition technology and sent to the server. The server analyzes the received conversation data and generates the next conversation topic and response content.

[0824] Input: Conversation between the user and the virtual character

[0825] Output: Next conversation topic and reply

[0826] Step 4:

[0827] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color in real time and sends the data to a server. The server analyzes the received data and checks for any abnormalities in the user's health. If an abnormality is detected, a warning message is generated and sent to the device. The device then notifies the user of the warning message by displaying it or by voice.

[0828] Input: User's voice tone and facial expression data

[0829] Output: Health analysis results and warning messages

[0830] Step 5:

[0831] The device monitors the content of calls and messages in real time, and if it determines there is a risk, it sends the content to the server. The server analyzes the message content and evaluates the possibility of fraud or crime. If a risk is confirmed, it generates an alert message and sends it to the device. The device then notifies the user of the alert message by displaying it or by voice.

[0832] Input: Call and message content

[0833] Output: Risk assessment results and alert message

[0834] Step 6:

[0835] The device continuously records conversations with the user and sends the data to a server. The server analyzes the received data and checks for signs of dementia. If dementia is suspected, it can automatically notify local doctors and public institutions.

[0836] Input: Continuously recorded conversation

[0837] Output: Evaluation of dementia symptoms and notification to doctors and public authorities

[0838] Step 7:

[0839] The server designs a user-friendly interface and sends the design data to the terminal, which displays the interface on the screen with large buttons and voice navigation, helping the user to operate it easily.

[0840] Input: Design data in a user-friendly interface

[0841] Output: The interface displayed on the device screen.

[0842] Step 8:

[0843] The device and emotion engine use the built-in camera and microphone to collect the user's voice and facial expression data. This data is sent to the emotion engine, which analyzes it and recognizes the user's emotions (e.g., joy, anger, sadness, and happiness). The recognized emotion data is sent to the server, which then generates a response or appropriate action.

[0844] Input: User's voice and facial expression data

[0845] Output: Recognized emotion data and generated responses or actions

[0846] Step 9:

[0847] The server evaluates the user's emotional and health states and recommends personalized meals based on them. The recommended meal data is sent to the terminal. When the user selects a recommended meal, the server arranges an order with a delivery service and sends the order information to the terminal. The terminal displays the delivery information to the user.

[0848] Input: User's emotional and health state

[0849] Output: Recommended meal data and delivery information

[0850] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0851] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0852] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0853] [Second embodiment]

[0854] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0855] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0856] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0857] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0858] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0859] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0860] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0861] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0862] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0863] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0864] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0865] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0866] The present invention is a life support system primarily targeted at elderly people living alone, and is designed to solve various problems faced by elderly people. Specific embodiments for carrying out the present invention are described below.

[0867] 1. Collecting personal information and creating a customized avatar

[0868] User

[0869] During initial registration, users enter personal information into the interface, such as their hobbies, past life history, and personal preferences.

[0870] Terminal

[0871] The terminal collects the information entered by the user and sends it to the server.

[0872] server

[0873] The server analyzes the received personal information and uses databases and machine learning algorithms to customize the appearance and personality of the virtual character.

[0874] Data of a customized virtual character is generated and transmitted to a terminal.

[0875] Terminal

[0876] The terminal displays the received virtual character on the screen and starts an initial conversation with the user.

[0877] Specific examples

[0878] When a user enters information such as "I love dogs and used to be a nurse," the server generates an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on the screen and asks, "Hello. Is there anything I can help you with?"

[0879] 2. Everyday conversation reduces feelings of alienation

[0880] Terminal

[0881] The terminal periodically receives topics for talking to the user from the server.

[0882] Start the conversation with everyday topics such as "How was your day?"

[0883] User

[0884] The user can have everyday conversations with the virtual character, for example, by replying, "The weather was nice today, so I went for a walk in the park."

[0885] Terminal

[0886] The terminal converts the user's response into text data using voice recognition technology and sends it to the server.

[0887] server

[0888] The server analyzes the received conversation content, generates the next conversation topic and reply content, and sends them to the terminal.

[0889] Specific examples

[0890] When a user says, "I went shopping at the local supermarket today," the device converts the information into text and sends it to the server. The server then generates data to ask the next question, "What did you buy at the supermarket?", and sends it to the device.

[0891] 3. Health monitoring and abnormality detection

[0892] Terminal

[0893] The device uses a built-in camera and microphone to monitor the user's tone of voice, facial expression, and other information in real time.

[0894] The monitoring results are sent to the server.

[0895] server

[0896] The server analyzes the received data and detects any abnormalities in health status.

[0897] If an abnormality is detected, a warning message is generated and sent to the terminal.

[0898] Terminal

[0899] The device will notify the user of the warning message on the screen or via voice, and if necessary, will automatically notify public authorities.

[0900] Specific examples

[0901] If the device detects abnormalities such as "a trembling voice" or "a pale complexion," it sends that information to the server. The server suspects "abnormal blood pressure" and displays a warning on the device saying, "You seem unwell. Would you like to call a doctor?"

[0902] 4. Risk assessment to prevent fraud and crime

[0903] Terminal

[0904] The device monitors the content of calls and messages in real time and sends any content deemed risky to the server.

[0905] server

[0906] The server analyzes the content of incoming messages and applies algorithms to determine the risk of fraud or crime.

[0907] If a risk is identified, an alert message is generated and sent to the device.

[0908] Terminal

[0909] The device will notify the user of the alert via visual or audio notification, and if necessary, will automatically notify trusted contacts.

[0910] Specific examples

[0911] The device receives a message saying "You have received a large bill" and sends it to the server. The server determines that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[0912] 5. Detecting and responding to suspected dementia

[0913] Terminal

[0914] The device continuously records the conversation with the user and transmits it to the server.

[0915] server

[0916] The server analyzes the received data and detects patterns that may indicate dementia.

[0917] If there is any suspicion, doctors and public authorities will be automatically notified.

[0918] Specific examples

[0919] The device records behaviors such as "repeating the same story over and over again, even yesterday" and "forgetting where the house is," and sends the records to a server. The server then detects possible early symptoms of dementia and automatically notifies a doctor.

[0920] 6. Providing a user-friendly interface

[0921] server

[0922] The server designs a simple and intuitive interface for seniors and sends it to the device.

[0923] Terminal

[0924] The device uses large buttons and voice navigation to make it easy for users to use.

[0925] Provide necessary operating instructions via voice or text.

[0926] Specific examples

[0927] The device is designed so that users can use it simply by touching the large buttons that display functions such as "talk to avatar" and "health check." Voice guidance is provided to help users navigate the device without any problems.

[0928] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[0929] The processing flow will be explained below.

[0930] Collecting personal information and creating a customized avatar

[0931] Step 1:

[0932] User

[0933] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[0934] Step 2:

[0935] Terminal

[0936] The terminal collects personal information entered by the user and transmits the data to the server.

[0937] Step 3:

[0938] server

[0939] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character optimized for the user.

[0940] Step 4:

[0941] server

[0942] Data of the customized virtual character is generated and transmitted to the terminal.

[0943] Step 5:

[0944] Terminal

[0945] The terminal displays the received virtual character on the screen, and the user can begin the initial interaction.

[0946] Reducing feelings of alienation through everyday conversation

[0947] Step 1:

[0948] Terminal

[0949] The terminal periodically receives topics for talking to the user from the server.

[0950] Step 2:

[0951] Terminal

[0952] Start the conversation with everyday topics such as "How was your day?"

[0953] Step 3:

[0954] User

[0955] The user converses with the virtual character, replying, for example, "I went to the park today."

[0956] Step 4:

[0957] Terminal

[0958] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[0959] Step 5:

[0960] server

[0961] The server analyzes the received conversation content and creates information to generate the next conversation topic and reply content. The created information is sent to the device.

[0962] Step 6:

[0963] Terminal

[0964] The device uses the received information to prepare the next conversation topic.

[0965] Health monitoring and anomaly detection

[0966] Step 1:

[0967] Terminal

[0968] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[0969] Step 2:

[0970] Terminal

[0971] The collected monitoring data is sent to the server.

[0972] Step 3:

[0973] server

[0974] The server analyzes the received data and checks for any abnormalities in the person's health.

[0975] Step 4:

[0976] server

[0977] If an abnormality is detected, a warning message is generated and sent to the terminal.

[0978] Step 5:

[0979] Terminal

[0980] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[0981] Risk assessment for fraud and crime prevention

[0982] Step 1:

[0983] Terminal

[0984] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[0985] Step 2:

[0986] server

[0987] The server analyzes the content of the received messages and assesses the possibility of fraud or criminal activity.

[0988] Step 3:

[0989] server

[0990] If a risk is identified, an alert message is generated and sent to the device.

[0991] Step 4:

[0992] Terminal

[0993] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[0994] Detecting and responding to suspected dementia

[0995] Step 1:

[0996] Terminal

[0997] The device continuously records the conversation with the user and transmits the data to a server.

[0998] Step 2:

[0999] server

[1000] The server analyzes the received data and checks for signs of dementia.

[1001] Step 3:

[1002] server

[1003] If dementia is suspected, local doctors and public institutions will be automatically notified.

[1004] Providing a user-friendly interface

[1005] Step 1:

[1006] server

[1007] Design a simple and intuitive interface for the elderly and send the design data to the device.

[1008] Step 2:

[1009] Terminal

[1010] The device displays a user-friendly interface on the screen with simple buttons, large text, and voice navigation.

[1011] Step 3:

[1012] Terminal

[1013] The system provides voice or text guidance on how to operate the system, helping users to use it easily.

[1014] Example 1

[1015] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1016] Elderly people living alone face a sense of isolation due to a lack of daily conversation, undetected health problems, the risk of becoming involved in fraud or crime, and the difficulty of early detection of dementia. These issues significantly reduce the quality of life and safety of the elderly. Conventional technologies have the difficulty of solving these problems comprehensively and efficiently.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1018] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the personal information of the user using a machine learning algorithm, and means for analyzing daily conversations with the user and reducing the sense of alienation using a natural language processing algorithm. This makes it possible to provide a sense of psychological security through the generation of a virtual character optimized for the individual based on the personal information entered by the user and daily conversations.

[1019] The server also includes a means for monitoring the user's health condition in real time using a built-in camera and microphone and using a data analysis algorithm to detect abnormalities, and a means for monitoring the content of calls and messages in real time and applying a natural language processing algorithm to determine the risk of fraud or crime and issue a warning, thereby enabling early detection of the user's health abnormalities and the risk of fraud or crime and prompting appropriate measures.

[1020] Furthermore, the server includes a means for continuously recording the contents of the user's conversations, automatically notifying local doctors or public institutions if dementia is suspected, and a means for providing a simple and intuitive interface for the elderly, which will enable early detection of dementia, appropriate follow-up, and highly user-friendly system operation.

[1021] "Personal information" refers to information about a user in general, such as the user's hobbies, past life history, personal preferences, etc.

[1022] A "machine learning algorithm" refers to a computational method for learning patterns from data and making predictions or classifications.

[1023] "Virtual character" refers to a virtual person or animal generated on a computer based on a user's personal information.

[1024] "Natural language processing algorithms" refer to algorithms that analyze and understand human language.

[1025] "Built-in camera" refers to a device that is built into a device and is used to capture video.

[1026] A "microphone" refers to a device used to record sound.

[1027] "Data analysis algorithms" refer to methods used to analyze collected data and detect anomalies and patterns.

[1028] "Real-time" refers to processing or reaction occurring immediately in the current time.

[1029] "Risk of fraud or crime" refers to the possibility of being involved in fraudulent or criminal activity.

[1030] "Automatic notification" refers to the ability of the system to automatically notify pre-defined contacts when certain conditions are met.

[1031] The term "elderly" generally refers to people aged 65 and over.

[1032] "Interface" refers to the screen and operating means through which the user interacts with the system.

[1033] "Continuous recording" refers to the consistent collection and storage of data over a period of time.

[1034] "Alienation" refers to the psychological state of feeling isolated from society or community.

[1035] "What is typing?"

[1036] Refers to a device or method for inputting characters.

[1037] The present invention is a life support system for elderly people living alone, designed to solve various problems faced by elderly people. The system generates a customized virtual character using the user's personal information, and performs daily conversations, health management, fraud prevention, and early dementia detection. Specific embodiments are as follows.

[1038] 1. Collecting personal information and creating a customized avatar

[1039] User

[1040] The user uses the interface to input personal information such as their hobbies, past life history, and personal preferences.

[1041] Terminal

[1042] The device collects the information entered by the user and sends it to the server in JSON format or similar.

[1043] server

[1044] The server uses machine learning algorithms (e.g., TensorFlow) to analyze the received personal information and search a database to generate a profile of a virtual character that best suits the user.

[1045] The server transmits the generated avatar data to the terminal.

[1046] Terminal

[1047] The terminal displays the received avatar data on the screen and starts an initial dialogue with the user.

[1048] Specific examples

[1049] If a user enters information like "I love dogs and used to be a nurse," the device sends that information to the server. The server uses a machine learning algorithm to generate an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on its screen and asks, "Hello. Is there anything I can help you with?"

[1050] 2. Everyday conversation reduces feelings of alienation

[1051] Terminal

[1052] The device periodically receives conversation topics from the server, for example, "How was your day?"

[1053] User

[1054] The user can have everyday conversations with the virtual character, replying with things like, "The weather was nice today, so I went for a walk in the park."

[1055] Terminal

[1056] The device converts the user's response into text data using voice recognition technology (e.g., Google Speech-to-Text) and sends it to the server.

[1057] server

[1058] The server analyzes the received conversation content using a natural language processing algorithm (e.g., GPT-3), generates the next conversation topic and response content, and sends them to the device.

[1059] Specific examples

[1060] When a user says, "I went shopping at the local supermarket today," the device uses voice recognition technology to convert the content into text and send it to the server. The server then uses a natural language processing algorithm to generate data to ask the question, "What did you buy at the supermarket?" and sends it to the device.

[1061] 3. Health monitoring and abnormality detection

[1062] Terminal

[1063] The device uses a built-in camera (e.g., a general HD camera) and microphone (e.g., a high-sensitivity microphone) to monitor the user's voice tone, facial expression, etc. in real time.

[1064] Terminal

[1065] The monitoring results are sent to the server.

[1066] server

[1067] The server analyzes the received data and uses algorithms (e.g., TensorFlow) to detect abnormalities in health status.

[1068] server

[1069] If an abnormality is detected, the server generates a warning message and sends it to the terminal.

[1070] Terminal

[1071] The device will notify the user of the warning message on the screen and via voice, and if necessary, will automatically notify public authorities.

[1072] Specific examples

[1073] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to a server. The server then uses a data analysis algorithm to suspect an abnormality in blood pressure and displays a warning on the device saying, "You appear unwell. Would you like to call a doctor?"

[1074] 4. Risk assessment to prevent fraud and crime

[1075] Terminal

[1076] The device monitors the content of calls and messages in real time.

[1077] Terminal

[1078] Any content deemed to be risky is sent to the server.

[1079] server

[1080] The server analyzes incoming messages using natural language processing algorithms to determine the risk of fraud or crime.

[1081] server

[1082] If a risk is identified, an alert message is generated and sent to the device.

[1083] Terminal

[1084] The device will notify the user of alerts visually and audibly, and if necessary, will automatically notify trusted contacts.

[1085] Specific examples

[1086] The device receives a message saying "You have received a large bill" and sends it to the server. The server uses a natural language processing algorithm to determine that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[1087] 5. Detecting and responding to suspected dementia

[1088] Terminal

[1089] The device continuously records the conversation with the user and transmits it to the server.

[1090] server

[1091] The server analyzes the received data and detects patterns that may indicate dementia (e.g., memory loss, inappropriate behavior).

[1092] server

[1093] In case of suspicion, an automatic notification is generated and sent to doctors and public authorities.

[1094] Specific examples

[1095] The device records behaviors such as "repeating the same story over and over again, even yesterday" or "forgetting where home is," and sends the records to a server. The server then uses a machine learning model to detect possible early symptoms of dementia and automatically notify a doctor.

[1096] 6. Providing a user-friendly interface

[1097] server

[1098] The server designs a simple and intuitive interface for seniors and sends it to the device.

[1099] Terminal

[1100] The device uses large buttons and voice navigation to make it easy for users to use.

[1101] Terminal

[1102] Provide necessary operating instructions via voice or text.

[1103] Specific examples

[1104] The device is equipped with large buttons for functions such as "talk to avatar" and "health check," and is designed so that users can use it simply by touching them. Voice guidance is provided to help users navigate the device without any problems.

[1105] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[1106] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1107] Specific processing steps of the system

[1108] 1. Collecting personal information and creating a customized avatar

[1109] Step 1: Enter user personal information

[1110] The user uses the interface to input personal information such as hobbies, past life history, and personal preferences.

[1111] Input: User's hobbies, past life history, personal preferences.

[1112] Output: User's personal information data.

[1113] Step 2: Submit your information

[1114] The device organizes the personal information entered by the user and sends it to the server in JSON format or similar.

[1115] Input: User's personal information data.

[1116] Output: Personal information data in JSON format.

[1117] Step 3: Data analysis and avatar generation

[1118] The server inputs the received personal information into a machine learning algorithm (e.g., TensorFlow) to analyze it, and searches a database to generate a profile of a virtual character that best suits the user.

[1119] Input: Personal information data in JSON format.

[1120] Output: A profile of the generated virtual character.

[1121] Step 4: Send and display your avatar

[1122] The server transmits the generated avatar data to the terminal.

[1123] Input: The profile of the generated virtual character.

[1124] Output: Avatar data sent to the device.

[1125] The terminal displays the received avatar data on the screen and starts an initial dialogue with the user.

[1126] Input: Avatar data sent from the server.

[1127] Output: A virtual character displayed on the screen.

[1128] 2. Everyday conversation reduces feelings of alienation

[1129] Step 1: Receiving conversation topics

[1130] The terminal periodically receives conversation topics from the server.

[1131] Input: Conversation topic data from the server.

[1132] Output: Conversation topic data saved on your device.

[1133] Step 2: Start a conversation

[1134] The device talks to the user about everyday topics such as "How was your day?"

[1135] Input: Conversation topic data.

[1136] Output: Voice or text conversation start.

[1137] Step 3: User response

[1138] The user engages in everyday conversation with the virtual character, replying, "The weather was nice today, so I went for a walk in the park."

[1139] Input: Question from terminal.

[1140] Output: The user's response.

[1141] Step 4: Convert the audio data

[1142] The device converts the user's response into text data using voice recognition technology (e.g., Google Speech-to-Text) and sends it to the server.

[1143] Input: The user's spoken response.

[1144] Output: The response converted to text data.

[1145] Step 5: Analyzing conversation content and generating next topics

[1146] The server analyzes the received conversation content using a natural language processing algorithm (e.g., GPT-3) and generates the next conversation topic and response content.

[1147] Input: The user's response converted into text data.

[1148] Output: Next conversation topic and reply.

[1149] The server transmits the generated data to the terminal.

[1150] Input: Next conversation topic and reply content.

[1151] Output: The next conversation topic and reply sent to your device.

[1152] 3. Health monitoring and abnormality detection

[1153] Step 1: Monitoring your health data

[1154] The device uses a built-in camera (e.g., a general HD camera) and microphone (e.g., a high-sensitivity microphone) to monitor the user's voice tone, facial expression, etc. in real time.

[1155] Input: User's video and audio data.

[1156] Output: Monitored health data.

[1157] Step 2: Sending data

[1158] The terminal transmits the monitoring results to the server.

[1159] Input: Monitored health data.

[1160] Output: Health data sent to the server.

[1161] Step 3: Data analysis and anomaly detection

[1162] The server analyzes the received data and uses algorithms (e.g., TensorFlow) to detect abnormalities in health status.

[1163] Input: Health data received by the server.

[1164] Output: Anomaly detection results.

[1165] Step 4: Generate and send a warning message

[1166] If an abnormality is detected, the server generates a warning message and sends it to the terminal.

[1167] Input: Anomaly detection results.

[1168] Output: Generated warning message data.

[1169] Step 5: Notification of warning messages

[1170] The device will notify the user of the warning message on the screen and via voice, and if necessary, will automatically notify public authorities.

[1171] Input: The alert message data sent by the server.

[1172] Output: Warning notification to the user and automatic notification to public authorities.

[1173] 4. Risk assessment to prevent fraud and crime

[1174] Step 1: Monitor calls and messages

[1175] The device monitors the content of calls and messages in real time.

[1176] Input: User's call and message data.

[1177] Output: Monitoring result data.

[1178] Step 2: Submit risk data

[1179] The device sends any content that is determined to be risky to the server.

[1180] Input: Monitoring result data.

[1181] Output: The risk data sent to the server.

[1182] Step 3: Risk analysis and alert generation

[1183] The server analyzes incoming messages using natural language processing algorithms to determine the risk of fraud or crime.

[1184] Input: Risk data sent to the server.

[1185] Output: Risk assessment result and alert message.

[1186] Step 4: Alert Notification

[1187] The device will notify the user of alerts visually and audibly, and if necessary, will automatically notify trusted contacts.

[1188] Input: The alert message data sent from the server.

[1189] Output: Alert notification to user and automatic notification to trusted contacts.

[1190] 5. Detecting and responding to suspected dementia

[1191] Step 1: Record the conversation

[1192] The device continuously records the conversation with the user and transmits it to the server.

[1193] Input: User conversation data.

[1194] Output: The transcript data sent to the server.

[1195] Step 2: Data analysis and pattern detection

[1196] The server analyzes the received data and detects patterns that may indicate dementia (e.g., memory loss, inappropriate behavior).

[1197] Input: Conversation recording data sent to the server.

[1198] Output: Pattern detection results.

[1199] Step 3: Generate notifications

[1200] The server generates and sends automatic notifications to doctors and public authorities in case of suspicion.

[1201] Input: Pattern detection results.

[1202] Output: Automatic notification to doctors and public authorities.

[1203] 6. Providing a user-friendly interface

[1204] Step 1: Design the interface

[1205] The server designs a simple and intuitive interface for seniors.

[1206] Input: Usability design data.

[1207] Output: The designed interface data.

[1208] Step 2: Send Interface

[1209] The server sends the designed interface to the terminal.

[1210] Input: The designed interface data.

[1211] Output: Interface data sent to the terminal.

[1212] Step 3: View the interface

[1213] The device uses large buttons and voice navigation to make it easy for users to use.

[1214] Input: Interface data sent by the server.

[1215] Output: A user-friendly interface displayed on the screen.

[1216] Step 4: Providing operation guides

[1217] The device will provide necessary operating instructions via voice or text.

[1218] Input: User operation status and requests.

[1219] Output: Voice and text guide.

[1220] (Application example 1)

[1221] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1222] In modern society, elderly people living alone face various challenges in their daily lives. In particular, they need to deal with feelings of loneliness due to a lack of daily conversation, inadequate health monitoring, the risk of becoming a victim of fraud and crime, and the progression of dementia. Another major issue is the lack of dietary suggestions tailored to individual dietary preferences and health conditions. There is a need to develop a system that can solve these issues and provide an environment where elderly people living alone can live with peace of mind.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1224] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the user's personal information, means for analyzing daily conversations with the user to reduce feelings of alienation, means for monitoring the user's health condition and detecting abnormalities, means for assessing the risk of fraud or crime and issuing a warning against suspicious communications, means for automatically notifying local doctors or public institutions if dementia is suspected, means for providing a user-friendly interface, means for suggesting customized dishes based on the user's personal information, and means for suggesting healthy dishes based on the user's health condition. This enables multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[1225] "User's personal information" refers to information necessary for the system to provide personalized services for each user, such as the user's hobbies, preferences, past lifestyle history, health status, and allergy information.

[1226] A "customized virtual character" is a virtual entity that is specially designed based on the user's personal information, is friendly to the user, and is used for daily conversations and providing services.

[1227] "Means for reducing feelings of alienation" refers to techniques and methods for reducing feelings of loneliness and alienation by having a virtual character regularly engage in everyday conversations with the user, giving the user a sense of social connection.

[1228] "Means for monitoring health status and detecting abnormalities" refers to a function that uses technologies such as voice recognition and facial color analysis to continuously monitor the user's health status and issue an alert if an abnormality is detected.

[1229] "Means for determining the risk of fraud or crime and issuing warnings for suspicious communications" refers to technology that monitors the content of calls and messages and issues a warning to users if it is determined that there is a risk of fraud or crime based on predefined keywords or patterns.

[1230] "Means for automatically notifying local doctors and public institutions if dementia is suspected" is a system that analyzes a user's conversation patterns and behavior, and if it detects suspicion of dementia, automatically notifies local doctors and public institutions.

[1231] "Means for providing a user-friendly interface" refers to a user interface design that provides large buttons, voice navigation, simple operation screens, and other features that are generally easy for seniors to use.

[1232] "Means for providing customized food suggestions" refers to technologies and algorithms that provide personalized food suggestions based on a user's personal information (such as preferences and allergy information).

[1233] The "means for suggesting healthy meals" is a mechanism for suggesting healthy meal menus based on the user's health condition and nutritional balance.

[1234] A "generative AI model" is an artificial intelligence algorithm or model used to interact with users and analyze data.

[1235] A "prompt" is a question or introductory text that a generative AI model uses when engaging in dialogue or making suggestions.

[1236] This invention provides a food delivery service as part of a lifestyle assistance system for elderly people living alone. The system generates a customized virtual character based on the user's personal information and reduces the user's sense of alienation through everyday conversations. It also monitors the user's health and provides a sense of security by detecting abnormalities. It also has a function to assess the risk of fraud and crime and automatically notify local doctors and public institutions if dementia is suspected. The system is designed to be easy for anyone to use, providing a user-friendly interface.

[1237] The system mainly consists of the following hardware and software:

[1238] Hardware: Smartphone, built-in camera, microphone

[1239] Software: Speech recognition technology (Google Cloud Speech-to-Text API), machine learning algorithms (Python's scikit-learn library), voice response technology (Google Text-to-Speech API), interface design (React Native)

[1240] 1. Collecting personal information and creating a customized avatar

[1241] The server collects personal information, such as hobbies, past life history, and personal preferences, entered by the user during initial registration. Based on this information, a virtual chef character is generated. For example, based on the information that "I like pasta and have no allergies," a friendly virtual chef is created and sent to the terminal. The terminal begins an initial dialogue with the user by displaying the virtual chef.

[1242] 2. Everyday conversation reduces feelings of alienation

[1243] The device periodically receives topics from the server to talk to the user. Conversations such as "What would you like to eat today?" are automatically generated. Through everyday conversations with the virtual chef, the user can reduce their sense of alienation. For example, if the user replies "I would like to eat salad today," the device converts this into text data and sends it to the server. The server then generates data to ask the question "What did you think of the salad?" in the next conversation.

[1244] 3. Health monitoring and abnormality detection

[1245] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color in real time. This data is sent to a server, and if an abnormality is detected based on the analysis results, a warning message is generated and sent to the device. For example, if a trembling voice or abnormal facial color is detected, a warning message will be sent saying, "You seem unwell. Would you like to call a doctor?"

[1246] 4. Risk assessment to prevent fraud and crime

[1247] The device monitors the content of calls and messages in real time, and if it determines there is a risk, it sends it to the server. The server applies an algorithm to determine the risk of fraud or crime, generates an alert message, and sends it to the device. For example, if a message containing a high bill is received, a warning will be displayed saying, "This may be a scam. Please ignore it."

[1248] 5. Detecting and responding to suspected dementia

[1249] The device continuously records the conversations with the user and sends them to a server. The server analyzes the received data and automatically notifies doctors and public institutions if dementia is suspected. For example, if the user repeats the same story multiple times or forgets where their home is, the server will automatically notify them that they may be experiencing early symptoms of dementia.

[1250] 6. Providing a user-friendly interface

[1251] The device is designed to be easy for users to operate, using large buttons and voice navigation. The server designs a simple and intuitive interface and sends it to the device. For example, it has large buttons such as "Talk to an avatar" and "Health check," which users can use simply by touching them.

[1252] Examples and prompts

[1253] As a specific example of a conversation, if a user says, "What should I eat today?", the virtual chef will suggest, "How about a fresh salad?". If a user also says, "I went shopping at the local supermarket today," the server will generate data to ask, "What did you buy at the supermarket?" in the next conversation.

[1254] Prompt Sentence Examples

[1255] "Is there anything in particular you'd like to eat today?"

[1256] "How are you feeling? Is there anything that's bothering you?"

[1257] "Want to try a new dish?"

[1258] This will enable multifaceted support for the lives of elderly people living alone, providing them with a safe and comfortable life.

[1259] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1260] Step 1:

[1261] The user inputs personal information such as their hobbies, lifestyle history, personal preferences, and allergy information into the device (smartphone). This information is sent to the server via the device. This inputs the user's personal information into the server.

[1262] Step 2:

[1263] After receiving the user's personal information, the server analyzes the data using machine learning algorithms to generate a virtual character suited to the user, whose appearance and personality are customized to suit the user's preferences.

[1264] Step 3:

[1265] The server transmits the data of the generated customized virtual character to the terminal, which receives the data and displays the virtual character on the screen, allowing the user to start interacting with the virtual character.

[1266] Step 4:

[1267] The terminal periodically receives conversation topics from the server. The server creates a prompt and sends it to the terminal. For example, a prompt such as "Is there anything in particular you'd like to eat today?"

[1268] Step 5:

[1269] The user converses with the virtual character, and the user's response (e.g., "I want to eat salad today") is converted into text data using voice recognition technology and sent to the server.

[1270] Step 6:

[1271] The server processes the received conversation content to generate the next conversation topic and suggestions. For example, if a user says they want to eat salad, the next topic generated will be "What did you think of the salad?"

[1272] Step 7:

[1273] The device uses a built-in camera and microphone to monitor the user's voice tone, facial expression, etc. in real time, and the results of this monitoring are sent to the server.

[1274] Step 8:

[1275] The server analyzes the monitoring data and detects abnormalities in health status. If an abnormality is detected, a warning message is generated and sent to the device. For example, if there is a trembling voice or a change in complexion, a warning message will be displayed saying, "You seem unwell. Would you like to contact a doctor?"

[1276] Step 9:

[1277] The device monitors the content of the user's calls and messages in real time, and if it determines that there is a risk of fraud or crime, it sends the content to a server, which then analyzes the data to determine the risk of fraud or crime.

[1278] Step 10:

[1279] If the server detects a fraud or criminal risk, it generates an alert message and sends it to the device, which then displays a warning such as "This may be fraud. Please ignore."

[1280] Step 11:

[1281] The device continuously records conversations with the user and sends them to a server, which analyzes the data and detects patterns that may indicate dementia.

[1282] Step 12:

[1283] If the server suspects dementia, it will automatically notify local doctors and public institutions, for example, saying, "There are suspicions of early symptoms of dementia."

[1284] Step 13:

[1285] The server designs a simple and intuitive interface for the elderly and sends it to the device. The device uses large buttons and voice navigation to make it easy for users to operate. Buttons such as "Talk to an avatar" and "Health check" are provided.

[1286] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1287] The present invention is a life support system for elderly people living alone, and aims to provide more personalized support by recognizing the user's emotions through the combination of an emotion engine. Specific embodiments for carrying out the present invention are described below.

[1288] 1. Collecting personal information and creating a customized avatar

[1289] User

[1290] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[1291] Terminal

[1292] The terminal collects the information entered by the user and sends it to the server.

[1293] server

[1294] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character.

[1295] Terminal

[1296] The customized virtual character data is displayed on the screen, and the user can start the initial interaction.

[1297] Specific examples

[1298] When a user enters information such as "I love cats and used to be a teacher," the server generates an avatar in the shape of a friendly cat and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[1299] 2. Everyday conversation reduces feelings of alienation

[1300] Terminal

[1301] The device periodically receives topics from the server to talk to the user about, starting a conversation with an everyday topic such as "How was your day?"

[1302] User

[1303] The user can have everyday conversations with the virtual character, for example, by replying, "I went shopping at the local supermarket today."

[1304] Terminal

[1305] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[1306] server

[1307] The server analyzes the received conversation content, creates information for generating the next conversation topic and reply content, and sends it to the terminal.

[1308] Specific examples

[1309] When a user says, "The weather was nice today, so I went for a walk in the park," the device converts the content into text and sends it to the server. The server then generates data to ask the question "How was your walk in the park?" in the next conversation and sends it to the device.

[1310] 3. Health monitoring and abnormality detection

[1311] Terminal

[1312] The built-in camera and microphone are used to monitor the user's tone of voice and facial expression in real time, and the collected monitoring data is sent to a server.

[1313] server

[1314] The server analyzes the received data and checks for any abnormalities in the health status. If an abnormality is detected, it generates a warning message and sends it to the device.

[1315] Terminal

[1316] The device will display a warning message to the user on screen or via voice, and if necessary, will automatically notify public authorities or emergency contacts.

[1317] Specific examples

[1318] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to the server. The server suspects the user may be in poor health and displays a warning on the device saying, "You appear to be unwell. Would you like to call a doctor?"

[1319] 4. Risk assessment to prevent fraud and crime

[1320] Terminal

[1321] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[1322] server

[1323] The server analyzes the content of the received message and evaluates the possibility of fraud or crime. If a risk is identified, an alert message is generated and sent to the device.

[1324] Terminal

[1325] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[1326] Specific examples

[1327] The device detects the message "I received a bill but I don't recognize it" and sends it to the server. The server determines that it is likely a scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[1328] 5. Detecting and responding to suspected dementia

[1329] Terminal

[1330] The device continuously records the conversation with the user and transmits the data to a server.

[1331] server

[1332] The server analyzes the received data and checks for signs of dementia. If dementia is suspected, it automatically notifies local doctors and public institutions.

[1333] Specific examples

[1334] The device records behaviors such as "repeats the same story" and "forgets where home is" and sends them to a server. The server then runs a system that suspects "early symptoms of dementia" and automatically notifies a doctor.

[1335] 6. Providing a user-friendly interface

[1336] server

[1337] Design a simple and intuitive interface for the elderly and send the design data to the device.

[1338] Terminal

[1339] The device displays a user-friendly interface with large buttons and voice navigation, and provides voice or text instructions to help users navigate easily.

[1340] Specific examples

[1341] The device's interface has large buttons for "talk to avatar" and "health check," and is designed to allow users to easily operate it using voice navigation.

[1342] 7. Introduction of Emotion Engine and Emotion Recognition

[1343] Terminal and Emotion Engine

[1344] The device uses a built-in camera and microphone to collect the user's voice and facial expression data, which is then sent to the emotion engine.

[1345] Emotion Engine

[1346] The emotion engine analyzes the received voice and facial expression data to recognize the user's emotions (joy, anger, sadness, happiness, etc.), and sends the recognized emotion data to the server.

[1347] server

[1348] The server generates optimal responses and actions based on the emotional data received from the emotion engine, depending on the user's emotional state.

[1349] Terminal

[1350] The terminal allows the virtual character to speak to the user based on the responses and actions received from the server.

[1351] Specific examples

[1352] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[1353] The processing flow will be explained below.

[1354] Collecting personal information and creating a customized avatar

[1355] Step 1:

[1356] User

[1357] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[1358] Step 2:

[1359] Terminal

[1360] The terminal collects the information entered by the user and sends it to the server.

[1361] Step 3:

[1362] server

[1363] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character.

[1364] Step 4:

[1365] server

[1366] Data of the customized virtual character is generated and transmitted to the terminal.

[1367] Step 5:

[1368] Terminal

[1369] A customized virtual character is displayed on the screen, and the user can initiate an initial interaction with this avatar.

[1370] Specific examples

[1371] When a user enters information such as "I love cats and used to be a teacher," the server generates an avatar in the shape of a friendly cat and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[1372] Reducing feelings of alienation through everyday conversation

[1373] Step 1:

[1374] Terminal

[1375] The terminal periodically receives topics for talking to the user from the server.

[1376] Step 2:

[1377] Terminal

[1378] Start the conversation with everyday topics such as "How was your day?"

[1379] Step 3:

[1380] User

[1381] The user can converse with the virtual character, for example, by saying, "I went shopping at the local supermarket today."

[1382] Step 4:

[1383] Terminal

[1384] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[1385] Step 5:

[1386] server

[1387] The server analyzes the received conversation content, creates information for generating the next conversation topic and reply content, and sends it to the terminal.

[1388] Step 6:

[1389] Terminal

[1390] The device uses the received information to prepare the next conversation topic.

[1391] Specific examples

[1392] When a user says, "The weather was nice today, so I went for a walk in the park," the device converts the content into text and sends it to the server. The server then generates data to ask the question "How was your walk in the park?" in the next conversation and sends it to the device.

[1393] Health monitoring and anomaly detection

[1394] Step 1:

[1395] Terminal

[1396] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[1397] Step 2:

[1398] Terminal

[1399] The collected monitoring data is sent to the server.

[1400] Step 3:

[1401] server

[1402] The server analyzes the received data and checks for any abnormalities in the person's health.

[1403] Step 4:

[1404] server

[1405] If an abnormality is detected, a warning message is generated and sent to the terminal.

[1406] Step 5:

[1407] Terminal

[1408] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[1409] Specific examples

[1410] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to the server. The server suspects the user may be in poor health and displays a warning on the device saying, "You appear to be unwell. Would you like to call a doctor?"

[1411] Risk assessment for fraud and crime prevention

[1412] Step 1:

[1413] Terminal

[1414] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[1415] Step 2:

[1416] server

[1417] The server analyzes the content of the received messages and applies algorithms to determine the risk of fraud or crime.

[1418] Step 3:

[1419] server

[1420] If a risk is identified, an alert message is generated and sent to the device.

[1421] Step 4:

[1422] Terminal

[1423] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[1424] Specific examples

[1425] The device receives a message saying, "I received a large bill but I don't recognize it," and sends it to the server. The server determines that it is likely a scam and displays a warning on the device saying, "This may be a scam. Please ignore it."

[1426] Detecting and responding to suspected dementia

[1427] Step 1:

[1428] Terminal

[1429] The device continuously records the conversation with the user and transmits the data to a server.

[1430] Step 2:

[1431] server

[1432] The server analyzes the received data and checks for any problems with memory or cognitive ability.

[1433] Step 3:

[1434] server

[1435] If dementia is suspected, doctors and public institutions will be automatically notified.

[1436] Specific examples

[1437] The device records such things as "Recently, the patient has been repeating the same things over and over again" and "Forgetting where home is" and sends the information to a server. The server then runs a system that suspects "early symptoms of dementia" and automatically notifies a doctor.

[1438] Providing a user-friendly interface

[1439] Step 1:

[1440] server

[1441] Design a simple and intuitive interface for the elderly and send the design data to the device.

[1442] Step 2:

[1443] Terminal

[1444] The device displays a user-friendly interface on the screen with large buttons and voice navigation.

[1445] Step 3:

[1446] Terminal

[1447] The system provides voice or text guidance on how to operate the system, helping users to use it easily.

[1448] Specific examples

[1449] The device's interface has large buttons for "talk to avatar" and "health check," and is designed to allow users to easily operate it using voice navigation.

[1450] Introducing an emotion engine and emotion recognition

[1451] Step 1:

[1452] Terminal and Emotion Engine

[1453] The device uses a built-in camera and microphone to collect the user's voice and facial expression data and send it to the emotion engine.

[1454] Step 2:

[1455] Emotion Engine

[1456] The emotion engine analyzes the received voice and facial expression data to recognize the user's emotions (joy, anger, sadness, happiness, etc.), and sends the recognized emotion data to the server.

[1457] Step 3:

[1458] server

[1459] The server generates optimal responses and actions based on the emotional data received from the emotion engine, depending on the user's emotional state.

[1460] Step 4:

[1461] Terminal

[1462] The terminal allows the virtual character to speak to the user based on the responses and actions received from the server.

[1463] Specific examples

[1464] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[1465] Example 2

[1466] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1467] In modern society, the number of elderly people living alone is increasing, resulting in a wide range of problems, including feelings of isolation and alienation, declining health, the risk of fraud and crime, and early detection of dementia. Conventional solutions to these problems are insufficient, and there is a need to provide an environment where elderly people can live with peace of mind. The objective of this invention is to provide a system that comprehensively solves these issues.

[1468] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting personal information of the user; means for generating a customized virtual character based on the personal information of the user; means for converting the user's daily conversation into text data and analyzing it using voice recognition technology; means for monitoring the user's health status and detecting abnormalities; means for determining the risk of fraud or crime and issuing a warning about suspicious communications; means for automatically notifying local doctors or public institutions if dementia is suspected; means for providing a user-friendly interface including large buttons and voice navigation; and means for recognizing the user's emotions using an emotion engine and generating responses and actions according to the user's emotional state. This makes it possible to provide comprehensive support for elderly people to live with peace of mind.

[1469] "Users" refer to the elderly people who use this system.

[1470] "Personal information" refers to data such as a user's hobbies, past life history, and personal preferences.

[1471] "Virtual character" refers to a virtual character generated based on the user's personal information.

[1472] "Speech recognition technology" refers to technology that converts a user's voice into text data.

[1473] "Monitoring" refers to observing a user's health condition and behavior and collecting data.

[1474] "Abnormal" refers to a state in which the user's health condition or behavior is different from normal and requires emergency response.

[1475] "Risk of fraud or crime" refers to the possibility of fraud or criminal activity against a user.

[1476] "Warning" refers to a warning issued to the user when a risk of fraud or crime is detected.

[1477] "Suspected dementia" refers to a state in which the user shows early symptoms or signs of dementia.

[1478] A "user-friendly interface" refers to an intuitive operating screen designed to be easy for seniors to use.

[1479] "Emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[1480] "Responses and actions" refer to responses and actions generated by the server depending on the user's emotional state.

[1481] This invention is a life support system for elderly people living alone, and aims to provide more personalized support by recognizing the user's emotions through the combination of an emotion engine. This system is composed of multiple hardware and software components.

[1482] Hardware and Software

[1483] The system consists of the following main components:

[1484] server

[1485] Database for collecting and analyzing users' personal information

[1486] Machine learning algorithms for generating virtual characters (e.g., Amazon Web Services' SageMaker or Google Cloud's AutoML)

[1487] Voice recognition technology (e.g., Google Cloud's Speech-to-Text API)

[1488] Models for analyzing health status, fraud risk, and dementia symptoms (e.g., TensorFlow and PyTorch)

[1489] Terminal

[1490] An interface for users to enter personal information and record conversations

[1491] Devices that collect audio and video data using built-in cameras and microphones (e.g., tablets and smartphones)

[1492] User-friendly interface with large buttons and voice navigation

[1493] Emotion Engine

[1494] An engine for analyzing the user's voice and facial expressions to recognize their emotional state (e.g., Emotion API)

[1495] Software that works in conjunction with a server to generate responses and actions according to the user's emotional state

[1496] Data processing and calculation

[1497] The system includes the following processes:

[1498] 1. Collection of personal information and avatar generation

[1499] Through the interface, users input personal information such as hobbies, past life history, and personal preferences. The device sends this information to a server, which then uses machine learning algorithms to analyze the personal information and generate a virtual character (avatar) tailored to the user. The generated avatar is then sent to the device, where the user can begin interacting with it.

[1500] 2. Assistance with everyday conversation

[1501] The device periodically receives conversation topics sent from the server and speaks to the user. The user's responses are converted into text data using voice recognition technology and sent to the server. The server then generates the next conversation topic and response content and sends them to the device.

[1502] 3. Health monitoring

[1503] The system uses a built-in camera and microphone to monitor the user's tone of voice and facial expression, and sends the data to a server. The server analyzes the data and, if an abnormality is detected, generates a warning message and sends it to the device.

[1504] 4. Preventing fraud and crime

[1505] The device monitors the user's calls and messages and sends any content deemed risky to the server. The server analyzes the message content and assesses the risk of fraud or crime. If necessary, it generates an alert message and sends it to the device.

[1506] 5. Detection of suspected dementia

[1507] The device continuously records conversations with the user and sends the data to a server, which analyzes the data and automatically notifies local doctors and public institutions if there are signs of dementia.

[1508] 6. User-friendly interface

[1509] The server designs a simple and intuitive interface for seniors and sends the design data to the device, which displays it and provides voice or text instructions on how to operate it.

[1510] 7. Use of Emotion Engines

[1511] The device uses a built-in camera and microphone to collect the user's voice and facial expression and transmits them to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state. The server generates the optimal response or action based on the recognized emotional data and transmits it to the device.

[1512] Specific examples

[1513] When a user enters information such as "I love cats and used to be a teacher," the server analyzes the information, generates a friendly cat avatar, and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[1514] Prompt Sentence Examples

[1515] "Please give me a detailed explanation of the assisted living system for elderly people living alone. Please include specific examples."

[1516] As described above, the present invention can solve each of these problems by providing multifaceted and personalized support to elderly people living alone.

[1517] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1518] Step 1:

[1519] User

[1520] Through the interface, the user inputs personal information such as hobbies, past life history, personal preferences, etc. This provides personal information as input data.

[1521] Step 2:

[1522] Terminal

[1523] The terminal temporarily stores the personal information entered by the user and sends the data to the server. The input is the personal information entered by the user, and the output is the data to be sent to the server.

[1524] Step 3:

[1525] server

[1526] The server analyzes the received personal information and generates a virtual character using a database and machine learning algorithms. The input is the personal information received from the device, and the output is customized virtual character data.

[1527] Step 4:

[1528] Terminal

[1529] It receives data of a customized virtual character and displays it on the screen. Specifically, the user can start interacting with the generated avatar. The input is character data, and the output is display output.

[1530] Step 5:

[1531] Terminal

[1532] The terminal periodically receives conversation topics sent from the server. The input is the topic data sent from the server, and the output is the data for starting the conversation.

[1533] Step 6:

[1534] Terminal

[1535] Based on the topics received from the server, everyday conversations such as "How was your day?" are generated for the user and displayed as conversations from an avatar. The input is topic data, and the output is the display to the user.

[1536] Step 7:

[1537] User

[1538] The user can have everyday conversations with the virtual character, for example, saying, "I went shopping at the local supermarket today." This allows the input to be the user's voice data.

[1539] Step 8:

[1540] Terminal

[1541] Using speech recognition technology, the user's response is converted into text data and sent to the server as a conversation log. Specifically, Google Cloud's Speech-to-Text API is used. The input is the user's voice, and the output is text data sent to the server.

[1542] Step 9:

[1543] server

[1544] The received conversation content is analyzed, and data for generating the next conversation topic and reply content is created and sent to the terminal. The input is the text data to be analyzed, and the output is the next conversation data.

[1545] Step 10:

[1546] Terminal

[1547] The built-in camera and microphone are used to monitor the user's voice tone and facial expression in real time, and the monitoring data is sent to the server. The input is real-time audio and video data, and the output is monitoring data sent to the server.

[1548] Step 11:

[1549] server

[1550] The received data is analyzed to check for any abnormalities in the health status. If an abnormality is detected, a warning message is generated and sent to the device. TensorFlow is used as a specific example of operation. The input is monitoring data, and the output is a warning message.

[1551] Step 12:

[1552] Terminal

[1553] The warning message is displayed on the screen or is output as an audio message to the user. If necessary, public institutions and emergency contacts are automatically notified. The input is the warning message, and the output is the user notification and emergency contact.

[1554] Step 13:

[1555] Terminal

[1556] The content of calls and messages is monitored in real time, and any content deemed to be risky is sent to the server. The input is message data, and the output is risk data sent to the server.

[1557] Step 14:

[1558] server

[1559] It analyzes the content of received messages and evaluates the risk of fraud or crime. If a risk is confirmed, it generates an alert message and sends it to the terminal. The input is the message data to be analyzed, and the output is the alert message.

[1560] Step 15:

[1561] Terminal

[1562] The alert is notified to the user via a visual or audio message, and if necessary, automatically notifies trusted contacts. The input is the alert message, and the output is the user notification and contact notification.

[1563] Step 16:

[1564] Terminal

[1565] It continuously records the conversation with the user and sends the data to the server. The input is the conversation data, and the output is the data sent to the server.

[1566] Step 17:

[1567] server

[1568] The received data is analyzed to check for signs of dementia. If dementia is suspected, local doctors and public institutions are automatically notified. The input is conversation data, and the output is notification data.

[1569] Step 18:

[1570] server

[1571] We design a simple and intuitive interface for the elderly and send the design data to the terminal. The input is the interface design data, and the output is the design data for the terminal.

[1572] Step 19:

[1573] Terminal

[1574] A user-friendly interface with large buttons and voice navigation is displayed on the screen, and operation instructions are provided via voice or text. Input is design data, and output is displayed on the screen.

[1575] Step 20:

[1576] Terminal

[1577] The built-in camera and microphone are used to collect the user's voice and facial expression data and send it to the emotion engine. The input is the voice and facial expression data, and the output is the data sent to the emotion engine.

[1578] Step 21:

[1579] Emotion Engine

[1580] The received voice and facial expression data is analyzed to recognize the user's emotional state. The recognized emotional data is sent to the server. The input is the data to be analyzed, and the output is emotional data.

[1581] Step 22:

[1582] server

[1583] Based on the emotional data received from the emotion engine, it generates the optimal response and action according to the user's emotional state. The input is emotional data, and the output is response and action data.

[1584] Step 23:

[1585] Terminal

[1586] The virtual character speaks to the user based on the responses and actions received from the server. The input is the response and action data, and the output is the display to the user.

[1587] Specific examples of operation

[1588] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[1589] Through these steps, this system provides multifaceted support to elderly people living alone, helping them live with peace of mind.

[1590] (Application example 2)

[1591] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1592] Existing support systems are insufficient to address the issues faced by elderly people living alone, such as feelings of isolation, lack of health management, risk of fraud and crime, and early detection of dementia. In addition to these issues, elderly people living alone often face difficulties in daily food selection and nutritional management. Existing systems do not offer personalized food recommendations based on emotional and health status, or systems that can arrange for food delivery. Therefore, there is a need for a comprehensive support system to help elderly people living alone continue to live safely and healthily.

[1593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1594] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the user's personal information, means for analyzing daily conversations with the user to reduce feelings of alienation, means for monitoring the user's health and detecting abnormalities, means for assessing the risk of fraud and crime and issuing a warning against suspicious communications, means for automatically notifying local doctors and public institutions if dementia is suspected, means for providing a user-friendly interface, means for evaluating the user's emotional state and health and recommending individual meals, and means for ordering the recommended meals from a delivery service. This comprehensively solves the various challenges faced by elderly people living alone in their daily lives, enabling them to live safe and healthy lives.

[1595] "Personal information" refers to information such as the user's hobbies, past life history, and personal preferences.

[1596] A "virtual character" is a virtual character that is customized based on a user's personal information.

[1597] "Daily conversation" is a dialogue between a user and a system on everyday topics.

[1598] "Health status" is information relating to the user's current physical condition and health.

[1599] "Risk of fraud and crime" is a risk assessment of contacts or situations that may involve fraud or crime.

[1600] Dementia is a condition in which abnormalities in memory, judgment, and thinking ability are observed.

[1601] A "user-friendly interface" is a simple and intuitive user interface that even elderly people can easily operate.

[1602] The "emotional state" refers to the user's emotional state, such as joy, anger, sadness, or pleasure.

[1603] "Meal recommendation" refers to suggesting suitable meals based on the user's emotional and health state.

[1604] "Delivery service" is a service that delivers meals selected by the user to their home.

[1605] The present invention is a life support system designed for elderly people living alone, which utilizes emotion recognition technology and health monitoring technology to provide personalized meal recommendations and delivery arrangements. Specific embodiments of the present invention are described below.

[1606] Collection of personal information

[1607] The device collects personal information such as the user's hobbies, past life history, personal preferences, etc. This personal information is input by the user through the device's interface.

[1608] Creating a customized virtual character

[1609] The server analyzes the collected personal information and uses machine learning algorithms to customize the virtual character. The customized virtual character data is sent to the device and displayed on the screen. For example, if a user enters information such as "I love cats and used to be a teacher," the server will generate an avatar in the shape of a gentle cat.

[1610] Reducing feelings of alienation through everyday conversation

[1611] The device periodically receives conversation topics from the server and engages in daily conversations with the user. The server analyzes the content of the user's conversation and generates the next conversation topic and response content. For example, if the user says, "The weather was nice today, so I went for a walk in the park," the server generates data to ask, "How was your walk in the park?"

[1612] Health monitoring and anomaly detection

[1613] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color, and sends the data to a server. The server analyzes the data and checks for any abnormalities in the user's health. If an abnormality is detected, the server generates a warning message and sends it to the device. For example, if the device detects abnormalities such as a trembling voice or pale complexion, the server will display a warning that the user may be in poor health.

[1614] Risk assessment for fraud and crime prevention

[1615] The device monitors the content of calls and messages in real time and sends any content deemed to be risky to the server. The server then uses a database to analyze the content of the message and assess the possibility of fraud or crime. If a risk is confirmed, the server generates an alert message and sends it to the device. For example, if the device detects a message saying, "I received a bill, but I don't recognize it," the server will display a warning saying, "This may be fraud."

[1616] Detecting and responding to suspected dementia

[1617] The device continuously records conversations with the user and sends the data to a server. The server analyzes the data and checks for signs of dementia. If dementia is suspected, a local doctor or public institution is automatically notified. For example, if the device records content such as "repeating the same story" or "forgetting where home is," the server will suspect early symptoms of dementia and automatically notify a doctor.

[1618] Providing a user-friendly interface

[1619] The server designs a simple and intuitive interface for seniors and sends the design data to the device. The device then displays the interface on the screen, featuring large buttons and voice navigation, and provides voice or text guidance on how to operate it. For example, the interface has large buttons for "talk to avatar" and "health check," making it easy for users to operate.

[1620] Introducing an emotion engine and emotion recognition

[1621] The device and emotion engine use the built-in camera and microphone to collect the user's voice and facial expression data. The emotion engine analyzes this data and recognizes the user's emotions (e.g., joy, anger, sadness, and happiness). The recognized emotion data is sent to the server, which then generates a response and appropriate action. For example, if the emotion engine senses that the user's voice tone sounds "lonely," the server generates a friendly response such as "How are you today? Tell me something," and the device's virtual character conveys this response to the user.

[1622] Meal recommendations and delivery arrangements

[1623] The server evaluates the user's emotional and health states and recommends individual meals based on them. When the user selects a recommended meal, the server arranges for the order to be placed with a delivery service. For example, if the user says, "I've been feeling sad lately," the server will recommend a nutritious meal that is good for both body and mind based on emotion recognition data and health data. The meal selected by the user is delivered to the user's home by a delivery service.

[1624] Example prompt sentence:

[1625] We are developing a food delivery service app that recommends and arranges delivery of optimal meals when the user is sad or in poor health. Specifically, we combine an emotion engine with health monitoring functionality to evaluate the user's emotions and health status in real time and recommend individual meals based on that. This also includes the ability to arrange meal delivery. Please suggest the best way to make meal recommendations and automate delivery arrangements.

[1626] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1627] Step 1:

[1628] The user inputs personal information using the device. The device collects input data such as hobbies, past life history, and personal preferences, and sends this data to the server. The server stores the received data in a database. It also generates a user profile based on this information.

[1629] Input: Personal information entered by the user

[1630] Output: A database containing the collected personal information

[1631] Step 2:

[1632] The server analyzes the collected personal information and customizes the virtual character using a machine learning algorithm. The generated virtual character data is sent to the device. The device displays the customized virtual character on the screen and starts the initial interaction with the user.

[1633] Input: Personal information collected

[1634] Output: Customized virtual character data

[1635] Step 3:

[1636] The device periodically receives conversation topics from the server and initiates daily conversations with the user. When the user converses with the virtual character, the content is converted into text data using voice recognition technology and sent to the server. The server analyzes the received conversation data and generates the next conversation topic and response content.

[1637] Input: Conversation between the user and the virtual character

[1638] Output: Next conversation topic and reply

[1639] Step 4:

[1640] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color in real time and sends the data to a server. The server analyzes the received data and checks for any abnormalities in the user's health. If an abnormality is detected, a warning message is generated and sent to the device. The device then notifies the user of the warning message by displaying it or by voice.

[1641] Input: User's voice tone and facial expression data

[1642] Output: Health analysis results and warning messages

[1643] Step 5:

[1644] The device monitors the content of calls and messages in real time, and if it determines there is a risk, it sends the content to the server. The server analyzes the message content and evaluates the possibility of fraud or crime. If a risk is confirmed, it generates an alert message and sends it to the device. The device then notifies the user of the alert message by displaying it or by voice.

[1645] Input: Call and message content

[1646] Output: Risk assessment results and alert message

[1647] Step 6:

[1648] The device continuously records conversations with the user and sends the data to a server. The server analyzes the received data and checks for signs of dementia. If dementia is suspected, it can automatically notify local doctors and public institutions.

[1649] Input: Continuously recorded conversation

[1650] Output: Evaluation of dementia symptoms and notification to doctors and public authorities

[1651] Step 7:

[1652] The server designs a user-friendly interface and sends the design data to the terminal, which displays the interface on the screen with large buttons and voice navigation, helping the user to operate it easily.

[1653] Input: Design data in a user-friendly interface

[1654] Output: The interface displayed on the device screen.

[1655] Step 8:

[1656] The device and emotion engine use the built-in camera and microphone to collect the user's voice and facial expression data. This data is sent to the emotion engine, which analyzes it and recognizes the user's emotions (e.g., joy, anger, sadness, and happiness). The recognized emotion data is sent to the server, which then generates a response or appropriate action.

[1657] Input: User's voice and facial expression data

[1658] Output: Recognized emotion data and generated responses or actions

[1659] Step 9:

[1660] The server evaluates the user's emotional and health states and recommends personalized meals based on them. The recommended meal data is sent to the terminal. When the user selects a recommended meal, the server arranges an order with a delivery service and sends the order information to the terminal. The terminal displays the delivery information to the user.

[1661] Input: User's emotional and health state

[1662] Output: Recommended meal data and delivery information

[1663] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1664] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1665] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1666] [Third embodiment]

[1667] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1668] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1669] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1670] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1671] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1672] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1673] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1674] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1675] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1676] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1677] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1678] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1679] The present invention is a life support system primarily targeted at elderly people living alone, and is designed to solve various problems faced by elderly people. Specific embodiments for carrying out the present invention are described below.

[1680] 1. Collecting personal information and creating a customized avatar

[1681] User

[1682] During initial registration, users enter personal information into the interface, such as their hobbies, past life history, and personal preferences.

[1683] Terminal

[1684] The terminal collects the information entered by the user and sends it to the server.

[1685] server

[1686] The server analyzes the received personal information and uses databases and machine learning algorithms to customize the appearance and personality of the virtual character.

[1687] Data of a customized virtual character is generated and transmitted to a terminal.

[1688] Terminal

[1689] The terminal displays the received virtual character on the screen and starts an initial conversation with the user.

[1690] Specific examples

[1691] When a user enters information such as "I love dogs and used to be a nurse," the server generates an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on the screen and asks, "Hello. Is there anything I can help you with?"

[1692] 2. Everyday conversation reduces feelings of alienation

[1693] Terminal

[1694] The terminal periodically receives topics for talking to the user from the server.

[1695] Start the conversation with everyday topics such as "How was your day?"

[1696] User

[1697] The user can have everyday conversations with the virtual character, for example, by replying, "The weather was nice today, so I went for a walk in the park."

[1698] Terminal

[1699] The terminal converts the user's response into text data using voice recognition technology and sends it to the server.

[1700] server

[1701] The server analyzes the received conversation content, generates the next conversation topic and reply content, and sends them to the terminal.

[1702] Specific examples

[1703] When a user says, "I went shopping at the local supermarket today," the device converts the information into text and sends it to the server. The server then generates data to ask the next question, "What did you buy at the supermarket?", and sends it to the device.

[1704] 3. Health monitoring and abnormality detection

[1705] Terminal

[1706] The device uses a built-in camera and microphone to monitor the user's tone of voice, facial expression, and other information in real time.

[1707] The monitoring results are sent to the server.

[1708] server

[1709] The server analyzes the received data and detects any abnormalities in health status.

[1710] If an abnormality is detected, a warning message is generated and sent to the terminal.

[1711] Terminal

[1712] The device will notify the user of the warning message on the screen or via voice, and if necessary, will automatically notify public authorities.

[1713] Specific examples

[1714] If the device detects abnormalities such as "a trembling voice" or "a pale complexion," it sends that information to the server. The server suspects "abnormal blood pressure" and displays a warning on the device saying, "You seem unwell. Would you like to call a doctor?"

[1715] 4. Risk assessment to prevent fraud and crime

[1716] Terminal

[1717] The device monitors the content of calls and messages in real time and sends any content deemed risky to the server.

[1718] server

[1719] The server analyzes the content of incoming messages and applies algorithms to determine the risk of fraud or crime.

[1720] If a risk is identified, an alert message is generated and sent to the device.

[1721] Terminal

[1722] The device will notify the user of the alert via visual or audio notification, and if necessary, will automatically notify trusted contacts.

[1723] Specific examples

[1724] The device receives a message saying "You have received a large bill" and sends it to the server. The server determines that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[1725] 5. Detecting and responding to suspected dementia

[1726] Terminal

[1727] The device continuously records the conversation with the user and transmits it to the server.

[1728] server

[1729] The server analyzes the received data and detects patterns that may indicate dementia.

[1730] If there is any suspicion, doctors and public authorities will be automatically notified.

[1731] Specific examples

[1732] The device records behaviors such as "repeating the same story over and over again, even yesterday" and "forgetting where the house is," and sends the records to a server. The server then detects possible early symptoms of dementia and automatically notifies a doctor.

[1733] 6. Providing a user-friendly interface

[1734] server

[1735] The server designs a simple and intuitive interface for seniors and sends it to the device.

[1736] Terminal

[1737] The device uses large buttons and voice navigation to make it easy for users to use.

[1738] Provide necessary operating instructions via voice or text.

[1739] Specific examples

[1740] The device is designed so that users can use it simply by touching the large buttons that display functions such as "talk to avatar" and "health check." Voice guidance is provided to help users navigate the device without any problems.

[1741] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[1742] The processing flow will be explained below.

[1743] Collecting personal information and creating a customized avatar

[1744] Step 1:

[1745] User

[1746] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[1747] Step 2:

[1748] Terminal

[1749] The terminal collects personal information entered by the user and transmits the data to the server.

[1750] Step 3:

[1751] server

[1752] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character optimized for the user.

[1753] Step 4:

[1754] server

[1755] Data of the customized virtual character is generated and transmitted to the terminal.

[1756] Step 5:

[1757] Terminal

[1758] The terminal displays the received virtual character on the screen, and the user can begin the initial interaction.

[1759] Reducing feelings of alienation through everyday conversation

[1760] Step 1:

[1761] Terminal

[1762] The terminal periodically receives topics for talking to the user from the server.

[1763] Step 2:

[1764] Terminal

[1765] Start the conversation with everyday topics such as "How was your day?"

[1766] Step 3:

[1767] User

[1768] The user converses with the virtual character, replying, for example, "I went to the park today."

[1769] Step 4:

[1770] Terminal

[1771] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[1772] Step 5:

[1773] server

[1774] The server analyzes the received conversation content and creates information to generate the next conversation topic and reply content. The created information is sent to the device.

[1775] Step 6:

[1776] Terminal

[1777] The device uses the received information to prepare the next conversation topic.

[1778] Health monitoring and anomaly detection

[1779] Step 1:

[1780] Terminal

[1781] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[1782] Step 2:

[1783] Terminal

[1784] The collected monitoring data is sent to the server.

[1785] Step 3:

[1786] server

[1787] The server analyzes the received data and checks for any abnormalities in the person's health.

[1788] Step 4:

[1789] server

[1790] If an abnormality is detected, a warning message is generated and sent to the terminal.

[1791] Step 5:

[1792] Terminal

[1793] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[1794] Risk assessment for fraud and crime prevention

[1795] Step 1:

[1796] Terminal

[1797] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[1798] Step 2:

[1799] server

[1800] The server analyzes the content of the received messages and assesses the possibility of fraud or criminal activity.

[1801] Step 3:

[1802] server

[1803] If a risk is identified, an alert message is generated and sent to the device.

[1804] Step 4:

[1805] Terminal

[1806] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[1807] Detecting and responding to suspected dementia

[1808] Step 1:

[1809] Terminal

[1810] The device continuously records the conversation with the user and transmits the data to a server.

[1811] Step 2:

[1812] server

[1813] The server analyzes the received data and checks for signs of dementia.

[1814] Step 3:

[1815] server

[1816] If dementia is suspected, local doctors and public institutions will be automatically notified.

[1817] Providing a user-friendly interface

[1818] Step 1:

[1819] server

[1820] Design a simple and intuitive interface for the elderly and send the design data to the device.

[1821] Step 2:

[1822] Terminal

[1823] The device displays a user-friendly interface on the screen with simple buttons, large text, and voice navigation.

[1824] Step 3:

[1825] Terminal

[1826] The system provides voice or text guidance on how to operate the system, helping users to use it easily.

[1827] Example 1

[1828] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1829] Elderly people living alone face a sense of isolation due to a lack of daily conversation, undetected health problems, the risk of becoming involved in fraud or crime, and the difficulty of early detection of dementia. These issues significantly reduce the quality of life and safety of the elderly. Conventional technologies have the difficulty of solving these problems comprehensively and efficiently.

[1830] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1831] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the personal information of the user using a machine learning algorithm, and means for analyzing daily conversations with the user and reducing the sense of alienation using a natural language processing algorithm. This makes it possible to provide a sense of psychological security through the generation of a virtual character optimized for the individual based on the personal information entered by the user and daily conversations.

[1832] The server also includes a means for monitoring the user's health condition in real time using a built-in camera and microphone and using a data analysis algorithm to detect abnormalities, and a means for monitoring the content of calls and messages in real time and applying a natural language processing algorithm to determine the risk of fraud or crime and issue a warning, thereby enabling early detection of the user's health abnormalities and the risk of fraud or crime and prompting appropriate measures.

[1833] Furthermore, the server includes a means for continuously recording the contents of the user's conversations, automatically notifying local doctors or public institutions if dementia is suspected, and a means for providing a simple and intuitive interface for the elderly, which will enable early detection of dementia, appropriate follow-up, and highly user-friendly system operation.

[1834] "Personal information" refers to information about a user in general, such as the user's hobbies, past life history, personal preferences, etc.

[1835] A "machine learning algorithm" refers to a computational method for learning patterns from data and making predictions or classifications.

[1836] "Virtual character" refers to a virtual person or animal generated on a computer based on a user's personal information.

[1837] "Natural language processing algorithms" refer to algorithms that analyze and understand human language.

[1838] "Built-in camera" refers to a device that is built into a device and is used to capture video.

[1839] A "microphone" refers to a device used to record sound.

[1840] "Data analysis algorithms" refer to methods used to analyze collected data and detect anomalies and patterns.

[1841] "Real-time" refers to processing or reaction occurring immediately in the current time.

[1842] "Risk of fraud or crime" refers to the possibility of being involved in fraudulent or criminal activity.

[1843] "Automatic notification" refers to the ability of the system to automatically notify pre-defined contacts when certain conditions are met.

[1844] The term "elderly" generally refers to people aged 65 and over.

[1845] "Interface" refers to the screen and operating means through which the user interacts with the system.

[1846] "Continuous recording" refers to the consistent collection and storage of data over a period of time.

[1847] "Alienation" refers to the psychological state of feeling isolated from society or community.

[1848] "What is typing?"

[1849] Refers to a device or method for inputting characters.

[1850] The present invention is a life support system for elderly people living alone, designed to solve various problems faced by elderly people. The system generates a customized virtual character using the user's personal information, and performs daily conversations, health management, fraud prevention, and early dementia detection. Specific embodiments are as follows.

[1851] 1. Collecting personal information and creating a customized avatar

[1852] User

[1853] The user uses the interface to input personal information such as their hobbies, past life history, and personal preferences.

[1854] Terminal

[1855] The device collects the information entered by the user and sends it to the server in JSON format or similar.

[1856] server

[1857] The server uses machine learning algorithms (e.g., TensorFlow) to analyze the received personal information and search a database to generate a profile of a virtual character that best suits the user.

[1858] The server transmits the generated avatar data to the terminal.

[1859] Terminal

[1860] The terminal displays the received avatar data on the screen and starts an initial dialogue with the user.

[1861] Specific examples

[1862] If a user enters information like "I love dogs and used to be a nurse," the device sends that information to the server. The server uses a machine learning algorithm to generate an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on its screen and asks, "Hello. Is there anything I can help you with?"

[1863] 2. Everyday conversation reduces feelings of alienation

[1864] Terminal

[1865] The device periodically receives conversation topics from the server, for example, "How was your day?"

[1866] User

[1867] The user can have everyday conversations with the virtual character, replying with things like, "The weather was nice today, so I went for a walk in the park."

[1868] Terminal

[1869] The device converts the user's response into text data using voice recognition technology (e.g., Google Speech-to-Text) and sends it to the server.

[1870] server

[1871] The server analyzes the received conversation content using a natural language processing algorithm (e.g., GPT-3), generates the next conversation topic and response content, and sends them to the device.

[1872] Specific examples

[1873] When a user says, "I went shopping at the local supermarket today," the device uses voice recognition technology to convert the content into text and send it to the server. The server then uses a natural language processing algorithm to generate data to ask the question, "What did you buy at the supermarket?" and sends it to the device.

[1874] 3. Health monitoring and abnormality detection

[1875] Terminal

[1876] The device uses a built-in camera (e.g., a general HD camera) and microphone (e.g., a high-sensitivity microphone) to monitor the user's voice tone, facial expression, etc. in real time.

[1877] Terminal

[1878] The monitoring results are sent to the server.

[1879] server

[1880] The server analyzes the received data and uses algorithms (e.g., TensorFlow) to detect abnormalities in health status.

[1881] server

[1882] If an abnormality is detected, the server generates a warning message and sends it to the terminal.

[1883] Terminal

[1884] The device will notify the user of the warning message on the screen and via voice, and if necessary, will automatically notify public authorities.

[1885] Specific examples

[1886] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to a server. The server then uses a data analysis algorithm to suspect an abnormality in blood pressure and displays a warning on the device saying, "You appear unwell. Would you like to call a doctor?"

[1887] 4. Risk assessment to prevent fraud and crime

[1888] Terminal

[1889] The device monitors the content of calls and messages in real time.

[1890] Terminal

[1891] Any content deemed to be risky is sent to the server.

[1892] server

[1893] The server analyzes incoming messages using natural language processing algorithms to determine the risk of fraud or crime.

[1894] server

[1895] If a risk is identified, an alert message is generated and sent to the device.

[1896] Terminal

[1897] The device will notify the user of alerts visually and audibly, and if necessary, will automatically notify trusted contacts.

[1898] Specific examples

[1899] The device receives a message saying "You have received a large bill" and sends it to the server. The server uses a natural language processing algorithm to determine that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[1900] 5. Detecting and responding to suspected dementia

[1901] Terminal

[1902] The device continuously records the conversation with the user and transmits it to the server.

[1903] server

[1904] The server analyzes the received data and detects patterns that may indicate dementia (e.g., memory loss, inappropriate behavior).

[1905] server

[1906] In case of suspicion, an automatic notification is generated and sent to doctors and public authorities.

[1907] Specific examples

[1908] The device records behaviors such as "repeating the same story over and over again, even yesterday" or "forgetting where home is," and sends the records to a server. The server then uses a machine learning model to detect possible early symptoms of dementia and automatically notify a doctor.

[1909] 6. Providing a user-friendly interface

[1910] server

[1911] The server designs a simple and intuitive interface for seniors and sends it to the device.

[1912] Terminal

[1913] The device uses large buttons and voice navigation to make it easy for users to use.

[1914] Terminal

[1915] Provide necessary operating instructions via voice or text.

[1916] Specific examples

[1917] The device is equipped with large buttons for functions such as "talk to avatar" and "health check," and is designed so that users can use it simply by touching them. Voice guidance is provided to help users navigate the device without any problems.

[1918] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[1919] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1920] Specific processing steps of the system

[1921] 1. Collecting personal information and creating a customized avatar

[1922] Step 1: Enter user personal information

[1923] The user uses the interface to input personal information such as hobbies, past life history, and personal preferences.

[1924] Input: User's hobbies, past life history, personal preferences.

[1925] Output: User's personal information data.

[1926] Step 2: Submit your information

[1927] The device organizes the personal information entered by the user and sends it to the server in JSON format or similar.

[1928] Input: User's personal information data.

[1929] Output: Personal information data in JSON format.

[1930] Step 3: Data analysis and avatar generation

[1931] The server inputs the received personal information into a machine learning algorithm (e.g., TensorFlow) to analyze it, and searches a database to generate a profile of a virtual character that best suits the user.

[1932] Input: Personal information data in JSON format.

[1933] Output: A profile of the generated virtual character.

[1934] Step 4: Send and display your avatar

[1935] The server transmits the generated avatar data to the terminal.

[1936] Input: The profile of the generated virtual character.

[1937] Output: Avatar data sent to the device.

[1938] The terminal displays the received avatar data on the screen and starts an initial dialogue with the user.

[1939] Input: Avatar data sent from the server.

[1940] Output: A virtual character displayed on the screen.

[1941] 2. Everyday conversation reduces feelings of alienation

[1942] Step 1: Receiving conversation topics

[1943] The terminal periodically receives conversation topics from the server.

[1944] Input: Conversation topic data from the server.

[1945] Output: Conversation topic data saved on your device.

[1946] Step 2: Start a conversation

[1947] The device talks to the user about everyday topics such as "How was your day?"

[1948] Input: Conversation topic data.

[1949] Output: Voice or text conversation start.

[1950] Step 3: User response

[1951] The user engages in everyday conversation with the virtual character, replying, "The weather was nice today, so I went for a walk in the park."

[1952] Input: Question from terminal.

[1953] Output: The user's response.

[1954] Step 4: Convert the audio data

[1955] The device converts the user's response into text data using voice recognition technology (e.g., Google Speech-to-Text) and sends it to the server.

[1956] Input: The user's spoken response.

[1957] Output: The response converted to text data.

[1958] Step 5: Analyzing conversation content and generating next topics

[1959] The server analyzes the received conversation content using a natural language processing algorithm (e.g., GPT-3) and generates the next conversation topic and response content.

[1960] Input: The user's response converted into text data.

[1961] Output: Next conversation topic and reply.

[1962] The server transmits the generated data to the terminal.

[1963] Input: Next conversation topic and reply content.

[1964] Output: The next conversation topic and reply sent to your device.

[1965] 3. Health monitoring and abnormality detection

[1966] Step 1: Monitoring your health data

[1967] The device uses a built-in camera (e.g., a general HD camera) and microphone (e.g., a high-sensitivity microphone) to monitor the user's voice tone, facial expression, etc. in real time.

[1968] Input: User's video and audio data.

[1969] Output: Monitored health data.

[1970] Step 2: Sending data

[1971] The terminal transmits the monitoring results to the server.

[1972] Input: Monitored health data.

[1973] Output: Health data sent to the server.

[1974] Step 3: Data analysis and anomaly detection

[1975] The server analyzes the received data and uses algorithms (e.g., TensorFlow) to detect abnormalities in health status.

[1976] Input: Health data received by the server.

[1977] Output: Anomaly detection results.

[1978] Step 4: Generate and send a warning message

[1979] If an abnormality is detected, the server generates a warning message and sends it to the terminal.

[1980] Input: Anomaly detection results.

[1981] Output: Generated warning message data.

[1982] Step 5: Notification of warning messages

[1983] The device will notify the user of the warning message on the screen and via voice, and if necessary, will automatically notify public authorities.

[1984] Input: The alert message data sent by the server.

[1985] Output: Warning notification to the user and automatic notification to public authorities.

[1986] 4. Risk assessment to prevent fraud and crime

[1987] Step 1: Monitor calls and messages

[1988] The device monitors the content of calls and messages in real time.

[1989] Input: User's call and message data.

[1990] Output: Monitoring result data.

[1991] Step 2: Submit risk data

[1992] The device sends any content that is determined to be risky to the server.

[1993] Input: Monitoring result data.

[1994] Output: The risk data sent to the server.

[1995] Step 3: Risk analysis and alert generation

[1996] The server analyzes incoming messages using natural language processing algorithms to determine the risk of fraud or crime.

[1997] Input: Risk data sent to the server.

[1998] Output: Risk assessment result and alert message.

[1999] Step 4: Alert Notification

[2000] The device will notify the user of alerts visually and audibly, and if necessary, will automatically notify trusted contacts.

[2001] Input: The alert message data sent from the server.

[2002] Output: Alert notification to user and automatic notification to trusted contacts.

[2003] 5. Detecting and responding to suspected dementia

[2004] Step 1: Record the conversation

[2005] The device continuously records the conversation with the user and transmits it to the server.

[2006] Input: User conversation data.

[2007] Output: The transcript data sent to the server.

[2008] Step 2: Data analysis and pattern detection

[2009] The server analyzes the received data and detects patterns that may indicate dementia (e.g., memory loss, inappropriate behavior).

[2010] Input: Conversation recording data sent to the server.

[2011] Output: Pattern detection results.

[2012] Step 3: Generate notifications

[2013] The server generates and sends automatic notifications to doctors and public authorities in case of suspicion.

[2014] Input: Pattern detection results.

[2015] Output: Automatic notification to doctors and public authorities.

[2016] 6. Providing a user-friendly interface

[2017] Step 1: Design the interface

[2018] The server designs a simple and intuitive interface for seniors.

[2019] Input: Usability design data.

[2020] Output: The designed interface data.

[2021] Step 2: Send Interface

[2022] The server sends the designed interface to the terminal.

[2023] Input: The designed interface data.

[2024] Output: Interface data sent to the terminal.

[2025] Step 3: View the interface

[2026] The device uses large buttons and voice navigation to make it easy for users to use.

[2027] Input: Interface data sent by the server.

[2028] Output: A user-friendly interface displayed on the screen.

[2029] Step 4: Providing operation guides

[2030] The device will provide necessary operating instructions via voice or text.

[2031] Input: User operation status and requests.

[2032] Output: Voice and text guide.

[2033] (Application example 1)

[2034] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2035] In modern society, elderly people living alone face various challenges in their daily lives. In particular, they need to deal with feelings of loneliness due to a lack of daily conversation, inadequate health monitoring, the risk of becoming a victim of fraud and crime, and the progression of dementia. Another major issue is the lack of dietary suggestions tailored to individual dietary preferences and health conditions. There is a need to develop a system that can solve these issues and provide an environment where elderly people living alone can live with peace of mind.

[2036] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2037] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the user's personal information, means for analyzing daily conversations with the user to reduce feelings of alienation, means for monitoring the user's health condition and detecting abnormalities, means for assessing the risk of fraud or crime and issuing a warning against suspicious communications, means for automatically notifying local doctors or public institutions if dementia is suspected, means for providing a user-friendly interface, means for suggesting customized dishes based on the user's personal information, and means for suggesting healthy dishes based on the user's health condition. This enables multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[2038] "User's personal information" refers to information necessary for the system to provide personalized services for each user, such as the user's hobbies, preferences, past lifestyle history, health status, and allergy information.

[2039] A "customized virtual character" is a virtual entity that is specially designed based on the user's personal information, is friendly to the user, and is used for daily conversations and providing services.

[2040] "Means for reducing feelings of alienation" refers to techniques and methods for reducing feelings of loneliness and alienation by having a virtual character regularly engage in everyday conversations with the user, giving the user a sense of social connection.

[2041] "Means for monitoring health status and detecting abnormalities" refers to a function that uses technologies such as voice recognition and facial color analysis to continuously monitor the user's health status and issue an alert if an abnormality is detected.

[2042] "Means for determining the risk of fraud or crime and issuing warnings for suspicious communications" refers to technology that monitors the content of calls and messages and issues a warning to users if it is determined that there is a risk of fraud or crime based on predefined keywords or patterns.

[2043] "Means for automatically notifying local doctors and public institutions if dementia is suspected" is a system that analyzes a user's conversation patterns and behavior, and if it detects suspicion of dementia, automatically notifies local doctors and public institutions.

[2044] "Means for providing a user-friendly interface" refers to a user interface design that provides large buttons, voice navigation, simple operation screens, and other features that are generally easy for seniors to use.

[2045] "Means for providing customized food suggestions" refers to technologies and algorithms that provide personalized food suggestions based on a user's personal information (such as preferences and allergy information).

[2046] The "means for suggesting healthy meals" is a mechanism for suggesting healthy meal menus based on the user's health condition and nutritional balance.

[2047] A "generative AI model" is an artificial intelligence algorithm or model used to interact with users and analyze data.

[2048] A "prompt" is a question or introductory text that a generative AI model uses when engaging in dialogue or making suggestions.

[2049] This invention provides a food delivery service as part of a lifestyle assistance system for elderly people living alone. The system generates a customized virtual character based on the user's personal information and reduces the user's sense of alienation through everyday conversations. It also monitors the user's health and provides a sense of security by detecting abnormalities. It also has a function to assess the risk of fraud and crime and automatically notify local doctors and public institutions if dementia is suspected. The system is designed to be easy for anyone to use, providing a user-friendly interface.

[2050] The system mainly consists of the following hardware and software:

[2051] Hardware: Smartphone, built-in camera, microphone

[2052] Software: Speech recognition technology (Google Cloud Speech-to-Text API), machine learning algorithms (Python's scikit-learn library), voice response technology (Google Text-to-Speech API), interface design (React Native)

[2053] 1. Collecting personal information and creating a customized avatar

[2054] The server collects personal information, such as hobbies, past life history, and personal preferences, entered by the user during initial registration. Based on this information, a virtual chef character is generated. For example, based on the information that "I like pasta and have no allergies," a friendly virtual chef is created and sent to the terminal. The terminal begins an initial dialogue with the user by displaying the virtual chef.

[2055] 2. Everyday conversation reduces feelings of alienation

[2056] The device periodically receives topics from the server to talk to the user. Conversations such as "What would you like to eat today?" are automatically generated. Through everyday conversations with the virtual chef, the user can reduce their sense of alienation. For example, if the user replies "I would like to eat salad today," the device converts this into text data and sends it to the server. The server then generates data to ask the question "What did you think of the salad?" in the next conversation.

[2057] 3. Health monitoring and abnormality detection

[2058] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color in real time. This data is sent to a server, and if an abnormality is detected based on the analysis results, a warning message is generated and sent to the device. For example, if a trembling voice or abnormal facial color is detected, a warning message will be sent saying, "You seem unwell. Would you like to call a doctor?"

[2059] 4. Risk assessment to prevent fraud and crime

[2060] The device monitors the content of calls and messages in real time, and if it determines there is a risk, it sends it to the server. The server applies an algorithm to determine the risk of fraud or crime, generates an alert message, and sends it to the device. For example, if a message containing a high bill is received, a warning will be displayed saying, "This may be a scam. Please ignore it."

[2061] 5. Detecting and responding to suspected dementia

[2062] The device continuously records the conversations with the user and sends them to a server. The server analyzes the received data and automatically notifies doctors and public institutions if dementia is suspected. For example, if the user repeats the same story multiple times or forgets where their home is, the server will automatically notify them that they may be experiencing early symptoms of dementia.

[2063] 6. Providing a user-friendly interface

[2064] The device is designed to be easy for users to operate, using large buttons and voice navigation. The server designs a simple and intuitive interface and sends it to the device. For example, it has large buttons such as "Talk to an avatar" and "Health check," which users can use simply by touching them.

[2065] Examples and prompts

[2066] As a specific example of a conversation, if a user says, "What should I eat today?", the virtual chef will suggest, "How about a fresh salad?". If a user also says, "I went shopping at the local supermarket today," the server will generate data to ask, "What did you buy at the supermarket?" in the next conversation.

[2067] Prompt Sentence Examples

[2068] "Is there anything in particular you'd like to eat today?"

[2069] "How are you feeling? Is there anything that's bothering you?"

[2070] "Want to try a new dish?"

[2071] This will enable multifaceted support for the lives of elderly people living alone, providing them with a safe and comfortable life.

[2072] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2073] Step 1:

[2074] The user inputs personal information such as their hobbies, lifestyle history, personal preferences, and allergy information into the device (smartphone). This information is sent to the server via the device. This inputs the user's personal information into the server.

[2075] Step 2:

[2076] After receiving the user's personal information, the server analyzes the data using machine learning algorithms to generate a virtual character suited to the user, whose appearance and personality are customized to suit the user's preferences.

[2077] Step 3:

[2078] The server transmits the data of the generated customized virtual character to the terminal, which receives the data and displays the virtual character on the screen, allowing the user to start interacting with the virtual character.

[2079] Step 4:

[2080] The terminal periodically receives conversation topics from the server. The server creates a prompt and sends it to the terminal. For example, a prompt such as "Is there anything in particular you'd like to eat today?"

[2081] Step 5:

[2082] The user converses with the virtual character, and the user's response (e.g., "I want to eat salad today") is converted into text data using voice recognition technology and sent to the server.

[2083] Step 6:

[2084] The server processes the received conversation content to generate the next conversation topic and suggestions. For example, if a user says they want to eat salad, the next topic generated will be "What did you think of the salad?"

[2085] Step 7:

[2086] The device uses a built-in camera and microphone to monitor the user's voice tone, facial expression, etc. in real time, and the results of this monitoring are sent to the server.

[2087] Step 8:

[2088] The server analyzes the monitoring data and detects abnormalities in health status. If an abnormality is detected, a warning message is generated and sent to the device. For example, if there is a trembling voice or a change in complexion, a warning message will be displayed saying, "You seem unwell. Would you like to contact a doctor?"

[2089] Step 9:

[2090] The device monitors the content of the user's calls and messages in real time, and if it determines that there is a risk of fraud or crime, it sends the content to a server, which then analyzes the data to determine the risk of fraud or crime.

[2091] Step 10:

[2092] If the server detects a fraud or criminal risk, it generates an alert message and sends it to the device, which then displays a warning such as "This may be fraud. Please ignore."

[2093] Step 11:

[2094] The device continuously records conversations with the user and sends them to a server, which analyzes the data and detects patterns that may indicate dementia.

[2095] Step 12:

[2096] If the server suspects dementia, it will automatically notify local doctors and public institutions, for example, saying, "There are suspicions of early symptoms of dementia."

[2097] Step 13:

[2098] The server designs a simple and intuitive interface for the elderly and sends it to the device. The device uses large buttons and voice navigation to make it easy for users to operate. Buttons such as "Talk to an avatar" and "Health check" are provided.

[2099] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2100] The present invention is a life support system for elderly people living alone, and aims to provide more personalized support by recognizing the user's emotions through the combination of an emotion engine. Specific embodiments for carrying out the present invention are described below.

[2101] 1. Collecting personal information and creating a customized avatar

[2102] User

[2103] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[2104] Terminal

[2105] The terminal collects the information entered by the user and sends it to the server.

[2106] server

[2107] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character.

[2108] Terminal

[2109] The customized virtual character data is displayed on the screen, and the user can start the initial interaction.

[2110] Specific examples

[2111] When a user enters information such as "I love cats and used to be a teacher," the server generates an avatar in the shape of a friendly cat and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[2112] 2. Everyday conversation reduces feelings of alienation

[2113] Terminal

[2114] The device periodically receives topics from the server to talk to the user about, starting a conversation with an everyday topic such as "How was your day?"

[2115] User

[2116] The user can have everyday conversations with the virtual character, for example, by replying, "I went shopping at the local supermarket today."

[2117] Terminal

[2118] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[2119] server

[2120] The server analyzes the received conversation content, creates information for generating the next conversation topic and reply content, and sends it to the terminal.

[2121] Specific examples

[2122] When a user says, "The weather was nice today, so I went for a walk in the park," the device converts the content into text and sends it to the server. The server then generates data to ask the question "How was your walk in the park?" in the next conversation and sends it to the device.

[2123] 3. Health monitoring and abnormality detection

[2124] Terminal

[2125] The built-in camera and microphone are used to monitor the user's tone of voice and facial expression in real time, and the collected monitoring data is sent to a server.

[2126] server

[2127] The server analyzes the received data and checks for any abnormalities in the health status. If an abnormality is detected, it generates a warning message and sends it to the device.

[2128] Terminal

[2129] The device will display a warning message to the user on screen or via voice, and if necessary, will automatically notify public authorities or emergency contacts.

[2130] Specific examples

[2131] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to the server. The server suspects the user may be in poor health and displays a warning on the device saying, "You appear to be unwell. Would you like to call a doctor?"

[2132] 4. Risk assessment to prevent fraud and crime

[2133] Terminal

[2134] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[2135] server

[2136] The server analyzes the content of the received message and evaluates the possibility of fraud or crime. If a risk is identified, an alert message is generated and sent to the device.

[2137] Terminal

[2138] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[2139] Specific examples

[2140] The device detects the message "I received a bill but I don't recognize it" and sends it to the server. The server determines that it is likely a scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[2141] 5. Detecting and responding to suspected dementia

[2142] Terminal

[2143] The device continuously records the conversation with the user and transmits the data to a server.

[2144] server

[2145] The server analyzes the received data and checks for signs of dementia. If dementia is suspected, it automatically notifies local doctors and public institutions.

[2146] Specific examples

[2147] The device records behaviors such as "repeats the same story" and "forgets where home is" and sends them to a server. The server then runs a system that suspects "early symptoms of dementia" and automatically notifies a doctor.

[2148] 6. Providing a user-friendly interface

[2149] server

[2150] Design a simple and intuitive interface for the elderly and send the design data to the device.

[2151] Terminal

[2152] The device displays a user-friendly interface with large buttons and voice navigation, and provides voice or text instructions to help users navigate easily.

[2153] Specific examples

[2154] The device's interface has large buttons for "talk to avatar" and "health check," and is designed to allow users to easily operate it using voice navigation.

[2155] 7. Introduction of Emotion Engine and Emotion Recognition

[2156] Terminal and Emotion Engine

[2157] The device uses a built-in camera and microphone to collect the user's voice and facial expression data, which is then sent to the emotion engine.

[2158] Emotion Engine

[2159] The emotion engine analyzes the received voice and facial expression data to recognize the user's emotions (joy, anger, sadness, happiness, etc.), and sends the recognized emotion data to the server.

[2160] server

[2161] The server generates optimal responses and actions based on the emotional data received from the emotion engine, depending on the user's emotional state.

[2162] Terminal

[2163] The terminal allows the virtual character to speak to the user based on the responses and actions received from the server.

[2164] Specific examples

[2165] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[2166] The processing flow will be explained below.

[2167] Collecting personal information and creating a customized avatar

[2168] Step 1:

[2169] User

[2170] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[2171] Step 2:

[2172] Terminal

[2173] The terminal collects the information entered by the user and sends it to the server.

[2174] Step 3:

[2175] server

[2176] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character.

[2177] Step 4:

[2178] server

[2179] Data of the customized virtual character is generated and transmitted to the terminal.

[2180] Step 5:

[2181] Terminal

[2182] A customized virtual character is displayed on the screen, and the user can initiate an initial interaction with this avatar.

[2183] Specific examples

[2184] When a user enters information such as "I love cats and used to be a teacher," the server generates an avatar in the shape of a friendly cat and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[2185] Reducing feelings of alienation through everyday conversation

[2186] Step 1:

[2187] Terminal

[2188] The terminal periodically receives topics for talking to the user from the server.

[2189] Step 2:

[2190] Terminal

[2191] Start the conversation with everyday topics such as "How was your day?"

[2192] Step 3:

[2193] User

[2194] The user can converse with the virtual character, for example, by saying, "I went shopping at the local supermarket today."

[2195] Step 4:

[2196] Terminal

[2197] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[2198] Step 5:

[2199] server

[2200] The server analyzes the received conversation content, creates information for generating the next conversation topic and reply content, and sends it to the terminal.

[2201] Step 6:

[2202] Terminal

[2203] The device uses the received information to prepare the next conversation topic.

[2204] Specific examples

[2205] When a user says, "The weather was nice today, so I went for a walk in the park," the device converts the content into text and sends it to the server. The server then generates data to ask the question "How was your walk in the park?" in the next conversation and sends it to the device.

[2206] Health monitoring and anomaly detection

[2207] Step 1:

[2208] Terminal

[2209] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[2210] Step 2:

[2211] Terminal

[2212] The collected monitoring data is sent to the server.

[2213] Step 3:

[2214] server

[2215] The server analyzes the received data and checks for any abnormalities in the person's health.

[2216] Step 4:

[2217] server

[2218] If an abnormality is detected, a warning message is generated and sent to the terminal.

[2219] Step 5:

[2220] Terminal

[2221] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[2222] Specific examples

[2223] If the device detects abnormalities such as a trembling voice or pale complexion, it sends that information to the server. The server suspects the user may be in poor health and displays a warning on the device saying, "You appear to be unwell. Would you like to call a doctor?"

[2224] Risk assessment for fraud and crime prevention

[2225] Step 1:

[2226] Terminal

[2227] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[2228] Step 2:

[2229] server

[2230] The server analyzes the content of the received messages and applies algorithms to determine the risk of fraud or crime.

[2231] Step 3:

[2232] server

[2233] If a risk is identified, an alert message is generated and sent to the device.

[2234] Step 4:

[2235] Terminal

[2236] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[2237] Specific examples

[2238] The device receives a message saying, "I received a large bill but I don't recognize it," and sends it to the server. The server determines that it is likely a scam and displays a warning on the device saying, "This may be a scam. Please ignore it."

[2239] Detecting and responding to suspected dementia

[2240] Step 1:

[2241] Terminal

[2242] The device continuously records the conversation with the user and transmits the data to a server.

[2243] Step 2:

[2244] server

[2245] The server analyzes the received data and checks for any problems with memory or cognitive ability.

[2246] Step 3:

[2247] server

[2248] If dementia is suspected, doctors and public institutions will be automatically notified.

[2249] Specific examples

[2250] The device records such things as "Recently, the patient has been repeating the same things over and over again" and "Forgetting where home is" and sends the information to a server. The server then runs a system that suspects "early symptoms of dementia" and automatically notifies a doctor.

[2251] Providing a user-friendly interface

[2252] Step 1:

[2253] server

[2254] Design a simple and intuitive interface for the elderly and send the design data to the device.

[2255] Step 2:

[2256] Terminal

[2257] The device displays a user-friendly interface on the screen with large buttons and voice navigation.

[2258] Step 3:

[2259] Terminal

[2260] The system provides voice or text guidance on how to operate the system, helping users to use it easily.

[2261] Specific examples

[2262] The device's interface has large buttons for "talk to avatar" and "health check," and is designed to allow users to easily operate it using voice navigation.

[2263] Introducing an emotion engine and emotion recognition

[2264] Step 1:

[2265] Terminal and Emotion Engine

[2266] The device uses a built-in camera and microphone to collect the user's voice and facial expression data and send it to the emotion engine.

[2267] Step 2:

[2268] Emotion Engine

[2269] The emotion engine analyzes the received voice and facial expression data to recognize the user's emotions (joy, anger, sadness, happiness, etc.), and sends the recognized emotion data to the server.

[2270] Step 3:

[2271] server

[2272] The server generates optimal responses and actions based on the emotional data received from the emotion engine, depending on the user's emotional state.

[2273] Step 4:

[2274] Terminal

[2275] The terminal allows the virtual character to speak to the user based on the responses and actions received from the server.

[2276] Specific examples

[2277] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[2278] Example 2

[2279] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2280] In modern society, the number of elderly people living alone is increasing, resulting in a wide range of problems, including feelings of isolation and alienation, declining health, the risk of fraud and crime, and early detection of dementia. Conventional solutions to these problems are insufficient, and there is a need to provide an environment where elderly people can live with peace of mind. The objective of this invention is to provide a system that comprehensively solves these issues.

[2281] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting personal information of the user; means for generating a customized virtual character based on the personal information of the user; means for converting the user's daily conversation into text data and analyzing it using voice recognition technology; means for monitoring the user's health status and detecting abnormalities; means for determining the risk of fraud or crime and issuing a warning about suspicious communications; means for automatically notifying local doctors or public institutions if dementia is suspected; means for providing a user-friendly interface including large buttons and voice navigation; and means for recognizing the user's emotions using an emotion engine and generating responses and actions according to the user's emotional state. This makes it possible to provide comprehensive support for elderly people to live with peace of mind.

[2282] "Users" refer to the elderly people who use this system.

[2283] "Personal information" refers to data such as a user's hobbies, past life history, and personal preferences.

[2284] "Virtual character" refers to a virtual character generated based on the user's personal information.

[2285] "Speech recognition technology" refers to technology that converts a user's voice into text data.

[2286] "Monitoring" refers to observing a user's health condition and behavior and collecting data.

[2287] "Abnormal" refers to a state in which the user's health condition or behavior is different from normal and requires emergency response.

[2288] "Risk of fraud or crime" refers to the possibility of fraud or criminal activity against a user.

[2289] "Warning" refers to a warning issued to the user when a risk of fraud or crime is detected.

[2290] "Suspected dementia" refers to a state in which the user shows early symptoms or signs of dementia.

[2291] A "user-friendly interface" refers to an intuitive operating screen designed to be easy for seniors to use.

[2292] "Emotion engine" refers to technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[2293] "Responses and actions" refer to responses and actions generated by the server depending on the user's emotional state.

[2294] This invention is a life support system for elderly people living alone, and aims to provide more personalized support by recognizing the user's emotions through the combination of an emotion engine. This system is composed of multiple hardware and software components.

[2295] Hardware and Software

[2296] The system consists of the following main components:

[2297] server

[2298] Database for collecting and analyzing users' personal information

[2299] Machine learning algorithms for generating virtual characters (e.g., Amazon Web Services' SageMaker or Google Cloud's AutoML)

[2300] Voice recognition technology (e.g., Google Cloud's Speech-to-Text API)

[2301] Models for analyzing health status, fraud risk, and dementia symptoms (e.g., TensorFlow and PyTorch)

[2302] Terminal

[2303] An interface for users to enter personal information and record conversations

[2304] Devices that collect audio and video data using built-in cameras and microphones (e.g., tablets and smartphones)

[2305] User-friendly interface with large buttons and voice navigation

[2306] Emotion Engine

[2307] An engine for analyzing the user's voice and facial expressions to recognize their emotional state (e.g., Emotion API)

[2308] Software that works in conjunction with a server to generate responses and actions according to the user's emotional state

[2309] Data processing and calculation

[2310] The system includes the following processes:

[2311] 1. Collection of personal information and avatar generation

[2312] Through the interface, users input personal information such as hobbies, past life history, and personal preferences. The device sends this information to a server, which then uses machine learning algorithms to analyze the personal information and generate a virtual character (avatar) tailored to the user. The generated avatar is then sent to the device, where the user can begin interacting with it.

[2313] 2. Assistance with everyday conversation

[2314] The device periodically receives conversation topics sent from the server and speaks to the user. The user's responses are converted into text data using voice recognition technology and sent to the server. The server then generates the next conversation topic and response content and sends them to the device.

[2315] 3. Health monitoring

[2316] The system uses a built-in camera and microphone to monitor the user's tone of voice and facial expression, and sends the data to a server. The server analyzes the data and, if an abnormality is detected, generates a warning message and sends it to the device.

[2317] 4. Preventing fraud and crime

[2318] The device monitors the user's calls and messages and sends any content deemed risky to the server. The server analyzes the message content and assesses the risk of fraud or crime. If necessary, it generates an alert message and sends it to the device.

[2319] 5. Detection of suspected dementia

[2320] The device continuously records conversations with the user and sends the data to a server, which analyzes the data and automatically notifies local doctors and public institutions if there are signs of dementia.

[2321] 6. User-friendly interface

[2322] The server designs a simple and intuitive interface for seniors and sends the design data to the device, which displays it and provides voice or text instructions on how to operate it.

[2323] 7. Use of Emotion Engines

[2324] The device uses a built-in camera and microphone to collect the user's voice and facial expression and transmits them to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotional state. The server generates the optimal response or action based on the recognized emotional data and transmits it to the device.

[2325] Specific examples

[2326] When a user enters information such as "I love cats and used to be a teacher," the server analyzes the information, generates a friendly cat avatar, and sends it to the device. The device then displays the avatar on the screen and asks, "Hello, how was your day?"

[2327] Prompt Sentence Examples

[2328] "Please give me a detailed explanation of the assisted living system for elderly people living alone. Please include specific examples."

[2329] As described above, the present invention can solve each of these problems by providing multifaceted and personalized support to elderly people living alone.

[2330] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2331] Step 1:

[2332] User

[2333] Through the interface, the user inputs personal information such as hobbies, past life history, personal preferences, etc. This provides personal information as input data.

[2334] Step 2:

[2335] Terminal

[2336] The terminal temporarily stores the personal information entered by the user and sends the data to the server. The input is the personal information entered by the user, and the output is the data to be sent to the server.

[2337] Step 3:

[2338] server

[2339] The server analyzes the received personal information and generates a virtual character using a database and machine learning algorithms. The input is the personal information received from the device, and the output is customized virtual character data.

[2340] Step 4:

[2341] Terminal

[2342] It receives data of a customized virtual character and displays it on the screen. Specifically, the user can start interacting with the generated avatar. The input is character data, and the output is display output.

[2343] Step 5:

[2344] Terminal

[2345] The terminal periodically receives conversation topics sent from the server. The input is the topic data sent from the server, and the output is the data for starting the conversation.

[2346] Step 6:

[2347] Terminal

[2348] Based on the topics received from the server, everyday conversations such as "How was your day?" are generated for the user and displayed as conversations from an avatar. The input is topic data, and the output is the display to the user.

[2349] Step 7:

[2350] User

[2351] The user can have everyday conversations with the virtual character, for example, saying, "I went shopping at the local supermarket today." This allows the input to be the user's voice data.

[2352] Step 8:

[2353] Terminal

[2354] Using speech recognition technology, the user's response is converted into text data and sent to the server as a conversation log. Specifically, Google Cloud's Speech-to-Text API is used. The input is the user's voice, and the output is text data sent to the server.

[2355] Step 9:

[2356] server

[2357] The received conversation content is analyzed, and data for generating the next conversation topic and reply content is created and sent to the terminal. The input is the text data to be analyzed, and the output is the next conversation data.

[2358] Step 10:

[2359] Terminal

[2360] The built-in camera and microphone are used to monitor the user's voice tone and facial expression in real time, and the monitoring data is sent to the server. The input is real-time audio and video data, and the output is monitoring data sent to the server.

[2361] Step 11:

[2362] server

[2363] The received data is analyzed to check for any abnormalities in the health status. If an abnormality is detected, a warning message is generated and sent to the device. TensorFlow is used as a specific example of operation. The input is monitoring data, and the output is a warning message.

[2364] Step 12:

[2365] Terminal

[2366] The warning message is displayed on the screen or is output as an audio message to the user. If necessary, public institutions and emergency contacts are automatically notified. The input is the warning message, and the output is the user notification and emergency contact.

[2367] Step 13:

[2368] Terminal

[2369] The content of calls and messages is monitored in real time, and any content deemed to be risky is sent to the server. The input is message data, and the output is risk data sent to the server.

[2370] Step 14:

[2371] server

[2372] It analyzes the content of received messages and evaluates the risk of fraud or crime. If a risk is confirmed, it generates an alert message and sends it to the terminal. The input is the message data to be analyzed, and the output is the alert message.

[2373] Step 15:

[2374] Terminal

[2375] The alert is notified to the user via a visual or audio message, and if necessary, automatically notifies trusted contacts. The input is the alert message, and the output is the user notification and contact notification.

[2376] Step 16:

[2377] Terminal

[2378] It continuously records the conversation with the user and sends the data to the server. The input is the conversation data, and the output is the data sent to the server.

[2379] Step 17:

[2380] server

[2381] The received data is analyzed to check for signs of dementia. If dementia is suspected, local doctors and public institutions are automatically notified. The input is conversation data, and the output is notification data.

[2382] Step 18:

[2383] server

[2384] We design a simple and intuitive interface for the elderly and send the design data to the terminal. The input is the interface design data, and the output is the design data for the terminal.

[2385] Step 19:

[2386] Terminal

[2387] A user-friendly interface with large buttons and voice navigation is displayed on the screen, and operation instructions are provided via voice or text. Input is design data, and output is displayed on the screen.

[2388] Step 20:

[2389] Terminal

[2390] The built-in camera and microphone are used to collect the user's voice and facial expression data and send it to the emotion engine. The input is the voice and facial expression data, and the output is the data sent to the emotion engine.

[2391] Step 21:

[2392] Emotion Engine

[2393] The received voice and facial expression data is analyzed to recognize the user's emotional state. The recognized emotional data is sent to the server. The input is the data to be analyzed, and the output is emotional data.

[2394] Step 22:

[2395] server

[2396] Based on the emotional data received from the emotion engine, it generates the optimal response and action according to the user's emotional state. The input is emotional data, and the output is response and action data.

[2397] Step 23:

[2398] Terminal

[2399] The virtual character speaks to the user based on the responses and actions received from the server. The input is the response and action data, and the output is the display to the user.

[2400] Specific examples of operation

[2401] If the emotion engine senses that the user's voice tone is "lonely," the server generates a friendly response such as "How are you today? Tell me something," and sends it to the device. The virtual character on the device conveys this response to the user.

[2402] Through these steps, this system provides multifaceted support to elderly people living alone, helping them live with peace of mind.

[2403] (Application example 2)

[2404] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2405] Existing support systems are insufficient to address the issues faced by elderly people living alone, such as feelings of isolation, lack of health management, risk of fraud and crime, and early detection of dementia. In addition to these issues, elderly people living alone often face difficulties in daily food selection and nutritional management. Existing systems do not offer personalized food recommendations based on emotional and health status, or systems that can arrange for food delivery. Therefore, there is a need for a comprehensive support system to help elderly people living alone continue to live safely and healthily.

[2406] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2407] In this invention, the server includes means for collecting personal information of the user, means for generating a customized virtual character based on the user's personal information, means for analyzing daily conversations with the user to reduce feelings of alienation, means for monitoring the user's health and detecting abnormalities, means for assessing the risk of fraud and crime and issuing a warning against suspicious communications, means for automatically notifying local doctors and public institutions if dementia is suspected, means for providing a user-friendly interface, means for evaluating the user's emotional state and health and recommending individual meals, and means for ordering the recommended meals from a delivery service. This comprehensively solves the various challenges faced by elderly people living alone in their daily lives, enabling them to live safe and healthy lives.

[2408] "Personal information" refers to information such as the user's hobbies, past life history, and personal preferences.

[2409] A "virtual character" is a virtual character that is customized based on a user's personal information.

[2410] "Daily conversation" is a dialogue between a user and a system on everyday topics.

[2411] "Health status" is information relating to the user's current physical condition and health.

[2412] "Risk of fraud and crime" is a risk assessment of contacts or situations that may involve fraud or crime.

[2413] Dementia is a condition in which abnormalities in memory, judgment, and thinking ability are observed.

[2414] A "user-friendly interface" is a simple and intuitive user interface that even elderly people can easily operate.

[2415] The "emotional state" refers to the user's emotional state, such as joy, anger, sadness, or pleasure.

[2416] "Meal recommendation" refers to suggesting suitable meals based on the user's emotional and health state.

[2417] "Delivery service" is a service that delivers meals selected by the user to their home.

[2418] The present invention is a life support system designed for elderly people living alone, which utilizes emotion recognition technology and health monitoring technology to provide personalized meal recommendations and delivery arrangements. Specific embodiments of the present invention are described below.

[2419] Collection of personal information

[2420] The device collects personal information such as the user's hobbies, past life history, personal preferences, etc. This personal information is input by the user through the device's interface.

[2421] Creating a customized virtual character

[2422] The server analyzes the collected personal information and uses machine learning algorithms to customize the virtual character. The customized virtual character data is sent to the device and displayed on the screen. For example, if a user enters information such as "I love cats and used to be a teacher," the server will generate an avatar in the shape of a gentle cat.

[2423] Reducing feelings of alienation through everyday conversation

[2424] The device periodically receives conversation topics from the server and engages in daily conversations with the user. The server analyzes the content of the user's conversation and generates the next conversation topic and response content. For example, if the user says, "The weather was nice today, so I went for a walk in the park," the server generates data to ask, "How was your walk in the park?"

[2425] Health monitoring and anomaly detection

[2426] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color, and sends the data to a server. The server analyzes the data and checks for any abnormalities in the user's health. If an abnormality is detected, the server generates a warning message and sends it to the device. For example, if the device detects abnormalities such as a trembling voice or pale complexion, the server will display a warning that the user may be in poor health.

[2427] Risk assessment for fraud and crime prevention

[2428] The device monitors the content of calls and messages in real time and sends any content deemed to be risky to the server. The server then uses a database to analyze the content of the message and assess the possibility of fraud or crime. If a risk is confirmed, the server generates an alert message and sends it to the device. For example, if the device detects a message saying, "I received a bill, but I don't recognize it," the server will display a warning saying, "This may be fraud."

[2429] Detecting and responding to suspected dementia

[2430] The device continuously records conversations with the user and sends the data to a server. The server analyzes the data and checks for signs of dementia. If dementia is suspected, a local doctor or public institution is automatically notified. For example, if the device records content such as "repeating the same story" or "forgetting where home is," the server will suspect early symptoms of dementia and automatically notify a doctor.

[2431] Providing a user-friendly interface

[2432] The server designs a simple and intuitive interface for seniors and sends the design data to the device. The device then displays the interface on the screen, featuring large buttons and voice navigation, and provides voice or text guidance on how to operate it. For example, the interface has large buttons for "talk to avatar" and "health check," making it easy for users to operate.

[2433] Introducing an emotion engine and emotion recognition

[2434] The device and emotion engine use the built-in camera and microphone to collect the user's voice and facial expression data. The emotion engine analyzes this data and recognizes the user's emotions (e.g., joy, anger, sadness, and happiness). The recognized emotion data is sent to the server, which then generates a response and appropriate action. For example, if the emotion engine senses that the user's voice tone sounds "lonely," the server generates a friendly response such as "How are you today? Tell me something," and the device's virtual character conveys this response to the user.

[2435] Meal recommendations and delivery arrangements

[2436] The server evaluates the user's emotional and health states and recommends individual meals based on them. When the user selects a recommended meal, the server arranges for the order to be placed with a delivery service. For example, if the user says, "I've been feeling sad lately," the server will recommend a nutritious meal that is good for both body and mind based on emotion recognition data and health data. The meal selected by the user is delivered to the user's home by a delivery service.

[2437] Example prompt sentence:

[2438] We are developing a food delivery service app that recommends and arranges delivery of optimal meals when the user is sad or in poor health. Specifically, we combine an emotion engine with health monitoring functionality to evaluate the user's emotions and health status in real time and recommend individual meals based on that. This also includes the ability to arrange meal delivery. Please suggest the best way to make meal recommendations and automate delivery arrangements.

[2439] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2440] Step 1:

[2441] The user inputs personal information using the device. The device collects input data such as hobbies, past life history, and personal preferences, and sends this data to the server. The server stores the received data in a database. It also generates a user profile based on this information.

[2442] Input: Personal information entered by the user

[2443] Output: A database containing the collected personal information

[2444] Step 2:

[2445] The server analyzes the collected personal information and customizes the virtual character using a machine learning algorithm. The generated virtual character data is sent to the device. The device displays the customized virtual character on the screen and starts the initial interaction with the user.

[2446] Input: Personal information collected

[2447] Output: Customized virtual character data

[2448] Step 3:

[2449] The device periodically receives conversation topics from the server and initiates daily conversations with the user. When the user converses with the virtual character, the content is converted into text data using voice recognition technology and sent to the server. The server analyzes the received conversation data and generates the next conversation topic and response content.

[2450] Input: Conversation between the user and the virtual character

[2451] Output: Next conversation topic and reply

[2452] Step 4:

[2453] The device uses a built-in camera and microphone to monitor the user's voice tone and facial color in real time and sends the data to a server. The server analyzes the received data and checks for any abnormalities in the user's health. If an abnormality is detected, a warning message is generated and sent to the device. The device then notifies the user of the warning message by displaying it or by voice.

[2454] Input: User's voice tone and facial expression data

[2455] Output: Health analysis results and warning messages

[2456] Step 5:

[2457] The device monitors the content of calls and messages in real time, and if it determines there is a risk, it sends the content to the server. The server analyzes the message content and evaluates the possibility of fraud or crime. If a risk is confirmed, it generates an alert message and sends it to the device. The device then notifies the user of the alert message by displaying it or by voice.

[2458] Input: Call and message content

[2459] Output: Risk assessment results and alert message

[2460] Step 6:

[2461] The device continuously records conversations with the user and sends the data to a server. The server analyzes the received data and checks for signs of dementia. If dementia is suspected, it can automatically notify local doctors and public institutions.

[2462] Input: Continuously recorded conversation

[2463] Output: Evaluation of dementia symptoms and notification to doctors and public authorities

[2464] Step 7:

[2465] The server designs a user-friendly interface and sends the design data to the terminal, which displays the interface on the screen with large buttons and voice navigation, helping the user to operate it easily.

[2466] Input: Design data in a user-friendly interface

[2467] Output: The interface displayed on the device screen.

[2468] Step 8:

[2469] The device and emotion engine use the built-in camera and microphone to collect the user's voice and facial expression data. This data is sent to the emotion engine, which analyzes it and recognizes the user's emotions (e.g., joy, anger, sadness, and happiness). The recognized emotion data is sent to the server, which then generates a response or appropriate action.

[2470] Input: User's voice and facial expression data

[2471] Output: Recognized emotion data and generated responses or actions

[2472] Step 9:

[2473] The server evaluates the user's emotional and health states and recommends personalized meals based on them. The recommended meal data is sent to the terminal. When the user selects a recommended meal, the server arranges an order with a delivery service and sends the order information to the terminal. The terminal displays the delivery information to the user.

[2474] Input: User's emotional and health state

[2475] Output: Recommended meal data and delivery information

[2476] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2478] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2479] [Fourth embodiment]

[2480] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2481] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2482] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2483] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2484] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2485] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2486] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2487] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2488] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2491] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2492] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2493] The present invention is a life support system primarily targeted at elderly people living alone, and is designed to solve various problems faced by elderly people. Specific embodiments for carrying out the present invention are described below.

[2494] 1. Collecting personal information and creating a customized avatar

[2495] User

[2496] During initial registration, users enter personal information into the interface, such as their hobbies, past life history, and personal preferences.

[2497] Terminal

[2498] The terminal collects the information entered by the user and sends it to the server.

[2499] server

[2500] The server analyzes the received personal information and uses databases and machine learning algorithms to customize the appearance and personality of the virtual character.

[2501] Data of a customized virtual character is generated and transmitted to a terminal.

[2502] Terminal

[2503] The terminal displays the received virtual character on the screen and starts an initial conversation with the user.

[2504] Specific examples

[2505] When a user enters information such as "I love dogs and used to be a nurse," the server generates an avatar in the shape of a friendly dog ​​and sends it to the device. The device then displays the avatar on the screen and asks, "Hello. Is there anything I can help you with?"

[2506] 2. Everyday conversation reduces feelings of alienation

[2507] Terminal

[2508] The terminal periodically receives topics for talking to the user from the server.

[2509] Start the conversation with everyday topics such as "How was your day?"

[2510] User

[2511] The user can have everyday conversations with the virtual character, for example, by replying, "The weather was nice today, so I went for a walk in the park."

[2512] Terminal

[2513] The terminal converts the user's response into text data using voice recognition technology and sends it to the server.

[2514] server

[2515] The server analyzes the received conversation content, generates the next conversation topic and reply content, and sends them to the terminal.

[2516] Specific examples

[2517] When a user says, "I went shopping at the local supermarket today," the device converts the information into text and sends it to the server. The server then generates data to ask the next question, "What did you buy at the supermarket?", and sends it to the device.

[2518] 3. Health monitoring and abnormality detection

[2519] Terminal

[2520] The device uses a built-in camera and microphone to monitor the user's tone of voice, facial expression, and other information in real time.

[2521] The monitoring results are sent to the server.

[2522] server

[2523] The server analyzes the received data and detects any abnormalities in health status.

[2524] If an abnormality is detected, a warning message is generated and sent to the terminal.

[2525] Terminal

[2526] The device will notify the user of the warning message on the screen or via voice, and if necessary, will automatically notify public authorities.

[2527] Specific examples

[2528] If the device detects abnormalities such as "a trembling voice" or "a pale complexion," it sends that information to the server. The server suspects "abnormal blood pressure" and displays a warning on the device saying, "You seem unwell. Would you like to call a doctor?"

[2529] 4. Risk assessment to prevent fraud and crime

[2530] Terminal

[2531] The device monitors the content of calls and messages in real time and sends any content deemed risky to the server.

[2532] server

[2533] The server analyzes the content of incoming messages and applies algorithms to determine the risk of fraud or crime.

[2534] If a risk is identified, an alert message is generated and sent to the device.

[2535] Terminal

[2536] The device will notify the user of the alert via visual or audio notification, and if necessary, will automatically notify trusted contacts.

[2537] Specific examples

[2538] The device receives a message saying "You have received a large bill" and sends it to the server. The server determines that this is a possible scam and displays a warning on the device saying "This may be a scam. Please ignore it."

[2539] 5. Detecting and responding to suspected dementia

[2540] Terminal

[2541] The device continuously records the conversation with the user and transmits it to the server.

[2542] server

[2543] The server analyzes the received data and detects patterns that may indicate dementia.

[2544] If there is any suspicion, doctors and public authorities will be automatically notified.

[2545] Specific examples

[2546] The device records behaviors such as "repeating the same story over and over again, even yesterday" and "forgetting where the house is," and sends the records to a server. The server then detects possible early symptoms of dementia and automatically notifies a doctor.

[2547] 6. Providing a user-friendly interface

[2548] server

[2549] The server designs a simple and intuitive interface for seniors and sends it to the device.

[2550] Terminal

[2551] The device uses large buttons and voice navigation to make it easy for users to use.

[2552] Provide necessary operating instructions via voice or text.

[2553] Specific examples

[2554] The device is designed so that users can use it simply by touching the large buttons that display functions such as "talk to avatar" and "health check." Voice guidance is provided to help users navigate the device without any problems.

[2555] By combining the above elements, this system provides multifaceted support for the lives of elderly people living alone, enabling them to live a safe and comfortable life.

[2556] The processing flow will be explained below.

[2557] Collecting personal information and creating a customized avatar

[2558] Step 1:

[2559] User

[2560] Through the interface, users input personal information such as their hobbies, past life history, and personal preferences.

[2561] Step 2:

[2562] Terminal

[2563] The terminal collects personal information entered by the user and transmits the data to the server.

[2564] Step 3:

[2565] server

[2566] The server analyzes the received personal information and uses its accumulated database and machine learning algorithms to customize the appearance and personality of the virtual character optimized for the user.

[2567] Step 4:

[2568] server

[2569] Data of the customized virtual character is generated and transmitted to the terminal.

[2570] Step 5:

[2571] Terminal

[2572] The terminal displays the received virtual character on the screen, and the user can begin the initial interaction.

[2573] Reducing feelings of alienation through everyday conversation

[2574] Step 1:

[2575] Terminal

[2576] The terminal periodically receives topics for talking to the user from the server.

[2577] Step 2:

[2578] Terminal

[2579] Start the conversation with everyday topics such as "How was your day?"

[2580] Step 3:

[2581] User

[2582] The user converses with the virtual character, replying, for example, "I went to the park today."

[2583] Step 4:

[2584] Terminal

[2585] The device converts the user's responses into text data using voice recognition technology and sends it to the server as a conversation log.

[2586] Step 5:

[2587] server

[2588] The server analyzes the received conversation content and creates information to generate the next conversation topic and reply content. The created information is sent to the device.

[2589] Step 6:

[2590] Terminal

[2591] The device uses the received information to prepare the next conversation topic.

[2592] Health monitoring and anomaly detection

[2593] Step 1:

[2594] Terminal

[2595] It uses the built-in camera and microphone to monitor the user's tone of voice and facial expression in real time.

[2596] Step 2:

[2597] Terminal

[2598] The collected monitoring data is sent to the server.

[2599] Step 3:

[2600] server

[2601] The server analyzes the received data and checks for any abnormalities in the person's health.

[2602] Step 4:

[2603] server

[2604] If an abnormality is detected, a warning message is generated and sent to the terminal.

[2605] Step 5:

[2606] Terminal

[2607] The device will display a warning message to the user on screen or via audio, and if necessary, will automatically notify public authorities and emergency contacts.

[2608] Risk assessment for fraud and crime prevention

[2609] Step 1:

[2610] Terminal

[2611] The content of calls and messages is monitored in real time, and any content deemed risky is sent to the server.

[2612] Step 2:

[2613] server

[2614] The server analyzes the content of the received messages and assesses the possibility of fraud or criminal activity.

[2615] Step 3:

[2616] server

[2617] If a risk is identified, an alert message is generated and sent to the device.

[2618] Step 4:

[2619] Terminal

[2620] The device will notify the user of the alert via visual or audio notification, and will automatically notify trusted contacts if necessary.

[2621] Detecting and responding to suspected dementia

[2622] Step 1:

[2623] Terminal

[2624] The ...

Claims

1. A means for collecting personal information of users; means for generating a customized virtual character based on the user's personal information; A means for analyzing daily conversations with users and reducing their sense of alienation; A means for monitoring the health status of a user and detecting abnormalities; A means of assessing fraud and crime risk and issuing warnings about suspicious communications; A means of automatically notifying local doctors and public authorities if dementia is suspected, and a means for providing a user-friendly interface; A system including:

2. 10. The system of claim 1, further comprising means for monitoring changes in the user's voice and facial expression in real time.

3. 10. The system of claim 1, further comprising means for analyzing the database for keywords and patterns to determine fraud and crime risk.

Images