System

Abstract

A system is provided.SOLUTION: The system for promoting daily conversation to the elderly person includes a means for generating a topic based on weather information of the day, news information, and hobbies and preferences of a user and providing the topic to the user by voice, and a means for transmitting a message from a remote place to a terminal by a relative. A system comprising: means for transmitting a message to a user by voice at a designated time; means for retrying a predetermined number of times when there is no response from the user and notifying a relative if there is still no response; means for notifying a relative if there is no conversation for a predetermined period of time; and means for enabling the user to make an emergency call using a voice command or a button in an emergency.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When elderly people live alone or in a facility, opportunities for daily conversation decrease, which can lead to feelings of loneliness and impaired judgment. Furthermore, because distant relatives cannot keep track of the elderly's daily situation, anxiety about their health increases. The present invention aims to improve the quality of life for elderly people by increasing opportunities for conversation and providing support for emergency response and purchasing daily necessities. [Means for solving the problem]

[0005] The present invention is a system for encouraging daily conversation among the elderly. It includes a means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences, and providing these to the user via voice. It also includes a means for a relative to remotely send a message to the terminal and deliver the message to the user via voice at a specified time. Furthermore, if there is no response from the user, the system includes a means for retrying a certain number of times, notifying the relative if there is still no response, and a means for notifying the relative if there is no conversation for a certain period of time. In an emergency, the system also includes a means for the user to make an emergency call using a voice command or a button. Finally, it includes a means for linking with an online shopping site and allowing daily necessities to be purchased using a voice command. These functions can support the daily lives of the elderly and provide a sense of security to relatives.

[0006] The "system" is a set of interconnected devices and software designed to facilitate everyday conversations for older adults and assist them in communicating with their relatives.

[0007] "Weather information" is data about the weather conditions in a target area, including temperature, precipitation, wind speed, and the like.

[0008] "News information" refers to information about current events, particularly the major stories and topics of the day.

[0009] "Hobbies and interests" is information related to activities and things in which a user is particularly interested.

[0010] "Means for providing information by voice" refers to a mechanism that uses voice synthesis technology to convey information prepared in advance to the user in voice format.

[0011] The "means for sending a message" is a function that allows a relative to input a message from a remote location and send the message to the target terminal.

[0012] The "audio message conveying means" is a mechanism for conveying the received message to the user in audio form.

[0013] "Retry" is the process of attempting a particular action multiple times if the user does not respond to the same action.

[0014] The "means for notifying relatives" is a notification function for informing relatives of the situation when the user does not respond to an action.

[0015] "Means for calling emergency services" means a mechanism that allows a user to quickly contact emergency services in the event of an emergency using voice commands or buttons.

[0016] An "online shopping site" is a website for conducting e-commerce and a platform where you can purchase everyday items.

[0017] A "voice command" is a spoken input that utilizes voice recognition technology to receive user instructions and perform a specific operation.

[0018] "Daily necessities" are household products such as consumables and miscellaneous goods used in daily life. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system for supporting the lives of the elderly, aiming to promote daily conversations and facilitate communication with relatives. The system mainly includes the following elements: a server, a terminal used by the user, and a terminal used remotely by relatives.

[0041] Overall system overview

[0042] 1. Server:

[0043] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides them to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[0044] 2. Terminal:

[0045] The user's device uses voice synthesis technology to send information to the user. It also has the ability to recognize the user's voice and send response data to the server. In an emergency, an emergency call can be made using voice commands or a button. The device also has a function that links with online shopping sites to help with purchasing daily necessities.

[0046] 3. Family members' devices:

[0047] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[0048] Program processing

[0049] Daily conversation promotion features

[0050] server:

[0051] At 6:50 in the morning, it collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[0052] Device:

[0053] At 7:00 a.m., the latest information received from the server is analyzed using voice synthesis technology, and the system speaks to the user, saying things like, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0054] User:

[0055] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[0056] Device:

[0057] The system recognizes the user's response and sends the analysis results to the server. Based on the response received from the server, the system responds with "That's interesting. What era are you talking about?"

[0058] Message function from relatives

[0059] relatives:

[0060] The relative opens the smartphone app, enters the message content and desired sending time, and sends it to the server.

[0061] server:

[0062] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[0063] Device:

[0064] A message will be received at the specified time, telling the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[0065] User:

[0066] The user responds, "Okay."

[0067] Device:

[0068] The user's response is sent to the server. If there is no response, the message is sent again after 10 minutes, and this is repeated up to three times. If there is still no response, the server is notified.

[0069] server:

[0070] If the user does not respond and there is still no response after a certain number of retries, a notification is sent to the next of kin.

[0071] Emergency call function

[0072] User:

[0073] In an emergency, you can either say "help" to the device or press the emergency button.

[0074] Device:

[0075] When an emergency command is recognized, it will automatically call 119 or 110 and simultaneously send the user's location information. It will also notify relatives.

[0076] Online shopping integration function

[0077] User:

[0078] Speak to the device and say, "I'd like to order daily necessities."

[0079] Device:

[0080] It recognizes voice commands, sends requests to the server, receives a list of product candidates from the server, and presents them to the user via voice.

[0081] server:

[0082] The ordering process is initiated through the YAHOO Shopping API and order confirmation information is sent back to the terminal.

[0083] Device:

[0084] The user is notified by voice that the order has been completed.

[0085] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. By integrating functions for daily conversations, emergency response, and online shopping support, it contributes to improving the quality of life for elderly people.

[0086] The processing flow will be explained below.

[0087] Daily conversation promotion features

[0088] Server-side processing

[0089] Step 1:

[0090] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[0091] Step 2:

[0092] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[0093] Step 3:

[0094] The server transmits the acquired data to each user's terminal.

[0095] Terminal side processing

[0096] Step 1:

[0097] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[0098] Step 2:

[0099] The terminal analyzes the received information and generates content that speaks to the user.

[0100] Step 3:

[0101] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0102] User processing

[0103] Step 1:

[0104] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[0105] Terminal side processing

[0106] Step 1:

[0107] The device performs voice recognition on the user's response and sends the analysis results to the server.

[0108] Step 2:

[0109] The server analyzes the user's response, generates response data, and sends it to the terminal.

[0110] Step 3:

[0111] The device analyzes the received response data and responds to the user, "That's interesting. What era are you talking about?"

[0112] Message function from relatives

[0113] Processing for relatives

[0114] Step 1:

[0115] The relative opens the smartphone app and enters the message content and desired delivery time.

[0116] Step 2:

[0117] The relative taps the "Send" button to send the message to the server.

[0118] Server-side processing

[0119] Step 1:

[0120] The server stores the message received from the relative and the specified time in a database.

[0121] Step 2:

[0122] At the specified time, the server sends a message to the corresponding user's terminal.

[0123] Terminal side processing

[0124] Step 1:

[0125] The terminal receives a message from the server at a specified time.

[0126] Step 2:

[0127] The terminal will then audibly convey the received message to the user.

[0128] Step 3:

[0129] The terminal tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[0130] User processing

[0131] Step 1:

[0132] The user responds, "Okay."

[0133] Terminal side processing

[0134] Step 1:

[0135] The terminal recognizes the user's response and sends it to the server.

[0136] Step 2:

[0137] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[0138] Step 3:

[0139] If the device does not receive a response after three retries, it notifies the server.

[0140] Server-side processing

[0141] Step 1:

[0142] The server detects the user's lack of response and sends a notification to the next of kin.

[0143] Emergency call function

[0144] User processing

[0145] Step 1:

[0146] In an emergency, the user can either say "help" to the device or press the emergency button.

[0147] Terminal side processing

[0148] Step 1:

[0149] The device recognizes emergency commands or button triggers.

[0150] Step 2:

[0151] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[0152] Step 3:

[0153] The device will send a notification to relatives as soon as the emergency call is completed.

[0154] Online shopping integration function

[0155] User processing

[0156] Step 1:

[0157] The user speaks to the terminal saying, "I would like to order daily necessities."

[0158] Terminal side processing

[0159] Step 1:

[0160] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[0161] Step 2:

[0162] The terminal presents the candidate product list received from the server to the user by voice.

[0163] Step 3:

[0164] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[0165] Server-side processing

[0166] Step 1:

[0167] The server initiates the order process through the YAHOO Shopping API.

[0168] Step 2:

[0169] Once the order is complete, the server sends confirmation information to the terminal.

[0170] Terminal side processing

[0171] Step 1:

[0172] The terminal will notify the user by voice that the order has been completed.

[0173] Example 1

[0174] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0175] It is becoming increasingly difficult for elderly people to maintain daily contact with society, leading to increased isolation and health risks. It is also difficult for relatives who live far away to constantly check on the safety of their elderly relatives. Furthermore, it is difficult for elderly people to find a way to respond quickly in an emergency or to purchase daily necessities.

[0176] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0177] In this invention, the server includes: means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user via voice; means for a relative to remotely send a message to the terminal and deliver the message via voice to the user at a specified time; means for retrying a certain number of times if there is no response from the user and notifying the relative if there is still no response; means for retrying via voice if there is no response for a long period of time and notifying the relative if there is still no response; means for the user to make an emergency call using a voice command or button in an emergency; means for connecting with an online shopping system to purchase daily necessities using voice commands; means for collecting various information at a specified time and transmitting it to the terminal; means for voice recognition of the user's response, transmitting it to the server, analyzing the response from the server, and responding; and means for notifying the relative if there is no conversation for a certain period of time. This allows the elderly to maintain daily information and communication, allowing relatives to easily check the elderly's safety. It also enables quick response in emergencies and smooth purchasing of daily necessities.

[0178] "Weather information" is data about current and forecast atmospheric conditions, such as weather, temperature, precipitation, and wind speed.

[0179] "News information" refers to the latest reports on events and topics in various fields, including society, politics, economics, and culture.

[0180] "User's hobbies and preferences" refers to data and information related to the user's interests, favorite activities, favorite items, and the like.

[0181] "Means of providing by voice" is a function that uses voice synthesis technology to convey text or data to the user as voice output.

[0182] "Relatives" refers to family members or close friends of the user who are interested in the user's life and well-being.

[0183] A "message" is a word or content conveyed in the form of a message, and is information sent primarily from relatives to the user.

[0184] The "means for retrying if there is no response" is a function for sending the same message again if the user does not respond to a specified message.

[0185] "Means to call emergency services" is the ability for a user to contact emergency services via voice command or physical button in the event of an emergency.

[0186] An "online shopping system" is a system that allows you to search for and select products and complete the purchasing process via the Internet.

[0187] "Means of collecting various information" refers to the function of obtaining weather information, news information, hobby-related information, etc. from external data sources.

[0188] The "means for recognizing voice and responding" is a function that converts the user's voice into text data, sends it to the server, and generates an appropriate voice message based on the analysis results.

[0189] The "means for notifying when there is no conversation for a certain period of time" is a function that notifies relatives of the situation when the user does not talk to the system for a certain period of time.

[0190] This invention is a system for supporting the daily lives of elderly people, and aims to promote daily conversations and facilitate communication with relatives. The system includes a server, a terminal used by the user, and a terminal used by relatives remotely.

[0191] server

[0192] The server is responsible for periodically collecting information from multiple external data sources and providing it to the user's device. Specifically, it uses the OpenWeatherMap API as a weather API to obtain weather information, and the News API as a news API to obtain news information. In addition, information based on the user's hobbies and preferences is obtained from the user's hobby database. This information is collected at a specified time (for example, 6:50 every morning) and sent to the user's device.

[0193] In addition, the server receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response after a certain number of attempts. Furthermore, if there is no response for a long period of time, it retries by voice and eventually notifies the relatives.

[0194] User's device

[0195] The user's device is equipped with speech synthesis technology (e.g., Google Text-to-Speech) and speech recognition technology (e.g., Google Speech-to-Text). This device has the ability to convey information received from the server to the user by voice. For example, at 7:00 in the morning, the device might say, "Good morning. It's a sunny day today. The temperature is 22 degrees. How is the book you've been reading recently?"

[0196] When the user responds to this question, the device uses voice recognition technology to convert the user's response into text data and sends it to the server. The server then generates an appropriate response based on the received data and sends the result to the device. The device then uses voice synthesis technology to respond to the user, saying, "That's interesting. What era are you talking about?"

[0197] Relatives' devices

[0198] Relatives can use devices such as smartphones and PCs to send messages to elderly people from remote locations. Using a dedicated application, the relative inputs the message content and desired sending time, and sends it to the server. The server then sends the message to the user's device at the specified time and conveys it to the user by voice. For example, the user's device might say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[0199] Emergency call function

[0200] In an emergency, if the user says "help" or presses the emergency button, the device will recognize the emergency command and automatically call 119 or 110. At the same time, the user's location information will also be sent to the emergency services. An emergency notification will also be sent to relatives.

[0201] Online shopping integration function

[0202] When a user says to the terminal, "I would like to order daily necessities," the terminal recognizes the voice command and sends a request to the server. The server obtains a list of candidate products through the online shopping system's API (for example, YAHOO Shopping API) and sends it to the terminal. The terminal presents the candidate product list to the user by voice, recognizes the user's selection, and sends it back to the server. The server executes the ordering process based on the user's selection and returns order confirmation information to the terminal. The terminal then notifies the user by voice that the order has been completed.

[0203] The system will enable elderly people to maintain daily information and communication, allow relatives to easily check on their safety, and facilitate rapid response in emergencies and the smooth purchase of daily necessities.

[0204] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0205] Step 1:

[0206] Server: Collects weather information, news information, and user hobbies and preferences every morning at 6:50.

[0207] Inputs: OpenWeatherMap API, NewsAPI, Hobby Database

[0208] Data processing and calculation: Use API to get current weather and latest news. Get the latest information relevant to the user from the hobby database.

[0209] Output: Generates a set of weather, news, and interest information and prepares it for transmission to the user's device.

[0210] Step 2:

[0211] Terminal: Information received at 7:00 a.m. is analyzed using voice synthesis technology and provided to the user.

[0212] Input: A set of weather information, news information, and hobby / preference information received from the server

[0213] Data processing and calculation: Convert received information into a voice message using Google Text-to-Speech.

[0214] Output: Provide the user with a voice message saying, "Good morning. It's a sunny day today. The temperature is 72 degrees. How's your reading going?"

[0215] Step 3:

[0216] User: Responds to the device by saying, "Good morning. I've been reading historical novels lately."

[0217] Input: Voice response to terminal

[0218] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[0219] Output: Generate the converted text data "Good morning. The book I'm reading recently is a historical novel." and prepare to send it to the server.

[0220] Step 4:

[0221] Terminal: Sends the user's voice response to the server.

[0222] Input: User's speech converted to text

[0223] Data processing and calculation: Converts text data into a protocol for sending to the server.

[0224] Output: Text data sent to the server

[0225] Step 5:

[0226] Server: Analyzes the user's response and generates an appropriate response.

[0227] Input: Text data sent by the user

[0228] Data processing and computation: Analyzing text data using natural language processing techniques to generate appropriate responses. For example, a generative AI model could be used to generate the response, "That's interesting. What era are you talking about?"

[0229] Output: Sends the generated reply to the terminal.

[0230] Step 6:

[0231] Terminal: Provides the user with a voice response to the response received from the server.

[0232] Input: Text data sent from the server

[0233] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[0234] Output: Provides the user with a spoken message saying "That's interesting. What era are you talking about?"

[0235] Step 7:

[0236] Relatives: Use the smartphone app to enter the message content and desired delivery time, and send it to the server.

[0237] Input: Message content, desired sending time

[0238] Data processing and calculation: Data entered via the smartphone app is converted into a protocol for sending to the server.

[0239] Output: Message sent to the server and desired delivery time

[0240] Step 8:

[0241] Server: Sends messages from relatives to the user's device at the specified time.

[0242] Input: Message sent by relative and desired time of sending

[0243] Data processing and calculation: The message content is stored and sent to the user's device at the specified time.

[0244] Output: Message sent to user's device

[0245] Step 9:

[0246] Terminal: Provides the user with a voice message at the specified time.

[0247] Input: Message sent from the server

[0248] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[0249] Output: Provides a voice message saying "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready."

[0250] Step 10:

[0251] User: "Okay," replies the terminal.

[0252] Input: Voice response to terminal

[0253] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[0254] Output: Generates "I understand." as converted text data and prepares it to send to the server.

[0255] Step 11:

[0256] Terminal: Sends the user's voice response to the server.

[0257] Input: User's speech converted to text

[0258] Data processing and calculation: Converts text data into a protocol for sending to the server.

[0259] Output: Text data sent to the server

[0260] Step 12:

[0261] Server: If the user does not respond, perform a series of retries, eventually notifying the next of kin.

[0262] Input: No response status

[0263] Data processing and calculation: If there is no response, the message will be sent again after 10 minutes, and this will be repeated up to three times. If there is still no response, the next of kin will be notified.

[0264] Output: Message to notify relatives

[0265] Step 13:

[0266] User: In an emergency, say "help" or press the emergency button.

[0267] Input: Emergency voice command or button press

[0268] Data processing and calculation: Recognize emergency commands and collect necessary data (such as location information).

[0269] Output: Emergency services and immediate family notification

[0270] Step 14:

[0271] Terminal: Recognizes emergency commands and makes emergency calls.

[0272] Input: User voice command or button press

[0273] Data processing and calculation: Obtain location information and automatically call 119 or 110.

[0274] Output: Emergency call message and location information

[0275] Step 15:

[0276] Relatives: Receive emergency notifications.

[0277] Input: Urgent notification sent from the server

[0278] Data processing and calculation: Display emergency notifications.

[0279] Output: Emergency notification displayed on a relative's device

[0280] Step 16:

[0281] User: Says to the device, "I want to order some groceries."

[0282] Input: User's voice command

[0283] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[0284] Step 17:

[0285] Device: Recognizes voice commands and sends requests to the server.

[0286] Input: User's speech converted to text

[0287] Data processing and calculation: Converts text data into a protocol for sending to the server.

[0288] Output: Request sent to the server

[0289] Step 18:

[0290] Server: Obtains a list of product candidates through the API of the online shopping system.

[0291] Input: The request sent by the user

[0292] Data processing and calculation: Obtain a list of product candidates using the YAHOO Shopping API.

[0293] Output: Sends the product candidate list to the terminal.

[0294] Step 19:

[0295] Terminal: Presents a list of product candidates to the user via voice.

[0296] Input: Product candidate list sent from the server

[0297] Data processing and calculation: Use Google Text-to-Speech to convert the product candidate list into a voice message.

[0298] Output: Provide the user with a list of products as a voice message

[0299] (Application example 1)

[0300] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] In elderly life support systems, there is a lack of means to promote daily conversations, communicate with relatives, respond to emergencies, and support purchasing daily necessities. Furthermore, conventional systems rely on the devices used by the elderly, and the operation of the devices is complicated, making it difficult for the elderly to use them. Furthermore, there is a lack of a mechanism that allows relatives in remote locations to easily understand the status of the elderly.

[0302] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0303] In this invention, the server includes a means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice; a means for a relative to remotely send a message to the terminal and deliver the message by voice to the user at a specified time; and a means for generating appropriate responses to the user's questions using a generative AI model. This allows elderly people to easily enjoy conversations and smoothly communicate with their relatives. Furthermore, the use of a generative AI model enables more advanced and flexible responses, allowing the elderly to receive the information they need in a timely manner.

[0304] "Weather information" refers to meteorological data such as the climate conditions, temperature, and probability of precipitation in the elderly person's place of residence.

[0305] "News information" refers to the latest news data on domestic and international events, incidents, culture, economy, etc.

[0306] "Hobbies and Preference Data" refers to data that compiles information related to the user's interests and areas of interest.

[0307] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to provide appropriate answers and generate conversations in response to questions and utterances from users.

[0308] "Providing audio" refers to a computer using speech synthesis technology to audibly convey information to a user.

[0309] "Relatives" refers to family members and blood relatives of the user.

[0310] "Sending a message from a remote location" refers to a relative sending a message to the user's terminal from a physically distant location via the Internet.

[0311] The "designated time" refers to a specific time set in advance by the relative at which the message should be conveyed to the user.

[0312] "Voice command" refers to an instruction or command given by a user through a microphone.

[0313] "Online shopping site" refers to a website for purchasing products over the Internet.

[0314] "User location information" refers to data indicating the user's current physical location.

[0315] A "central processing unit" refers to a computer that functions as the main computing resource of a server, etc., and is responsible for controlling the entire system and processing data.

[0316] "Emergency services" refers to public agencies and services that respond to emergencies, such as fire departments, police, and emergency medical services.

[0317] "Daily necessities" refers to consumables and food items necessary for the daily lives of the elderly.

[0318] This invention is a system for supporting the lives of elderly people, and is realized mainly through cooperation between a server, a terminal used by the user, and a terminal used remotely by a relative. This system has the following configuration and functions.

[0319] 1. Server:

[0320] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. Specifically, every morning at 6:50, it collects the latest information from the weather API, news API, and user hobby database, and sends it to the user's device. It also has the function of generating appropriate responses to user questions using a generative AI model. It can also receive messages from relatives and send them to the user's device at a specified time. The server also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[0321] 2. On the user's device:

[0322] The user device uses speech synthesis technology to send information to the user, recognizes the user's voice using its speech recognition function, and sends it to the server. If the user responds, "The book I've been reading recently is a historical novel," the device recognizes the speech and sends it to the server. It receives the response from the server and uses a generative AI model to generate an appropriate response, such as, "That's interesting. What period is it set in?" In an emergency, the user can issue a voice command such as "Help" or press an emergency button, and the device will automatically make an emergency call, obtain the user's location information, and provide it to medical services. It also works with online shopping sites, recognizing the voice command "I'd like to order daily necessities" and initiating the ordering process via the server.

[0323] 3. Family members' devices:

[0324] Relatives can remotely send messages to the elderly using devices such as smartphones or PCs. When the relative enters the message and the time to send it, the data is sent to a server, and the server sends the message to the user's device at the specified time. For example, a message such as "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready" can be spoken to the user.

[0325] Hardware and software used

[0326] Speech synthesis technology: pyttsx3

[0327] Speech recognition technology: Google Speech Recognition API

[0328] Weather API and News API: Includes any external services

[0329] Generative AI model: using natural language processing techniques

[0330] Specific use cases

[0331] An elderly person is asked at 7:00 a.m. from a terminal, "Good morning. It's a sunny day today. The temperature is 22 degrees. What book have you been reading recently?" If the user responds, "The book I've been reading recently is a historical novel," the server analyzes the data and uses a generative AI model to provide an appropriate response, such as, "That's interesting. What period is it set in?"

[0332] Prompt Sentence Examples

[0333] "Today's weather is sunny and the temperature is 22 degrees. Tell us about the book you've been reading recently."

[0334] This system will enable the elderly to enjoy everyday conversations and communicate smoothly with their relatives. It will also enable emergency response and assistance with purchasing everyday items, improving the quality of life for the elderly.

[0335] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0336] Step 1:

[0337] (Input) The server collects information from the weather API, news API, and hobby database.

[0338] (Processing) Every morning at 6:50, the server sends a request to an external service to collect data related to weather, news, and hobbies.

[0339] (Output) Collected weather information, news information, and hobby-related data.

[0340] (Specific operation) The server sends a request to the API, receives weather and news information in response, and searches and collects related information from the user's hobby database.

[0341] Step 2:

[0342] (Input) Information collected by the server.

[0343] (Processing) The server sends the collected information to the user's terminal.

[0344] (Output) Weather, news, and hobby-related information sent to the user's device.

[0345] (Specific Operation) The server formats the collected data and sends it to the user's device, which receives this information.

[0346] Step 3:

[0347] (Input) Information sent to the user's device.

[0348] (Processing) The terminal speaks to the user using voice synthesis technology.

[0349] (Output) A voice message to the user.

[0350] (Specific operation) The user's device converts information about the weather, news, and hobbies using a speech synthesis engine (pyttsx3) and speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0351] Step 4:

[0352] (Input) The user's response.

[0353] The (processing) terminal uses voice recognition technology to convert the user's response into text.

[0354] (Output) The user's response in text format.

[0355] (Specific operation) When the user responds, "The book I've been reading recently is a historical novel," the device converts the speech into text using the Google Speech Recognition API.

[0356] Step 5:

[0357] (Input) The user's response in text form.

[0358] (Processing) The terminal sends the user's response to the server.

[0359] (Output) The user's response sent to the server.

[0360] (Specific operation) The terminal sends the converted text to the server via an HTTP request.

[0361] Step 6:

[0362] (Input) The user's reply text.

[0363] The (processing) server uses a generative AI model to generate an appropriate response.

[0364] (Output) The generated response.

[0365] (Specific operation) The server uses a generative AI model to generate a response such as "That's interesting. What period is it set in?" in response to "The book I've been reading lately is a historical novel."

[0366] Step 7:

[0367] (Input) The generated response.

[0368] (Processing) The server sends the generated response to the user's terminal.

[0369] (Output) The generated reply to the user's terminal.

[0370] (Specific operation) The response generated by the server is formatted for transmission to the user's terminal and sent to the terminal via an HTTP request.

[0371] Step 8:

[0372] (Input) The response sent by the server.

[0373] The (processing) terminal uses speech synthesis technology to communicate the response to the user.

[0374] (Output) A spoken response to the user.

[0375] (Specific operation) The device converts the response received from the server using a speech synthesis engine (pyttsx3) and speaks to the user, saying, "That's interesting. What era are you talking about?"

[0376] Step 9:

[0377] (Input) Message content and time sent by relatives.

[0378] (Processing) A relative uses a smartphone or computer from a remote location to send the message content and sending time to the server.

[0379] (Output) Message content and sending time stored on the server.

[0380] (Specific operation) A relative uses the application to enter the message content and sending time, and sends it to the server.

[0381] Step 10:

[0382] (Input) Message content and time sent from relatives.

[0383] (Processing) The server sends a message to the user's terminal at the specified time.

[0384] (Output) Messages from relatives to the user's terminal.

[0385] (Specific operation) The server monitors the specified time, and when the set time arrives, it sends a message to the user's terminal.

[0386] Step 11:

[0387] (Input) Message from relatives from the server.

[0388] (Processing) The terminal uses voice synthesis technology to convey the message to the user.

[0389] (Output) A voice message to the user.

[0390] (Specific operation) The terminal converts the message received from the server using a speech synthesis engine (pyttsx3) and tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[0391] Step 12:

[0392] (Input) If there is no response from the user.

[0393] (Processing) The terminal retries a certain number of times, and if there is no response, it notifies the server.

[0394] (Output) Notifies the server of the user's response status.

[0395] (Specific operation) The terminal waits for a response from the user, and if there is still no response, it retries a certain number of times and sends the message again, and if there is still no response, it notifies the server.

[0396] Step 13:

[0397] (Input) User response status notification received by the server.

[0398] (Processing) The server notifies the relatives of the user's response status.

[0399] (Output) Notify relatives of the user's response status.

[0400] (Specific operation) The server sends emails and application notifications to relatives to report the user's response status.

[0401] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0402] This invention is a system for supporting the lives of the elderly and promoting conversation. This system not only provides weather information, news information, and topics based on the user's hobbies and preferences, but also incorporates an emotion engine to recognize the user's emotions and achieve more appropriate communication. The following describes each component of this system and the program processing.

[0403] Overall system overview

[0404] 1. Server:

[0405] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time. In addition, it analyzes the user's emotional data via an emotion engine and generates feedback based on this.

[0406] 2. Terminal:

[0407] The user's device uses voice synthesis technology to send information to the user. It also has the function of recognizing the user's voice and sending response data to the server. In an emergency, an emergency call can be made using voice commands or a button. It also has a function that links with online shopping sites to assist with purchasing daily necessities. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the content of the conversation based on the results.

[0408] 3. Family members' devices:

[0409] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[0410] Program processing

[0411] Daily conversation promotion features

[0412] server:

[0413] Every morning, the server collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[0414] Device:

[0415] Every morning, the device uses voice synthesis technology to analyze the latest information received from the server and speaks to the user, such as, "Good morning. It's sunny today. The temperature is 22 degrees. How's the book you've been reading recently?" The emotion engine also analyzes the user's emotions at this point and adjusts the tone and content of the conversation.

[0416] User:

[0417] The user replies, "Good morning. I've been reading historical novels lately."

[0418] Device:

[0419] The device recognizes the user's response through voice recognition, performs emotional analysis using an emotion engine, and then sends the results to the server.

[0420] server:

[0421] The server analyzes the user's response and emotional data, generates appropriate response data, and sends it to the terminal.

[0422] Device:

[0423] Based on the received response data and emotion data, the device responds to the user by saying, "That's interesting. What era are you talking about?"

[0424] Message function from relatives

[0425] relatives:

[0426] The relative opens the smartphone app and enters the message content and desired delivery time.

[0427] server:

[0428] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[0429] Device:

[0430] The message will arrive at the specified time and tell the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready." At this time, the emotion engine will analyze the user's reaction and provide feedback on the emotion to the relatives.

[0431] User:

[0432] The user responds, "Okay."

[0433] Device:

[0434] The device recognizes the user's response and sends it to the server. If there is no response, it retries a certain number of times, and if there is still no response, it notifies the server.

[0435] server:

[0436] If the server does not respond for a certain period of time, it will send a notification to the next of kin.

[0437] Emergency call function

[0438] User:

[0439] In an emergency, you can either say "help" to the device or press the emergency button.

[0440] Device:

[0441] The device detects an emergency command or button trigger and automatically calls 119 or 110. At the same time, emotional data is also sent to the server, and relatives are notified.

[0442] Online shopping integration function

[0443] User:

[0444] The user speaks to the terminal saying, "I would like to order daily necessities."

[0445] Device:

[0446] The device analyzes the user's voice commands using a voice recognition engine and emotion engine, and sends a request to the server.

[0447] server:

[0448] The server initiates the order process through the online shopping API and sends the order confirmation information to the terminal.

[0449] Device:

[0450] The terminal will then notify the user that the order has been completed and analyze the user's emotions during the process, providing appropriate feedback.

[0451] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. It combines functions for daily conversations, emergency response, and online shopping support, improving the quality of life for elderly people.

[0452] The processing flow will be explained below.

[0453] Daily conversation promotion features

[0454] Server-side processing

[0455] Step 1:

[0456] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[0457] Step 2:

[0458] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[0459] Step 3:

[0460] The server transmits the acquired data to each user's terminal.

[0461] Terminal side processing

[0462] Step 1:

[0463] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[0464] Step 2:

[0465] The terminal analyzes the received information and generates content that speaks to the user.

[0466] Step 3:

[0467] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0468] User processing

[0469] Step 1:

[0470] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[0471] Terminal side processing

[0472] Step 1:

[0473] The device recognizes the user's response through voice recognition, analyzes the emotion using an emotion engine, and then sends the results to the server.

[0474] Server-side processing

[0475] Step 1:

[0476] The server analyzes the user's response content and emotional data, generates appropriate response data, and sends it to the terminal.

[0477] Terminal side processing

[0478] Step 1:

[0479] The device analyzes the received response data and emotion data and responds to the user, "That's interesting. What era are you talking about?"

[0480] Message function from relatives

[0481] Processing for relatives

[0482] Step 1:

[0483] The relative opens the smartphone app and enters the message content and desired delivery time.

[0484] Step 2:

[0485] The relative taps the "Send" button to send the message to the server.

[0486] Server-side processing

[0487] Step 1:

[0488] The server stores the message received from the relative and the specified time in a database.

[0489] Step 2:

[0490] At the specified time, the server sends a message to the corresponding user's terminal.

[0491] Terminal side processing

[0492] Step 1:

[0493] The terminal receives a message from the server at a specified time.

[0494] Step 2:

[0495] The device then audibly conveys the received message to the user, saying, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[0496] Step 3:

[0497] The device uses an emotion engine to analyze the user's reaction and transmits the emotion data to the server.

[0498] User processing

[0499] Step 1:

[0500] The user responds, "Okay."

[0501] Terminal side processing

[0502] Step 1:

[0503] The terminal recognizes the user's response and sends it to the server.

[0504] Step 2:

[0505] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[0506] Step 3:

[0507] If the device does not receive a response after three retries, it notifies the server.

[0508] Server-side processing

[0509] Step 1:

[0510] The server detects the user's lack of response and sends a notification to the next of kin.

[0511] Emergency call function

[0512] User processing

[0513] Step 1:

[0514] In an emergency, the user can either say "help" to the device or press the emergency button.

[0515] Terminal side processing

[0516] Step 1:

[0517] The device recognizes emergency commands or button triggers.

[0518] Step 2:

[0519] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[0520] Step 3:

[0521] The device will send a notification to relatives as soon as the emergency call is completed.

[0522] Online shopping integration function

[0523] User processing

[0524] Step 1:

[0525] The user speaks to the terminal saying, "I would like to order daily necessities."

[0526] Terminal side processing

[0527] Step 1:

[0528] The device analyzes the user's voice commands using a voice recognition engine and emotion engine, and sends a request to the server.

[0529] Step 2:

[0530] The terminal presents the candidate product list received from the server to the user by voice.

[0531] Step 3:

[0532] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[0533] Server-side processing

[0534] Step 1:

[0535] The server initiates the order process through an online shopping API.

[0536] Step 2:

[0537] Once the order is complete, the server sends confirmation information to the terminal.

[0538] Terminal side processing

[0539] Step 1:

[0540] The device notifies the user by voice when the order is complete, and transmits the emotion data obtained during the process to the server to provide appropriate feedback to the relatives.

[0541] Example 2

[0542] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0543] In modern society, elderly people often feel lonely, and the limited opportunities for daily conversation and communication often have a negative impact on their mental and psychological health. It is also difficult to contact relatives who live far away, making it difficult to detect abnormalities or emergencies in the elderly in a timely manner. Furthermore, it can be difficult to purchase daily necessities, raising concerns about a decline in quality of life.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0545] In this invention, the server includes means for generating weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice, means for a relative to send a message to the terminal from a remote location and deliver the message by voice to the user at a specified time, and means for analyzing the user's emotion data using an emotion recognition engine and adjusting the tone and content of the conversation. This allows the user to promote daily conversations, enable appropriate communication with relatives even when they are far away, and make it possible to respond to emergencies and purchase daily necessities smoothly.

[0546] "Weather information" refers to information such as temperature, precipitation, wind speed, and weather conditions provided based on meteorological data.

[0547] "News information" refers to reports and articles about the latest events and social topics.

[0548] "Hobbies and preferences" is information related to activities and topics that an individual is interested in and engages in for fun.

[0549] "Users" refer to the elderly people who use this system.

[0550] "Remote location" refers to a location that is physically separate, and typically refers to a case where the user and their relatives are in different locations.

[0551] "Message" refers to a message that a relative wants to convey to the user.

[0552] A "voice command" is a method in which a user issues instructions to a terminal using voice.

[0553] An "emotion recognition engine" is a technology for detecting and analyzing emotions from a user's voice and facial expressions.

[0554] "Emergency call" is a means of contacting a user to request help in an emergency.

[0555] An "online shopping site" is a website where you can purchase goods and services over the Internet.

[0556] "Speech synthesis technology" is a technology for converting text data into voice data.

[0557] A "voice recognition engine" is a technology for converting voice data into text data.

[0558] This invention relates to a system for supporting the lives of the elderly and promoting conversation. This system provides weather information, news information, and topics based on the user's hobbies and preferences, and also uses an emotion recognition engine to achieve more appropriate communication.

[0559] Overall system configuration

[0560] 1. Server:

[0561] We use weather APIs (e.g., OpenWeatherMap API) and news APIs (e.g., NewsAPI) to periodically collect data related to weather information, news information, and hobbies and preferences.

[0562] The system receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response for a certain period of time.

[0563] An emotion recognition engine (e.g., Affectiva, IBM Watson Tone Analyzer) is used to analyze the user's emotion data and generate feedback.

[0564] 2. Terminal:

[0565] Using speech synthesis technology (e.g., Google Text-to-Speech API), information is transmitted to the user via voice.

[0566] A speech recognition engine (e.g., Google Speech-to-Text API) is used to recognize the user's voice and send response data to the server.

[0567] In an emergency, use voice commands or press a button to make an emergency call.

[0568] It works in conjunction with online shopping sites and has the ability to assist with purchasing daily necessities using voice commands.

[0569] An emotion recognition engine is used to recognize emotions from the user's voice and facial expressions and adjust the content of the conversation.

[0570] 3. Family members' devices:

[0571] Using a smartphone or personal computer, messages can be sent to elderly people from remote locations, and notifications can be received in the event of an emergency or when safety confirmation is required.

[0572] Specific examples

[0573] Daily conversation promotion features

[0574] server:

[0575] Every morning, the server collects the latest information from weather APIs, news APIs, and the user's hobby database and sends it to the device.

[0576] Device:

[0577] Every morning, the device receives the latest information from the server, analyzes it using speech synthesis technology, and speaks to the user. Example: "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0578] User:

[0579] The user replies, "Good morning. I've been reading historical novels lately."

[0580] Device:

[0581] The device converts the user's response into text using a voice recognition engine, analyzes the emotion using an emotion recognition engine, and sends the results to the server.

[0582] server:

[0583] The server generates an appropriate response based on the user's response and emotional data and sends it to the device. Example: "That's interesting. What era are you talking about?"

[0584] Device:

[0585] The device uses voice synthesis technology to communicate with the user based on the received response and emotional data. Example: "That's interesting. What era are you talking about?"

[0586] Message function from relatives

[0587] relatives:

[0588] The relative opens the smartphone app and enters the message and desired delivery time. For example, "Please let them know I have a doctor's appointment tomorrow at 9:00 AM."

[0589] server:

[0590] The server stores the message content and the specified time received from the relative, and sends the message to the corresponding user's terminal when the specified time arrives.

[0591] Device:

[0592] The device receives the message at the specified time and uses speech synthesis technology to convey it to the user. For example, "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready." An emotion recognition engine analyzes the user's reaction and provides feedback on that emotion to the relative.

[0593] Emergency call function

[0594] User:

[0595] In an emergency, you can either say "help" to the device or press the emergency button.

[0596] Device:

[0597] The device will detect the emergency command or button trigger and immediately make an automatic call to 119 or 110. Example: "An elderly person needs help."

[0598] The device sends the emotional data along with the report to a server, and notifies relatives.

[0599] Online shopping integration function

[0600] User:

[0601] The user speaks to the terminal saying, "I would like to order daily necessities."

[0602] Device:

[0603] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[0604] server:

[0605] The server initiates the order process through an online shopping API. Example: "Your grocery order has been initiated."

[0606] Device:

[0607] The device notifies the user of order confirmation information from the server using voice synthesis technology and provides emotional feedback. Example: "Your order has been completed."

[0608] Prompt Sentence Examples

[0609] Based on the following conversation, please suggest topics that might interest users next:

[0610] User: "The book I've been reading lately is a historical novel."

[0611] System: "That's interesting. What era are you talking about?"

[0612] User: "It's from the Meiji period."

[0613] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0614] Specific explanation of processing steps

[0615] Daily conversation promotion features

[0616] server

[0617] Step 1:

[0618] The server collects weather information from a weather API at a fixed time every morning. The specific input is the request data to the API, and the output is the retrieved weather information. It also obtains the latest news information from a news API and extracts topics based on the user's hobbies and preferences from a hobby database.

[0619] Step 2:

[0620] The server integrates the collected weather information, news information, and hobby data into a JSON format and sends this data package to the terminal. The input data is each piece of collected information, and the output is the compiled JSON data.

[0621] Terminal

[0622] Step 3:

[0623] The device analyzes the JSON data received from the server. During analysis, it uses speech synthesis technology (e.g., Google Text-to-Speech API) to convert the transmitted information into voice data. Specifically, it converts text input into voice data and speaks to the user. Example: "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0624] User

[0625] Step 4:

[0626] The user responds to the voice guidance from the terminal. For example, they might say, "Good morning. The book I've been reading recently is a historical novel." This user's voice data becomes the input.

[0627] Terminal

[0628] Step 5:

[0629] The device converts the user's response into text data using a speech recognition engine (e.g., Google Speech-to-Text API). It then uses an emotion recognition engine to analyze emotional data from the user's voice. The input is the user's voice data, and the output is the recognized text data and emotional data.

[0630] Step 6:

[0631] The device sends the obtained text data and emotion data to the server. The input is the analysis results (text data and emotion data) on the device side, which are then sent to the server.

[0632] server

[0633] Step 7:

[0634] The server analyzes the received user response and emotional data. Specifically, it uses a natural language processing (NLP) model (e.g., OpenAI GPT-3) to generate an appropriate response. The input is the user response and emotional data, and the output is the generated response. Example: "That's interesting. What era are you talking about?"

[0635] Step 8:

[0636] The server sends the generated response along with the emotion data to the terminal. The input is the response content and emotion data, and the output is the transmitted data.

[0637] Terminal

[0638] Step 9:

[0639] The device receives the response and emotion data from the server and again uses speech synthesis technology to speak to the user. Example: "That's interesting. What era are you talking about?" The input data are the received response and emotion data, and the output is voice data.

[0640] Message function from relatives

[0641] relatives

[0642] Step 1:

[0643] The relative opens the smartphone app and inputs the message content and desired sending time. For example, "Please tell them I have a doctor's appointment tomorrow at 9:00 AM."

[0644] server

[0645] Step 2:

[0646] The server stores the message content received from the relatives and the specified sending time in a database. The input is the message content and sending time, and the output is the stored data.

[0647] Step 3:

[0648] When the designated sending time arrives, the server retrieves the stored message data and sends the message to the corresponding user's terminal. The input is the sending time and message data, and the output is the message to be sent.

[0649] Terminal

[0650] Step 4:

[0651] The device receives the message from the server at the specified time and conveys it to the user using speech synthesis technology. Example: "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready." At this time, the emotion recognition engine analyzes the user's reaction and generates emotional data. The input is the message, and the output is voice data and emotional data.

[0652] User

[0653] Step 5:

[0654] The user responds, "I understand." This voice data is the input.

[0655] Terminal

[0656] Step 6:

[0657] The device converts the user's response into text data using a speech recognition engine and sends it to the server. At the same time, emotion data is also sent to the server. The input is the user's voice data, and the output is text data and emotion data.

[0658] server

[0659] Step 7:

[0660] If there is no response for a certain period of time, the server will retry, and if there is still no response, it will send a notification to the relatives. The input is the response status, and the output is the notification message.

[0661] Emergency call function

[0662] User

[0663] Step 1:

[0664] The user can say "help" or press the emergency button on the terminal. This voice data or button down is the input.

[0665] Terminal

[0666] Step 2:

[0667] The device detects an emergency command or button trigger and immediately and automatically calls 119 or 110. Specifically, it uses voice synthesis technology to automatically generate the message content. The input is the emergency command or button trigger, and the output is the message data.

[0668] Step 3:

[0669] Along with the emergency call, the device sends emotional data to the server and notifies relatives. The input is the emergency call and emotional data, and the output is the transmitted data.

[0670] Online shopping integration function

[0671] User

[0672] Step 1:

[0673] The user speaks to the terminal, saying, "I would like to order daily necessities." This voice data is the input.

[0674] Terminal

[0675] Step 2:

[0676] The device analyzes the user's voice commands using a voice recognition engine and sends a request to the server. The input is voice data and the output is the voice recognition result.

[0677] server

[0678] Step 3:

[0679] The server initiates the order process through an online shopping API, placing the order with the necessary authentication information and product data. The input is the speech recognition result, and the output is the order confirmation data.

[0680] Step 4:

[0681] The server sends order confirmation information to the terminal. The input is the order confirmation data and the output is the transmission data.

[0682] Terminal

[0683] Step 5:

[0684] The device receives order confirmation information from the server and notifies the user using voice synthesis technology. For example, "Your order has been completed." At the same time, emotional feedback is provided. The input is order confirmation data, and the output is voice data and emotional feedback.

[0685] (Application example 2)

[0686] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0687] Isolation and lack of communication among the elderly are serious problems in modern society. It is also difficult for relatives living far away to keep track of the elderly's lifestyle and safety, which can delay emergency response. It is also difficult for elderly people to obtain timely entertainment and information that suits their emotions and moods. There is a need for a system that can solve these issues and improve the quality of life for the elderly.

[0688] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather information and news information and generating topics taking into account the user's hobbies and preferences, means for performing emotion analysis and recommending content suitable for the user, and means for relatives to send messages from remote locations and deliver them to the user at a specified time. This prevents the elderly from becoming isolated, enables appropriate communication to be maintained, and makes it easier for relatives to understand the elderly's condition even from remote locations. It also makes it possible to provide appropriate support and comfortable entertainment in emergencies and in daily life.

[0689] "Weather information" is data relating to the current weather conditions and forecasted weather conditions in the user's place of residence.

[0690] "News information" refers to the latest information and news reports about current events and social topics.

[0691] "User hobbies and preferences" is data about the activities and interests of the user.

[0692] "Emotion analysis" is the process of identifying and assessing a user's emotional state from their voice and facial expressions.

[0693] "Content" means digital data that provides information or entertainment, such as music, video, news, and articles.

[0694] "Relatives" refers to family members or relatives who are related by blood or legal relationship to the user.

[0695] An "emergency" is when a user is faced with a serious health or safety situation.

[0696] A "voice command" is an instruction that a user gives to a system using voice.

[0697] An "online shopping site" is a website for purchasing goods and services over the Internet.

[0698] "Server" means a computer system that collects, processes, and transmits data over a network.

[0699] A "terminal" is a device that a user can directly operate (such as a smartphone, smart glasses, a head-mounted display, or a robot).

[0700] This invention is a system for supporting the lives of the elderly and promoting conversation. This system provides weather information, news information, and topics based on the user's hobbies and preferences, and uses an emotion engine to evaluate the user's emotional state and provide appropriate content. Below, we will clearly explain the program processes and hardware and software configurations required to implement this invention.

[0701] server

[0702] A server is a computer system connected to the Internet that has the following functions:

[0703] 1. Data collection: Data is collected periodically from weather information API, news information API, and a database of user hobbies.

[0704] 2. Sentiment analysis: The user's voice data is sent to the sentiment analysis engine to evaluate the emotional state.

[0705] 3. Data transmission: Collected data and analysis results are sent to the device.

[0706] 4. Family collaboration: Receive messages and notifications from relatives and forward them to the appropriate device at the specified time.

[0707] 5. Emergency Calls: Receive emergency calls from users and contact emergency services as needed.

[0708] As a concrete example, every morning the server obtains the current weather conditions from a weather information API, collects news and topics based on the user's hobbies and preferences, and sends them to the device. At this time, the server analyzes the user's voice data from the previous day using an emotion analysis engine, and selects and provides topics that suit the user's mood.

[0709] Terminal

[0710] The device operated by the user (smartphone, smart glasses, head-mounted display, or robot) has the following functions:

[0711] 1. Voice recognition: Recognizes the user's voice instructions and sends the data to the server.

[0712] 2. Speech synthesis: The data received from the server is presented to the user as voice.

[0713] 3. Emotion analysis: Analyze the user's voice and facial expressions using the built-in emotion analysis engine.

[0714] 4. Message from relatives: A voice message from a relative will be delivered at the specified time.

[0715] 5. Emergency Call: In case of an emergency, the device receives voice commands or button inputs from the user and makes an emergency call.

[0716] As a concrete example, the device will speak to the user every morning saying, "Good morning. It's a sunny day today. How do you like the novel you've been reading recently?" and will then analyze the user's response using an emotion analysis engine to provide more appropriate topics of conversation.

[0717] User

[0718] Users can give voice instructions to the device and receive information. They can receive messages from relatives or purchase daily necessities using voice commands. In an emergency, they can also make an emergency call by saying "help" to the device.

[0719] relatives

[0720] Relatives can use smartphones or computers to send messages to the elderly and receive notifications in emergencies and regular safety checks, making it easier for relatives to keep track of the elderly's living conditions even when they are far away.

[0721] Example prompt sentence:

[0722] "Please wait as we are currently conducting an ongoing sentiment analysis."

[0723] "We recommend relaxing music to suit your mood."

[0724] "Good morning. It's a beautiful day today. How's the novel you've been reading lately?"

[0725] This will make it possible to provide an environment where elderly people can spend their days comfortably and safely.

[0726] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0727] Step 1:

[0728] Data collection (server)

[0729] The server collects the latest information from the weather information API, news information API, and user interest and preference database. Current weather data is obtained from the weather information API, and the latest news articles are collected from the news information API. Data related to the user's areas of interest is extracted from the user interest and preference database. This data is temporarily stored on the server.

[0730] Input: Weather information API, news information API, hobby and preference database

[0731] Output: Latest weather data, latest news data, hobby and preference data

[0732] Step 2:

[0733] Sentiment analysis (server)

[0734] The server sends the user's voice and text data to the emotion analysis engine to evaluate the user's emotional state. For example, if the user says "I'm tired," the emotion analysis engine will identify this as "fatigue" and return the result to the server.

[0735] Input: User voice or text data

[0736] Output: Emotion analysis results (e.g., fatigue)

[0737] Step 3:

[0738] Data transmission (server)

[0739] The server compiles the collected weather information, news information, user hobby data, and emotion analysis results and sends them to the user's device. This data becomes material for promoting daily conversations between users.

[0740] Input: Weather data, news data, hobby and preference data, emotion analysis results

[0741] Output: Consolidated data to send to the device

[0742] Step 4:

[0743] Speech synthesis and provision (terminal)

[0744] The device analyzes the data received from the server and provides it to the user using speech synthesis technology. For example, it might say something like, "Good morning. It's a sunny day today. How's the novel you've been reading recently?"

[0745] Input: Integrated data (weather data, news data, hobby and preference data, emotion analysis results)

[0746] Output: The audio message to be presented to the user

[0747] Step 5:

[0748] User response and reanalysis (terminal)

[0749] The user responds to the device, for example, saying, "The novel I've been reading recently is very interesting." The device recognizes this speech and sends it to the server. At the same time, the emotion analysis engine reanalyzes the user's emotional state.

[0750] Input: User's voice response

[0751] Output: Voice data sent to the server, reanalyzed emotional state

[0752] Step 6:

[0753] Message notification from relatives (server and terminal)

[0754] The server sends the message received from the relative to the terminal at the specified time. The terminal then delivers the message to the user by voice. For example, the terminal may say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[0755] Input: Message from relatives, specified time

[0756] Output: The audio message to be presented to the user

[0757] Step 7:

[0758] Emergency call (terminal)

[0759] In an emergency, the user can either say "help" to the device or press the emergency button, and the device will make an emergency call. The call will also include the user's location information.

[0760] Input: User's voice command or button press

[0761] Output: Emergency call with location information attached

[0762] Step 8:

[0763] Online shopping (terminal)

[0764] The user speaks to the device, saying, "I'd like to order some daily necessities." The device recognizes this command and sends it to the server. The server processes the order through the online shopping API and sends a confirmation to the device.

[0765] Input: User's voice command

[0766] Output: Online shopping order confirmation information

[0767] This will realize a system that can take into account the user's emotional state and provide appropriate information and communication.

[0768] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0769] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0770] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0771] [Second embodiment]

[0772] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0773] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0774] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0775] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0776] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0777] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0778] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0779] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0780] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0781] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0782] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0783] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0784] This invention is a system for supporting the lives of the elderly, aiming to promote daily conversations and facilitate communication with relatives. The system mainly includes the following elements: a server, a terminal used by the user, and a terminal used remotely by relatives.

[0785] Overall system overview

[0786] 1. Server:

[0787] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides them to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[0788] 2. Terminal:

[0789] The user's device uses voice synthesis technology to send information to the user. It also has the ability to recognize the user's voice and send response data to the server. In an emergency, an emergency call can be made using voice commands or a button. The device also has a function that links with online shopping sites to help with purchasing daily necessities.

[0790] 3. Family members' devices:

[0791] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[0792] Program processing

[0793] Daily conversation promotion features

[0794] server:

[0795] At 6:50 in the morning, it collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[0796] Device:

[0797] At 7:00 a.m., the latest information received from the server is analyzed using voice synthesis technology, and the system speaks to the user, saying things like, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0798] User:

[0799] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[0800] Device:

[0801] The system recognizes the user's response and sends the analysis results to the server. Based on the response received from the server, the system responds with "That's interesting. What era are you talking about?"

[0802] Message function from relatives

[0803] relatives:

[0804] The relative opens the smartphone app, enters the message content and desired sending time, and sends it to the server.

[0805] server:

[0806] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[0807] Device:

[0808] A message will be received at the specified time, telling the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[0809] User:

[0810] The user responds, "Okay."

[0811] Device:

[0812] The user's response is sent to the server. If there is no response, the message is sent again after 10 minutes, and this is repeated up to three times. If there is still no response, the server is notified.

[0813] server:

[0814] If the user does not respond and there is still no response after a certain number of retries, a notification is sent to the next of kin.

[0815] Emergency call function

[0816] User:

[0817] In an emergency, you can either say "help" to the device or press the emergency button.

[0818] Device:

[0819] When an emergency command is recognized, it will automatically call 119 or 110 and simultaneously send the user's location information. It will also notify relatives.

[0820] Online shopping integration function

[0821] User:

[0822] Speak to the device and say, "I'd like to order daily necessities."

[0823] Device:

[0824] It recognizes voice commands, sends requests to the server, receives a list of product candidates from the server, and presents them to the user via voice.

[0825] server:

[0826] The ordering process is initiated through the YAHOO Shopping API and order confirmation information is sent back to the terminal.

[0827] Device:

[0828] The user is notified by voice that the order has been completed.

[0829] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. By integrating functions for daily conversations, emergency response, and online shopping support, it contributes to improving the quality of life for elderly people.

[0830] The processing flow will be explained below.

[0831] Daily conversation promotion features

[0832] Server-side processing

[0833] Step 1:

[0834] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[0835] Step 2:

[0836] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[0837] Step 3:

[0838] The server transmits the acquired data to each user's terminal.

[0839] Terminal side processing

[0840] Step 1:

[0841] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[0842] Step 2:

[0843] The terminal analyzes the received information and generates content that speaks to the user.

[0844] Step 3:

[0845] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[0846] User processing

[0847] Step 1:

[0848] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[0849] Terminal side processing

[0850] Step 1:

[0851] The device performs voice recognition on the user's response and sends the analysis results to the server.

[0852] Step 2:

[0853] The server analyzes the user's response, generates response data, and sends it to the terminal.

[0854] Step 3:

[0855] The device analyzes the received response data and responds to the user, "That's interesting. What era are you talking about?"

[0856] Message function from relatives

[0857] Processing for relatives

[0858] Step 1:

[0859] The relative opens the smartphone app and enters the message content and desired delivery time.

[0860] Step 2:

[0861] The relative taps the "Send" button to send the message to the server.

[0862] Server-side processing

[0863] Step 1:

[0864] The server stores the message received from the relative and the specified time in a database.

[0865] Step 2:

[0866] At the specified time, the server sends a message to the corresponding user's terminal.

[0867] Terminal side processing

[0868] Step 1:

[0869] The terminal receives a message from the server at a specified time.

[0870] Step 2:

[0871] The terminal will then audibly convey the received message to the user.

[0872] Step 3:

[0873] The terminal tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[0874] User processing

[0875] Step 1:

[0876] The user responds, "Okay."

[0877] Terminal side processing

[0878] Step 1:

[0879] The terminal recognizes the user's response and sends it to the server.

[0880] Step 2:

[0881] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[0882] Step 3:

[0883] If the device does not receive a response after three retries, it notifies the server.

[0884] Server-side processing

[0885] Step 1:

[0886] The server detects the user's lack of response and sends a notification to the next of kin.

[0887] Emergency call function

[0888] User processing

[0889] Step 1:

[0890] In an emergency, the user can either say "help" to the device or press the emergency button.

[0891] Terminal side processing

[0892] Step 1:

[0893] The device recognizes emergency commands or button triggers.

[0894] Step 2:

[0895] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[0896] Step 3:

[0897] The device will send a notification to relatives as soon as the emergency call is completed.

[0898] Online shopping integration function

[0899] User processing

[0900] Step 1:

[0901] The user speaks to the terminal saying, "I would like to order daily necessities."

[0902] Terminal side processing

[0903] Step 1:

[0904] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[0905] Step 2:

[0906] The terminal presents the candidate product list received from the server to the user by voice.

[0907] Step 3:

[0908] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[0909] Server-side processing

[0910] Step 1:

[0911] The server initiates the order process through the YAHOO Shopping API.

[0912] Step 2:

[0913] Once the order is complete, the server sends confirmation information to the terminal.

[0914] Terminal side processing

[0915] Step 1:

[0916] The terminal will notify the user by voice that the order has been completed.

[0917] Example 1

[0918] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0919] It is becoming increasingly difficult for elderly people to maintain daily contact with society, leading to increased isolation and health risks. It is also difficult for relatives who live far away to constantly check on the safety of their elderly relatives. Furthermore, it is difficult for elderly people to find a way to respond quickly in an emergency or to purchase daily necessities.

[0920] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0921] In this invention, the server includes: means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user via voice; means for a relative to remotely send a message to the terminal and deliver the message via voice to the user at a specified time; means for retrying a certain number of times if there is no response from the user and notifying the relative if there is still no response; means for retrying via voice if there is no response for a long period of time and notifying the relative if there is still no response; means for the user to make an emergency call using a voice command or button in an emergency; means for connecting with an online shopping system to purchase daily necessities using voice commands; means for collecting various information at a specified time and transmitting it to the terminal; means for voice recognition of the user's response, transmitting it to the server, analyzing the response from the server, and responding; and means for notifying the relative if there is no conversation for a certain period of time. This allows the elderly to maintain daily information and communication, allowing relatives to easily check the elderly's safety. It also enables quick response in emergencies and smooth purchasing of daily necessities.

[0922] "Weather information" is data about current and forecast atmospheric conditions, such as weather, temperature, precipitation, and wind speed.

[0923] "News information" refers to the latest reports on events and topics in various fields, including society, politics, economics, and culture.

[0924] "User's hobbies and preferences" refers to data and information related to the user's interests, favorite activities, favorite items, and the like.

[0925] "Means of providing by voice" is a function that uses voice synthesis technology to convey text or data to the user as voice output.

[0926] "Relatives" refers to family members or close friends of the user who are interested in the user's life and well-being.

[0927] A "message" is a word or content conveyed in the form of a message, and is information sent primarily from relatives to the user.

[0928] The "means for retrying if there is no response" is a function for sending the same message again if the user does not respond to a specified message.

[0929] "Means to call emergency services" is the ability for a user to contact emergency services via voice command or physical button in the event of an emergency.

[0930] An "online shopping system" is a system that allows you to search for and select products and complete the purchasing process via the Internet.

[0931] "Means of collecting various information" refers to the function of obtaining weather information, news information, hobby-related information, etc. from external data sources.

[0932] The "means for recognizing voice and responding" is a function that converts the user's voice into text data, sends it to the server, and generates an appropriate voice message based on the analysis results.

[0933] The "means for notifying when there is no conversation for a certain period of time" is a function that notifies relatives of the situation when the user does not talk to the system for a certain period of time.

[0934] This invention is a system for supporting the daily lives of elderly people, and aims to promote daily conversations and facilitate communication with relatives. The system includes a server, a terminal used by the user, and a terminal used by relatives remotely.

[0935] server

[0936] The server is responsible for periodically collecting information from multiple external data sources and providing it to the user's device. Specifically, it uses the OpenWeatherMap API as a weather API to obtain weather information, and the News API as a news API to obtain news information. In addition, information based on the user's hobbies and preferences is obtained from the user's hobby database. This information is collected at a specified time (for example, 6:50 every morning) and sent to the user's device.

[0937] In addition, the server receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response after a certain number of attempts. Furthermore, if there is no response for a long period of time, it retries by voice and eventually notifies the relatives.

[0938] User's device

[0939] The user's device is equipped with speech synthesis technology (e.g., Google Text-to-Speech) and speech recognition technology (e.g., Google Speech-to-Text). This device has the ability to convey information received from the server to the user by voice. For example, at 7:00 in the morning, the device might say, "Good morning. It's a sunny day today. The temperature is 22 degrees. How is the book you've been reading recently?"

[0940] When the user responds to this question, the device uses voice recognition technology to convert the user's response into text data and sends it to the server. The server then generates an appropriate response based on the received data and sends the result to the device. The device then uses voice synthesis technology to respond to the user, saying, "That's interesting. What era are you talking about?"

[0941] Relatives' devices

[0942] Relatives can use devices such as smartphones and PCs to send messages to elderly people from remote locations. Using a dedicated application, the relative inputs the message content and desired sending time, and sends it to the server. The server then sends the message to the user's device at the specified time and conveys it to the user by voice. For example, the user's device might say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[0943] Emergency call function

[0944] In an emergency, if the user says "help" or presses the emergency button, the device will recognize the emergency command and automatically call 119 or 110. At the same time, the user's location information will also be sent to the emergency services. An emergency notification will also be sent to relatives.

[0945] Online shopping integration function

[0946] When a user says to the terminal, "I would like to order daily necessities," the terminal recognizes the voice command and sends a request to the server. The server obtains a list of candidate products through the online shopping system's API (for example, YAHOO Shopping API) and sends it to the terminal. The terminal presents the candidate product list to the user by voice, recognizes the user's selection, and sends it back to the server. The server executes the ordering process based on the user's selection and returns order confirmation information to the terminal. The terminal then notifies the user by voice that the order has been completed.

[0947] The system will enable elderly people to maintain daily information and communication, allow relatives to easily check on their safety, and facilitate rapid response in emergencies and the smooth purchase of daily necessities.

[0948] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0949] Step 1:

[0950] Server: Collects weather information, news information, and user hobbies and preferences every morning at 6:50.

[0951] Inputs: OpenWeatherMap API, NewsAPI, Hobby Database

[0952] Data processing and calculation: Use API to get current weather and latest news. Get the latest information relevant to the user from the hobby database.

[0953] Output: Generates a set of weather, news, and interest information and prepares it for transmission to the user's device.

[0954] Step 2:

[0955] Terminal: Information received at 7:00 a.m. is analyzed using voice synthesis technology and provided to the user.

[0956] Input: A set of weather information, news information, and hobby / preference information received from the server

[0957] Data processing and calculation: Convert received information into a voice message using Google Text-to-Speech.

[0958] Output: Provide the user with a voice message saying, "Good morning. It's a sunny day today. The temperature is 72 degrees. How's your reading going?"

[0959] Step 3:

[0960] User: Responds to the device by saying, "Good morning. I've been reading historical novels lately."

[0961] Input: Voice response to terminal

[0962] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[0963] Output: Generate the converted text data "Good morning. The book I'm reading recently is a historical novel." and prepare to send it to the server.

[0964] Step 4:

[0965] Terminal: Sends the user's voice response to the server.

[0966] Input: User's speech converted to text

[0967] Data processing and calculation: Converts text data into a protocol for sending to the server.

[0968] Output: Text data sent to the server

[0969] Step 5:

[0970] Server: Analyzes the user's response and generates an appropriate response.

[0971] Input: Text data sent by the user

[0972] Data processing and computation: Analyzing text data using natural language processing techniques to generate appropriate responses. For example, a generative AI model could be used to generate the response, "That's interesting. What era are you talking about?"

[0973] Output: Sends the generated reply to the terminal.

[0974] Step 6:

[0975] Terminal: Provides the user with a voice response to the response received from the server.

[0976] Input: Text data sent from the server

[0977] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[0978] Output: Provides the user with a spoken message saying "That's interesting. What era are you talking about?"

[0979] Step 7:

[0980] Relatives: Use the smartphone app to enter the message content and desired delivery time, and send it to the server.

[0981] Input: Message content, desired sending time

[0982] Data processing and calculation: Data entered via the smartphone app is converted into a protocol for sending to the server.

[0983] Output: Message sent to the server and desired delivery time

[0984] Step 8:

[0985] Server: Sends messages from relatives to the user's device at the specified time.

[0986] Input: Message sent by relative and desired time of sending

[0987] Data processing and calculation: The message content is stored and sent to the user's device at the specified time.

[0988] Output: Message sent to user's device

[0989] Step 9:

[0990] Terminal: Provides the user with a voice message at the specified time.

[0991] Input: Message sent from the server

[0992] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[0993] Output: Provides a voice message saying "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready."

[0994] Step 10:

[0995] User: "Okay," replies the terminal.

[0996] Input: Voice response to terminal

[0997] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[0998] Output: Generates "I understand." as converted text data and prepares it to send to the server.

[0999] Step 11:

[1000] Terminal: Sends the user's voice response to the server.

[1001] Input: User's speech converted to text

[1002] Data processing and calculation: Converts text data into a protocol for sending to the server.

[1003] Output: Text data sent to the server

[1004] Step 12:

[1005] Server: If the user does not respond, perform a series of retries, eventually notifying the next of kin.

[1006] Input: No response status

[1007] Data processing and calculation: If there is no response, the message will be sent again after 10 minutes, and this will be repeated up to three times. If there is still no response, the next of kin will be notified.

[1008] Output: Message to notify relatives

[1009] Step 13:

[1010] User: In an emergency, say "help" or press the emergency button.

[1011] Input: Emergency voice command or button press

[1012] Data processing and calculation: Recognize emergency commands and collect necessary data (such as location information).

[1013] Output: Emergency services and immediate family notification

[1014] Step 14:

[1015] Terminal: Recognizes emergency commands and makes emergency calls.

[1016] Input: User voice command or button press

[1017] Data processing and calculation: Obtain location information and automatically call 119 or 110.

[1018] Output: Emergency call message and location information

[1019] Step 15:

[1020] Relatives: Receive emergency notifications.

[1021] Input: Urgent notification sent from the server

[1022] Data processing and calculation: Display emergency notifications.

[1023] Output: Emergency notification displayed on a relative's device

[1024] Step 16:

[1025] User: Says to the device, "I want to order some groceries."

[1026] Input: User's voice command

[1027] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[1028] Step 17:

[1029] Device: Recognizes voice commands and sends requests to the server.

[1030] Input: User's speech converted to text

[1031] Data processing and calculation: Converts text data into a protocol for sending to the server.

[1032] Output: Request sent to the server

[1033] Step 18:

[1034] Server: Obtains a list of product candidates through the API of the online shopping system.

[1035] Input: The request sent by the user

[1036] Data processing and calculation: Obtain a list of product candidates using the YAHOO Shopping API.

[1037] Output: Sends the product candidate list to the terminal.

[1038] Step 19:

[1039] Terminal: Presents a list of product candidates to the user via voice.

[1040] Input: Product candidate list sent from the server

[1041] Data processing and calculation: Use Google Text-to-Speech to convert the product candidate list into a voice message.

[1042] Output: Provide the user with a list of products as a voice message

[1043] (Application example 1)

[1044] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1045] In elderly life support systems, there is a lack of means to promote daily conversations, communicate with relatives, respond to emergencies, and support purchasing daily necessities. Furthermore, conventional systems rely on the devices used by the elderly, and the operation of the devices is complicated, making it difficult for the elderly to use them. Furthermore, there is a lack of a mechanism that allows relatives in remote locations to easily understand the status of the elderly.

[1046] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1047] In this invention, the server includes a means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice; a means for a relative to remotely send a message to the terminal and deliver the message by voice to the user at a specified time; and a means for generating appropriate responses to the user's questions using a generative AI model. This allows elderly people to easily enjoy conversations and smoothly communicate with their relatives. Furthermore, the use of a generative AI model enables more advanced and flexible responses, allowing the elderly to receive the information they need in a timely manner.

[1048] "Weather information" refers to meteorological data such as the climate conditions, temperature, and probability of precipitation in the elderly person's place of residence.

[1049] "News information" refers to the latest news data on domestic and international events, incidents, culture, economy, etc.

[1050] "Hobbies and Preference Data" refers to data that compiles information related to the user's interests and areas of interest.

[1051] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to provide appropriate answers and generate conversations in response to questions and utterances from users.

[1052] "Providing audio" refers to a computer using speech synthesis technology to audibly convey information to a user.

[1053] "Relatives" refers to family members and blood relatives of the user.

[1054] "Sending a message from a remote location" refers to a relative sending a message to the user's terminal from a physically distant location via the Internet.

[1055] The "designated time" refers to a specific time set in advance by the relative at which the message should be conveyed to the user.

[1056] "Voice command" refers to an instruction or command given by a user through a microphone.

[1057] "Online shopping site" refers to a website for purchasing products over the Internet.

[1058] "User location information" refers to data indicating the user's current physical location.

[1059] A "central processing unit" refers to a computer that functions as the main computing resource of a server, etc., and is responsible for controlling the entire system and processing data.

[1060] "Emergency services" refers to public agencies and services that respond to emergencies, such as fire departments, police, and emergency medical services.

[1061] "Daily necessities" refers to consumables and food items necessary for the daily lives of the elderly.

[1062] This invention is a system for supporting the lives of elderly people, and is realized mainly through cooperation between a server, a terminal used by the user, and a terminal used remotely by a relative. This system has the following configuration and functions.

[1063] 1. Server:

[1064] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. Specifically, every morning at 6:50, it collects the latest information from the weather API, news API, and user hobby database, and sends it to the user's device. It also has the function of generating appropriate responses to user questions using a generative AI model. It can also receive messages from relatives and send them to the user's device at a specified time. The server also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[1065] 2. On the user's device:

[1066] The user device uses speech synthesis technology to send information to the user, recognizes the user's voice using its speech recognition function, and sends it to the server. If the user responds, "The book I've been reading recently is a historical novel," the device recognizes the speech and sends it to the server. It receives the response from the server and uses a generative AI model to generate an appropriate response, such as, "That's interesting. What period is it set in?" In an emergency, the user can issue a voice command such as "Help" or press an emergency button, and the device will automatically make an emergency call, obtain the user's location information, and provide it to medical services. It also works with online shopping sites, recognizing the voice command "I'd like to order daily necessities" and initiating the ordering process via the server.

[1067] 3. Family members' devices:

[1068] Relatives can remotely send messages to the elderly using devices such as smartphones or PCs. When the relative enters the message and the time to send it, the data is sent to a server, and the server sends the message to the user's device at the specified time. For example, a message such as "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready" can be spoken to the user.

[1069] Hardware and software used

[1070] Speech synthesis technology: pyttsx3

[1071] Speech recognition technology: Google Speech Recognition API

[1072] Weather API and News API: Includes any external services

[1073] Generative AI model: using natural language processing techniques

[1074] Specific use cases

[1075] An elderly person is asked at 7:00 a.m. from a terminal, "Good morning. It's a sunny day today. The temperature is 22 degrees. What book have you been reading recently?" If the user responds, "The book I've been reading recently is a historical novel," the server analyzes the data and uses a generative AI model to provide an appropriate response, such as, "That's interesting. What period is it set in?"

[1076] Prompt Sentence Examples

[1077] "Today's weather is sunny and the temperature is 22 degrees. Tell us about the book you've been reading recently."

[1078] This system will enable the elderly to enjoy everyday conversations and communicate smoothly with their relatives. It will also enable emergency response and assistance with purchasing everyday items, improving the quality of life for the elderly.

[1079] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1080] Step 1:

[1081] (Input) The server collects information from the weather API, news API, and hobby database.

[1082] (Processing) Every morning at 6:50, the server sends a request to an external service to collect data related to weather, news, and hobbies.

[1083] (Output) Collected weather information, news information, and hobby-related data.

[1084] (Specific operation) The server sends a request to the API, receives weather and news information in response, and searches and collects related information from the user's hobby database.

[1085] Step 2:

[1086] (Input) Information collected by the server.

[1087] (Processing) The server sends the collected information to the user's terminal.

[1088] (Output) Weather, news, and hobby-related information sent to the user's device.

[1089] (Specific Operation) The server formats the collected data and sends it to the user's device, which receives this information.

[1090] Step 3:

[1091] (Input) Information sent to the user's device.

[1092] (Processing) The terminal speaks to the user using voice synthesis technology.

[1093] (Output) A voice message to the user.

[1094] (Specific operation) The user's device converts information about the weather, news, and hobbies using a speech synthesis engine (pyttsx3) and speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1095] Step 4:

[1096] (Input) The user's response.

[1097] The (processing) terminal uses voice recognition technology to convert the user's response into text.

[1098] (Output) The user's response in text format.

[1099] (Specific operation) When the user responds, "The book I've been reading recently is a historical novel," the device converts the speech into text using the Google Speech Recognition API.

[1100] Step 5:

[1101] (Input) The user's response in text form.

[1102] (Processing) The terminal sends the user's response to the server.

[1103] (Output) The user's response sent to the server.

[1104] (Specific operation) The terminal sends the converted text to the server via an HTTP request.

[1105] Step 6:

[1106] (Input) The user's reply text.

[1107] The (processing) server uses a generative AI model to generate an appropriate response.

[1108] (Output) The generated response.

[1109] (Specific operation) The server uses a generative AI model to generate a response such as "That's interesting. What period is it set in?" in response to "The book I've been reading lately is a historical novel."

[1110] Step 7:

[1111] (Input) The generated response.

[1112] (Processing) The server sends the generated response to the user's terminal.

[1113] (Output) The generated reply to the user's terminal.

[1114] (Specific operation) The response generated by the server is formatted for transmission to the user's terminal and sent to the terminal via an HTTP request.

[1115] Step 8:

[1116] (Input) The response sent by the server.

[1117] The (processing) terminal uses speech synthesis technology to communicate the response to the user.

[1118] (Output) A spoken response to the user.

[1119] (Specific operation) The device converts the response received from the server using a speech synthesis engine (pyttsx3) and speaks to the user, saying, "That's interesting. What era are you talking about?"

[1120] Step 9:

[1121] (Input) Message content and time sent by relatives.

[1122] (Processing) A relative uses a smartphone or computer from a remote location to send the message content and sending time to the server.

[1123] (Output) Message content and sending time stored on the server.

[1124] (Specific operation) A relative uses the application to enter the message content and sending time, and sends it to the server.

[1125] Step 10:

[1126] (Input) Message content and time sent from relatives.

[1127] (Processing) The server sends a message to the user's terminal at the specified time.

[1128] (Output) Messages from relatives to the user's terminal.

[1129] (Specific operation) The server monitors the specified time, and when the set time arrives, it sends a message to the user's terminal.

[1130] Step 11:

[1131] (Input) Message from relatives from the server.

[1132] (Processing) The terminal uses voice synthesis technology to convey the message to the user.

[1133] (Output) A voice message to the user.

[1134] (Specific operation) The terminal converts the message received from the server using a speech synthesis engine (pyttsx3) and tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[1135] Step 12:

[1136] (Input) If there is no response from the user.

[1137] (Processing) The terminal retries a certain number of times, and if there is no response, it notifies the server.

[1138] (Output) Notifies the server of the user's response status.

[1139] (Specific operation) The terminal waits for a response from the user, and if there is still no response, it retries a certain number of times and sends the message again, and if there is still no response, it notifies the server.

[1140] Step 13:

[1141] (Input) User response status notification received by the server.

[1142] (Processing) The server notifies the relatives of the user's response status.

[1143] (Output) Notify relatives of the user's response status.

[1144] (Specific operation) The server sends emails and application notifications to relatives to report the user's response status.

[1145] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1146] This invention is a system for supporting the lives of the elderly and promoting conversation. This system not only provides weather information, news information, and topics based on the user's hobbies and preferences, but also incorporates an emotion engine to recognize the user's emotions and achieve more appropriate communication. The following describes each component of this system and the program processing.

[1147] Overall system overview

[1148] 1. Server:

[1149] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time. In addition, it analyzes the user's emotional data via an emotion engine and generates feedback based on this.

[1150] 2. Terminal:

[1151] The user's device uses voice synthesis technology to send information to the user. It also has the function of recognizing the user's voice and sending response data to the server. In an emergency, an emergency call can be made using voice commands or a button. It also has a function that links with online shopping sites to assist with purchasing daily necessities. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the content of the conversation based on the results.

[1152] 3. Family members' devices:

[1153] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[1154] Program processing

[1155] Daily conversation promotion features

[1156] server:

[1157] Every morning, the server collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[1158] Device:

[1159] Every morning, the device uses voice synthesis technology to analyze the latest information received from the server and speaks to the user, such as, "Good morning. It's sunny today. The temperature is 22 degrees. How's the book you've been reading recently?" The emotion engine also analyzes the user's emotions at this point and adjusts the tone and content of the conversation.

[1160] User:

[1161] The user replies, "Good morning. I've been reading historical novels lately."

[1162] Device:

[1163] The device recognizes the user's response through voice recognition, performs emotional analysis using an emotion engine, and then sends the results to the server.

[1164] server:

[1165] The server analyzes the user's response and emotional data, generates appropriate response data, and sends it to the terminal.

[1166] Device:

[1167] Based on the received response data and emotion data, the device responds to the user by saying, "That's interesting. What era are you talking about?"

[1168] Message function from relatives

[1169] relatives:

[1170] The relative opens the smartphone app and enters the message content and desired delivery time.

[1171] server:

[1172] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[1173] Device:

[1174] The message will arrive at the specified time and tell the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready." At this time, the emotion engine will analyze the user's reaction and provide feedback on the emotion to the relatives.

[1175] User:

[1176] The user responds, "Okay."

[1177] Device:

[1178] The device recognizes the user's response and sends it to the server. If there is no response, it retries a certain number of times, and if there is still no response, it notifies the server.

[1179] server:

[1180] If the server does not respond for a certain period of time, it will send a notification to the next of kin.

[1181] Emergency call function

[1182] User:

[1183] In an emergency, you can either say "help" to the device or press the emergency button.

[1184] Device:

[1185] The device detects an emergency command or button trigger and automatically calls 119 or 110. At the same time, emotional data is also sent to the server, and relatives are notified.

[1186] Online shopping integration function

[1187] User:

[1188] The user speaks to the terminal saying, "I would like to order daily necessities."

[1189] Device:

[1190] The device analyzes the user's voice commands using a voice recognition engine and emotion engine, and sends a request to the server.

[1191] server:

[1192] The server initiates the order process through the online shopping API and sends the order confirmation information to the terminal.

[1193] Device:

[1194] The terminal will then notify the user that the order has been completed and analyze the user's emotions during the process, providing appropriate feedback.

[1195] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. It combines functions for daily conversations, emergency response, and online shopping support, improving the quality of life for elderly people.

[1196] The processing flow will be explained below.

[1197] Daily conversation promotion features

[1198] Server-side processing

[1199] Step 1:

[1200] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[1201] Step 2:

[1202] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[1203] Step 3:

[1204] The server transmits the acquired data to each user's terminal.

[1205] Terminal side processing

[1206] Step 1:

[1207] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[1208] Step 2:

[1209] The terminal analyzes the received information and generates content that speaks to the user.

[1210] Step 3:

[1211] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1212] User processing

[1213] Step 1:

[1214] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[1215] Terminal side processing

[1216] Step 1:

[1217] The device recognizes the user's response through voice recognition, analyzes the emotion using an emotion engine, and then sends the results to the server.

[1218] Server-side processing

[1219] Step 1:

[1220] The server analyzes the user's response content and emotional data, generates appropriate response data, and sends it to the terminal.

[1221] Terminal side processing

[1222] Step 1:

[1223] The device analyzes the received response data and emotion data and responds to the user, "That's interesting. What era are you talking about?"

[1224] Message function from relatives

[1225] Processing for relatives

[1226] Step 1:

[1227] The relative opens the smartphone app and enters the message content and desired delivery time.

[1228] Step 2:

[1229] The relative taps the "Send" button to send the message to the server.

[1230] Server-side processing

[1231] Step 1:

[1232] The server stores the message received from the relative and the specified time in a database.

[1233] Step 2:

[1234] At the specified time, the server sends a message to the corresponding user's terminal.

[1235] Terminal side processing

[1236] Step 1:

[1237] The terminal receives a message from the server at a specified time.

[1238] Step 2:

[1239] The device then audibly conveys the received message to the user, saying, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[1240] Step 3:

[1241] The device uses an emotion engine to analyze the user's reaction and transmits the emotion data to the server.

[1242] User processing

[1243] Step 1:

[1244] The user responds, "Okay."

[1245] Terminal side processing

[1246] Step 1:

[1247] The terminal recognizes the user's response and sends it to the server.

[1248] Step 2:

[1249] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[1250] Step 3:

[1251] If the device does not receive a response after three retries, it notifies the server.

[1252] Server-side processing

[1253] Step 1:

[1254] The server detects the user's lack of response and sends a notification to the next of kin.

[1255] Emergency call function

[1256] User processing

[1257] Step 1:

[1258] In an emergency, the user can either say "help" to the device or press the emergency button.

[1259] Terminal side processing

[1260] Step 1:

[1261] The device recognizes emergency commands or button triggers.

[1262] Step 2:

[1263] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[1264] Step 3:

[1265] The device will send a notification to relatives as soon as the emergency call is completed.

[1266] Online shopping integration function

[1267] User processing

[1268] Step 1:

[1269] The user speaks to the terminal saying, "I would like to order daily necessities."

[1270] Terminal side processing

[1271] Step 1:

[1272] The device analyzes the user's voice commands using a voice recognition engine and emotion engine, and sends a request to the server.

[1273] Step 2:

[1274] The terminal presents the candidate product list received from the server to the user by voice.

[1275] Step 3:

[1276] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[1277] Server-side processing

[1278] Step 1:

[1279] The server initiates the order process through an online shopping API.

[1280] Step 2:

[1281] Once the order is complete, the server sends confirmation information to the terminal.

[1282] Terminal side processing

[1283] Step 1:

[1284] The device notifies the user by voice when the order is complete, and transmits the emotion data obtained during the process to the server to provide appropriate feedback to the relatives.

[1285] Example 2

[1286] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1287] In modern society, elderly people often feel lonely, and the limited opportunities for daily conversation and communication often have a negative impact on their mental and psychological health. It is also difficult to contact relatives who live far away, making it difficult to detect abnormalities or emergencies in the elderly in a timely manner. Furthermore, it can be difficult to purchase daily necessities, raising concerns about a decline in quality of life.

[1288] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1289] In this invention, the server includes means for generating weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice, means for a relative to send a message to the terminal from a remote location and deliver the message by voice to the user at a specified time, and means for analyzing the user's emotion data using an emotion recognition engine and adjusting the tone and content of the conversation. This allows the user to promote daily conversations, enable appropriate communication with relatives even when they are far away, and make it possible to respond to emergencies and purchase daily necessities smoothly.

[1290] "Weather information" refers to information such as temperature, precipitation, wind speed, and weather conditions provided based on meteorological data.

[1291] "News information" refers to reports and articles about the latest events and social topics.

[1292] "Hobbies and preferences" is information related to activities and topics that an individual is interested in and engages in for fun.

[1293] "Users" refer to the elderly people who use this system.

[1294] "Remote location" refers to a location that is physically separate, and typically refers to a case where the user and their relatives are in different locations.

[1295] "Message" refers to a message that a relative wants to convey to the user.

[1296] A "voice command" is a method in which a user issues instructions to a terminal using voice.

[1297] An "emotion recognition engine" is a technology for detecting and analyzing emotions from a user's voice and facial expressions.

[1298] "Emergency call" is a means of contacting a user to request help in an emergency.

[1299] An "online shopping site" is a website where you can purchase goods and services over the Internet.

[1300] "Speech synthesis technology" is a technology for converting text data into voice data.

[1301] A "voice recognition engine" is a technology for converting voice data into text data.

[1302] This invention relates to a system for supporting the lives of the elderly and promoting conversation. This system provides weather information, news information, and topics based on the user's hobbies and preferences, and also uses an emotion recognition engine to achieve more appropriate communication.

[1303] Overall system configuration

[1304] 1. Server:

[1305] We use weather APIs (e.g., OpenWeatherMap API) and news APIs (e.g., NewsAPI) to periodically collect data related to weather information, news information, and hobbies and preferences.

[1306] The system receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response for a certain period of time.

[1307] An emotion recognition engine (e.g., Affectiva, IBM Watson Tone Analyzer) is used to analyze the user's emotion data and generate feedback.

[1308] 2. Terminal:

[1309] Using speech synthesis technology (e.g., Google Text-to-Speech API), information is transmitted to the user via voice.

[1310] A speech recognition engine (e.g., Google Speech-to-Text API) is used to recognize the user's voice and send response data to the server.

[1311] In an emergency, use voice commands or press a button to make an emergency call.

[1312] It works in conjunction with online shopping sites and has the ability to assist with purchasing daily necessities using voice commands.

[1313] An emotion recognition engine is used to recognize emotions from the user's voice and facial expressions and adjust the content of the conversation.

[1314] 3. Family members' devices:

[1315] Using a smartphone or personal computer, messages can be sent to elderly people from remote locations, and notifications can be received in the event of an emergency or when safety confirmation is required.

[1316] Specific examples

[1317] Daily conversation promotion features

[1318] server:

[1319] Every morning, the server collects the latest information from weather APIs, news APIs, and the user's hobby database and sends it to the device.

[1320] Device:

[1321] Every morning, the device receives the latest information from the server, analyzes it using speech synthesis technology, and speaks to the user. Example: "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1322] User:

[1323] The user replies, "Good morning. I've been reading historical novels lately."

[1324] Device:

[1325] The device converts the user's response into text using a voice recognition engine, analyzes the emotion using an emotion recognition engine, and sends the results to the server.

[1326] server:

[1327] The server generates an appropriate response based on the user's response and emotional data and sends it to the device. Example: "That's interesting. What era are you talking about?"

[1328] Device:

[1329] The device uses voice synthesis technology to communicate with the user based on the received response and emotional data. Example: "That's interesting. What era are you talking about?"

[1330] Message function from relatives

[1331] relatives:

[1332] The relative opens the smartphone app and enters the message and desired delivery time. For example, "Please let them know I have a doctor's appointment tomorrow at 9:00 AM."

[1333] server:

[1334] The server stores the message content and the specified time received from the relative, and sends the message to the corresponding user's terminal when the specified time arrives.

[1335] Device:

[1336] The device receives the message at the specified time and uses speech synthesis technology to convey it to the user. For example, "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready." An emotion recognition engine analyzes the user's reaction and provides feedback on that emotion to the relative.

[1337] Emergency call function

[1338] User:

[1339] In an emergency, you can either say "help" to the device or press the emergency button.

[1340] Device:

[1341] The device will detect the emergency command or button trigger and immediately make an automatic call to 119 or 110. Example: "An elderly person needs help."

[1342] The device sends the emotional data along with the report to a server, and notifies relatives.

[1343] Online shopping integration function

[1344] User:

[1345] The user speaks to the terminal saying, "I would like to order daily necessities."

[1346] Device:

[1347] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[1348] server:

[1349] The server initiates the order process through an online shopping API. Example: "Your grocery order has been initiated."

[1350] Device:

[1351] The device notifies the user of order confirmation information from the server using voice synthesis technology and provides emotional feedback. Example: "Your order has been completed."

[1352] Prompt Sentence Examples

[1353] Based on the following conversation, please suggest topics that might interest users next:

[1354] User: "The book I've been reading lately is a historical novel."

[1355] System: "That's interesting. What era are you talking about?"

[1356] User: "It's from the Meiji period."

[1357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1358] Specific explanation of processing steps

[1359] Daily conversation promotion features

[1360] server

[1361] Step 1:

[1362] The server collects weather information from a weather API at a fixed time every morning. The specific input is the request data to the API, and the output is the retrieved weather information. It also obtains the latest news information from a news API and extracts topics based on the user's hobbies and preferences from a hobby database.

[1363] Step 2:

[1364] The server integrates the collected weather information, news information, and hobby data into a JSON format and sends this data package to the terminal. The input data is each piece of collected information, and the output is the compiled JSON data.

[1365] Terminal

[1366] Step 3:

[1367] The device analyzes the JSON data received from the server. During analysis, it uses speech synthesis technology (e.g., Google Text-to-Speech API) to convert the transmitted information into voice data. Specifically, it converts text input into voice data and speaks to the user. Example: "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1368] User

[1369] Step 4:

[1370] The user responds to the voice guidance from the terminal. For example, they might say, "Good morning. The book I've been reading recently is a historical novel." This user's voice data becomes the input.

[1371] Terminal

[1372] Step 5:

[1373] The device converts the user's response into text data using a speech recognition engine (e.g., Google Speech-to-Text API). It then uses an emotion recognition engine to analyze emotional data from the user's voice. The input is the user's voice data, and the output is the recognized text data and emotional data.

[1374] Step 6:

[1375] The device sends the obtained text data and emotion data to the server. The input is the analysis results (text data and emotion data) on the device side, which are then sent to the server.

[1376] server

[1377] Step 7:

[1378] The server analyzes the received user response and emotional data. Specifically, it uses a natural language processing (NLP) model (e.g., OpenAI GPT-3) to generate an appropriate response. The input is the user response and emotional data, and the output is the generated response. Example: "That's interesting. What era are you talking about?"

[1379] Step 8:

[1380] The server sends the generated response along with the emotion data to the terminal. The input is the response content and emotion data, and the output is the transmitted data.

[1381] Terminal

[1382] Step 9:

[1383] The device receives the response and emotion data from the server and again uses speech synthesis technology to speak to the user. Example: "That's interesting. What era are you talking about?" The input data are the received response and emotion data, and the output is voice data.

[1384] Message function from relatives

[1385] relatives

[1386] Step 1:

[1387] The relative opens the smartphone app and inputs the message content and desired sending time. For example, "Please tell them I have a doctor's appointment tomorrow at 9:00 AM."

[1388] server

[1389] Step 2:

[1390] The server stores the message content received from the relatives and the specified sending time in a database. The input is the message content and sending time, and the output is the stored data.

[1391] Step 3:

[1392] When the designated sending time arrives, the server retrieves the stored message data and sends the message to the corresponding user's terminal. The input is the sending time and message data, and the output is the message to be sent.

[1393] Terminal

[1394] Step 4:

[1395] The device receives the message from the server at the specified time and conveys it to the user using speech synthesis technology. Example: "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready." At this time, the emotion recognition engine analyzes the user's reaction and generates emotional data. The input is the message, and the output is voice data and emotional data.

[1396] User

[1397] Step 5:

[1398] The user responds, "I understand." This voice data is the input.

[1399] Terminal

[1400] Step 6:

[1401] The device converts the user's response into text data using a speech recognition engine and sends it to the server. At the same time, emotion data is also sent to the server. The input is the user's voice data, and the output is text data and emotion data.

[1402] server

[1403] Step 7:

[1404] If there is no response for a certain period of time, the server will retry, and if there is still no response, it will send a notification to the relatives. The input is the response status, and the output is the notification message.

[1405] Emergency call function

[1406] User

[1407] Step 1:

[1408] The user can say "help" or press the emergency button on the terminal. This voice data or button down is the input.

[1409] Terminal

[1410] Step 2:

[1411] The device detects an emergency command or button trigger and immediately and automatically calls 119 or 110. Specifically, it uses voice synthesis technology to automatically generate the message content. The input is the emergency command or button trigger, and the output is the message data.

[1412] Step 3:

[1413] Along with the emergency call, the device sends emotional data to the server and notifies relatives. The input is the emergency call and emotional data, and the output is the transmitted data.

[1414] Online shopping integration function

[1415] User

[1416] Step 1:

[1417] The user speaks to the terminal, saying, "I would like to order daily necessities." This voice data is the input.

[1418] Terminal

[1419] Step 2:

[1420] The device analyzes the user's voice commands using a voice recognition engine and sends a request to the server. The input is voice data and the output is the voice recognition result.

[1421] server

[1422] Step 3:

[1423] The server initiates the order process through an online shopping API, placing the order with the necessary authentication information and product data. The input is the speech recognition result, and the output is the order confirmation data.

[1424] Step 4:

[1425] The server sends order confirmation information to the terminal. The input is the order confirmation data and the output is the transmission data.

[1426] Terminal

[1427] Step 5:

[1428] The device receives order confirmation information from the server and notifies the user using voice synthesis technology. For example, "Your order has been completed." At the same time, emotional feedback is provided. The input is order confirmation data, and the output is voice data and emotional feedback.

[1429] (Application example 2)

[1430] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1431] Isolation and lack of communication among the elderly are serious problems in modern society. It is also difficult for relatives living far away to keep track of the elderly's lifestyle and safety, which can delay emergency response. It is also difficult for elderly people to obtain timely entertainment and information that suits their emotions and moods. There is a need for a system that can solve these issues and improve the quality of life for the elderly.

[1432] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather information and news information and generating topics taking into account the user's hobbies and preferences, means for performing emotion analysis and recommending content suitable for the user, and means for relatives to send messages from remote locations and deliver them to the user at a specified time. This prevents the elderly from becoming isolated, enables appropriate communication to be maintained, and makes it easier for relatives to understand the elderly's condition even from remote locations. It also makes it possible to provide appropriate support and comfortable entertainment in emergencies and in daily life.

[1433] "Weather information" is data relating to the current weather conditions and forecasted weather conditions in the user's place of residence.

[1434] "News information" refers to the latest information and news reports about current events and social topics.

[1435] "User hobbies and preferences" is data about the activities and interests of the user.

[1436] "Emotion analysis" is the process of identifying and assessing a user's emotional state from their voice and facial expressions.

[1437] "Content" means digital data that provides information or entertainment, such as music, video, news, and articles.

[1438] "Relatives" refers to family members or relatives who are related by blood or legal relationship to the user.

[1439] An "emergency" is when a user is faced with a serious health or safety situation.

[1440] A "voice command" is an instruction that a user gives to a system using voice.

[1441] An "online shopping site" is a website for purchasing goods and services over the Internet.

[1442] "Server" means a computer system that collects, processes, and transmits data over a network.

[1443] A "terminal" is a device that a user can directly operate (such as a smartphone, smart glasses, a head-mounted display, or a robot).

[1444] This invention is a system for supporting the lives of the elderly and promoting conversation. This system provides weather information, news information, and topics based on the user's hobbies and preferences, and uses an emotion engine to evaluate the user's emotional state and provide appropriate content. Below, we will clearly explain the program processes and hardware and software configurations required to implement this invention.

[1445] server

[1446] A server is a computer system connected to the Internet that has the following functions:

[1447] 1. Data collection: Data is collected periodically from weather information API, news information API, and a database of user hobbies.

[1448] 2. Sentiment analysis: The user's voice data is sent to the sentiment analysis engine to evaluate the emotional state.

[1449] 3. Data transmission: Collected data and analysis results are sent to the device.

[1450] 4. Family collaboration: Receive messages and notifications from relatives and forward them to the appropriate device at the specified time.

[1451] 5. Emergency Calls: Receive emergency calls from users and contact emergency services as needed.

[1452] As a concrete example, every morning the server obtains the current weather conditions from a weather information API, collects news and topics based on the user's hobbies and preferences, and sends them to the device. At this time, the server analyzes the user's voice data from the previous day using an emotion analysis engine, and selects and provides topics that suit the user's mood.

[1453] Terminal

[1454] The device operated by the user (smartphone, smart glasses, head-mounted display, or robot) has the following functions:

[1455] 1. Voice recognition: Recognizes the user's voice instructions and sends the data to the server.

[1456] 2. Speech synthesis: The data received from the server is presented to the user as voice.

[1457] 3. Emotion analysis: Analyze the user's voice and facial expressions using the built-in emotion analysis engine.

[1458] 4. Message from relatives: A voice message from a relative will be delivered at the specified time.

[1459] 5. Emergency Call: In case of an emergency, the device receives voice commands or button inputs from the user and makes an emergency call.

[1460] As a concrete example, the device will speak to the user every morning saying, "Good morning. It's a sunny day today. How do you like the novel you've been reading recently?" and will then analyze the user's response using an emotion analysis engine to provide more appropriate topics of conversation.

[1461] User

[1462] Users can give voice instructions to the device and receive information. They can receive messages from relatives or purchase daily necessities using voice commands. In an emergency, they can also make an emergency call by saying "help" to the device.

[1463] relatives

[1464] Relatives can use smartphones or computers to send messages to the elderly and receive notifications in emergencies and regular safety checks, making it easier for relatives to keep track of the elderly's living conditions even when they are far away.

[1465] Example prompt sentence:

[1466] "Please wait as we are currently conducting an ongoing sentiment analysis."

[1467] "We recommend relaxing music to suit your mood."

[1468] "Good morning. It's a beautiful day today. How's the novel you've been reading lately?"

[1469] This will make it possible to provide an environment where elderly people can spend their days comfortably and safely.

[1470] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1471] Step 1:

[1472] Data collection (server)

[1473] The server collects the latest information from the weather information API, news information API, and user interest and preference database. Current weather data is obtained from the weather information API, and the latest news articles are collected from the news information API. Data related to the user's areas of interest is extracted from the user interest and preference database. This data is temporarily stored on the server.

[1474] Input: Weather information API, news information API, hobby and preference database

[1475] Output: Latest weather data, latest news data, hobby and preference data

[1476] Step 2:

[1477] Sentiment analysis (server)

[1478] The server sends the user's voice and text data to the emotion analysis engine to evaluate the user's emotional state. For example, if the user says "I'm tired," the emotion analysis engine will identify this as "fatigue" and return the result to the server.

[1479] Input: User voice or text data

[1480] Output: Emotion analysis results (e.g., fatigue)

[1481] Step 3:

[1482] Data transmission (server)

[1483] The server compiles the collected weather information, news information, user hobby data, and emotion analysis results and sends them to the user's device. This data becomes material for promoting daily conversations between users.

[1484] Input: Weather data, news data, hobby and preference data, emotion analysis results

[1485] Output: Consolidated data to send to the device

[1486] Step 4:

[1487] Speech synthesis and provision (terminal)

[1488] The device analyzes the data received from the server and provides it to the user using speech synthesis technology. For example, it might say something like, "Good morning. It's a sunny day today. How's the novel you've been reading recently?"

[1489] Input: Integrated data (weather data, news data, hobby and preference data, emotion analysis results)

[1490] Output: The audio message to be presented to the user

[1491] Step 5:

[1492] User response and reanalysis (terminal)

[1493] The user responds to the device, for example, saying, "The novel I've been reading recently is very interesting." The device recognizes this speech and sends it to the server. At the same time, the emotion analysis engine reanalyzes the user's emotional state.

[1494] Input: User's voice response

[1495] Output: Voice data sent to the server, reanalyzed emotional state

[1496] Step 6:

[1497] Message notification from relatives (server and terminal)

[1498] The server sends the message received from the relative to the terminal at the specified time. The terminal then delivers the message to the user by voice. For example, the terminal may say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[1499] Input: Message from relatives, specified time

[1500] Output: The audio message to be presented to the user

[1501] Step 7:

[1502] Emergency call (terminal)

[1503] In an emergency, the user can either say "help" to the device or press the emergency button, and the device will make an emergency call. The call will also include the user's location information.

[1504] Input: User's voice command or button press

[1505] Output: Emergency call with location information attached

[1506] Step 8:

[1507] Online shopping (terminal)

[1508] The user speaks to the device, saying, "I'd like to order some daily necessities." The device recognizes this command and sends it to the server. The server processes the order through the online shopping API and sends a confirmation to the device.

[1509] Input: User's voice command

[1510] Output: Online shopping order confirmation information

[1511] This will realize a system that can take into account the user's emotional state and provide appropriate information and communication.

[1512] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1513] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1514] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1515] [Third embodiment]

[1516] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1517] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1518] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1519] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1520] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1521] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1522] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1523] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1524] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1525] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1526] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1527] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1528] This invention is a system for supporting the lives of the elderly, aiming to promote daily conversations and facilitate communication with relatives. The system mainly includes the following elements: a server, a terminal used by the user, and a terminal used remotely by relatives.

[1529] Overall system overview

[1530] 1. Server:

[1531] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides them to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[1532] 2. Terminal:

[1533] The user's device uses voice synthesis technology to send information to the user. It also has the ability to recognize the user's voice and send response data to the server. In an emergency, an emergency call can be made using voice commands or a button. The device also has a function that links with online shopping sites to help with purchasing daily necessities.

[1534] 3. Family members' devices:

[1535] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[1536] Program processing

[1537] Daily conversation promotion features

[1538] server:

[1539] At 6:50 in the morning, it collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[1540] Device:

[1541] At 7:00 a.m., the latest information received from the server is analyzed using voice synthesis technology, and the system speaks to the user, saying things like, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1542] User:

[1543] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[1544] Device:

[1545] The system recognizes the user's response and sends the analysis results to the server. Based on the response received from the server, the system responds with "That's interesting. What era are you talking about?"

[1546] Message function from relatives

[1547] relatives:

[1548] The relative opens the smartphone app, enters the message content and desired sending time, and sends it to the server.

[1549] server:

[1550] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[1551] Device:

[1552] A message will be received at the specified time, telling the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[1553] User:

[1554] The user responds, "Okay."

[1555] Device:

[1556] The user's response is sent to the server. If there is no response, the message is sent again after 10 minutes, and this is repeated up to three times. If there is still no response, the server is notified.

[1557] server:

[1558] If the user does not respond and there is still no response after a certain number of retries, a notification is sent to the next of kin.

[1559] Emergency call function

[1560] User:

[1561] In an emergency, you can either say "help" to the device or press the emergency button.

[1562] Device:

[1563] When an emergency command is recognized, it will automatically call 119 or 110 and simultaneously send the user's location information. It will also notify relatives.

[1564] Online shopping integration function

[1565] User:

[1566] Speak to the device and say, "I'd like to order daily necessities."

[1567] Device:

[1568] It recognizes voice commands, sends requests to the server, receives a list of product candidates from the server, and presents them to the user via voice.

[1569] server:

[1570] The ordering process is initiated through the YAHOO Shopping API and order confirmation information is sent back to the terminal.

[1571] Device:

[1572] The user is notified by voice that the order has been completed.

[1573] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. By integrating functions for daily conversations, emergency response, and online shopping support, it contributes to improving the quality of life for elderly people.

[1574] The processing flow will be explained below.

[1575] Daily conversation promotion features

[1576] Server-side processing

[1577] Step 1:

[1578] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[1579] Step 2:

[1580] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[1581] Step 3:

[1582] The server transmits the acquired data to each user's terminal.

[1583] Terminal side processing

[1584] Step 1:

[1585] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[1586] Step 2:

[1587] The terminal analyzes the received information and generates content that speaks to the user.

[1588] Step 3:

[1589] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1590] User processing

[1591] Step 1:

[1592] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[1593] Terminal side processing

[1594] Step 1:

[1595] The device performs voice recognition on the user's response and sends the analysis results to the server.

[1596] Step 2:

[1597] The server analyzes the user's response, generates response data, and sends it to the terminal.

[1598] Step 3:

[1599] The device analyzes the received response data and responds to the user, "That's interesting. What era are you talking about?"

[1600] Message function from relatives

[1601] Processing for relatives

[1602] Step 1:

[1603] The relative opens the smartphone app and enters the message content and desired delivery time.

[1604] Step 2:

[1605] The relative taps the "Send" button to send the message to the server.

[1606] Server-side processing

[1607] Step 1:

[1608] The server stores the message received from the relative and the specified time in a database.

[1609] Step 2:

[1610] At the specified time, the server sends a message to the corresponding user's terminal.

[1611] Terminal side processing

[1612] Step 1:

[1613] The terminal receives a message from the server at a specified time.

[1614] Step 2:

[1615] The terminal will then audibly convey the received message to the user.

[1616] Step 3:

[1617] The terminal tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[1618] User processing

[1619] Step 1:

[1620] The user responds, "Okay."

[1621] Terminal side processing

[1622] Step 1:

[1623] The terminal recognizes the user's response and sends it to the server.

[1624] Step 2:

[1625] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[1626] Step 3:

[1627] If the device does not receive a response after three retries, it notifies the server.

[1628] Server-side processing

[1629] Step 1:

[1630] The server detects the user's lack of response and sends a notification to the next of kin.

[1631] Emergency call function

[1632] User processing

[1633] Step 1:

[1634] In an emergency, the user can either say "help" to the device or press the emergency button.

[1635] Terminal side processing

[1636] Step 1:

[1637] The device recognizes emergency commands or button triggers.

[1638] Step 2:

[1639] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[1640] Step 3:

[1641] The device will send a notification to relatives as soon as the emergency call is completed.

[1642] Online shopping integration function

[1643] User processing

[1644] Step 1:

[1645] The user speaks to the terminal saying, "I would like to order daily necessities."

[1646] Terminal side processing

[1647] Step 1:

[1648] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[1649] Step 2:

[1650] The terminal presents the candidate product list received from the server to the user by voice.

[1651] Step 3:

[1652] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[1653] Server-side processing

[1654] Step 1:

[1655] The server initiates the order process through the YAHOO Shopping API.

[1656] Step 2:

[1657] Once the order is complete, the server sends confirmation information to the terminal.

[1658] Terminal side processing

[1659] Step 1:

[1660] The terminal will notify the user by voice that the order has been completed.

[1661] Example 1

[1662] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1663] It is becoming increasingly difficult for elderly people to maintain daily contact with society, leading to increased isolation and health risks. It is also difficult for relatives who live far away to constantly check on the safety of their elderly relatives. Furthermore, it is difficult for elderly people to find a way to respond quickly in an emergency or to purchase daily necessities.

[1664] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1665] In this invention, the server includes: means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user via voice; means for a relative to remotely send a message to the terminal and deliver the message via voice to the user at a specified time; means for retrying a certain number of times if there is no response from the user and notifying the relative if there is still no response; means for retrying via voice if there is no response for a long period of time and notifying the relative if there is still no response; means for the user to make an emergency call using a voice command or button in an emergency; means for connecting with an online shopping system to purchase daily necessities using voice commands; means for collecting various information at a specified time and transmitting it to the terminal; means for voice recognition of the user's response, transmitting it to the server, analyzing the response from the server, and responding; and means for notifying the relative if there is no conversation for a certain period of time. This allows the elderly to maintain daily information and communication, allowing relatives to easily check the elderly's safety. It also enables quick response in emergencies and smooth purchasing of daily necessities.

[1666] "Weather information" is data about current and forecast atmospheric conditions, such as weather, temperature, precipitation, and wind speed.

[1667] "News information" refers to the latest reports on events and topics in various fields, including society, politics, economics, and culture.

[1668] "User's hobbies and preferences" refers to data and information related to the user's interests, favorite activities, favorite items, and the like.

[1669] "Means of providing by voice" is a function that uses voice synthesis technology to convey text or data to the user as voice output.

[1670] "Relatives" refers to family members or close friends of the user who are interested in the user's life and well-being.

[1671] A "message" is a word or content conveyed in the form of a message, and is information sent primarily from relatives to the user.

[1672] The "means for retrying if there is no response" is a function for sending the same message again if the user does not respond to a specified message.

[1673] "Means to call emergency services" is the ability for a user to contact emergency services via voice command or physical button in the event of an emergency.

[1674] An "online shopping system" is a system that allows you to search for and select products and complete the purchasing process via the Internet.

[1675] "Means of collecting various information" refers to the function of obtaining weather information, news information, hobby-related information, etc. from external data sources.

[1676] The "means for recognizing voice and responding" is a function that converts the user's voice into text data, sends it to the server, and generates an appropriate voice message based on the analysis results.

[1677] The "means for notifying when there is no conversation for a certain period of time" is a function that notifies relatives of the situation when the user does not talk to the system for a certain period of time.

[1678] This invention is a system for supporting the daily lives of elderly people, and aims to promote daily conversations and facilitate communication with relatives. The system includes a server, a terminal used by the user, and a terminal used by relatives remotely.

[1679] server

[1680] The server is responsible for periodically collecting information from multiple external data sources and providing it to the user's device. Specifically, it uses the OpenWeatherMap API as a weather API to obtain weather information, and the News API as a news API to obtain news information. In addition, information based on the user's hobbies and preferences is obtained from the user's hobby database. This information is collected at a specified time (for example, 6:50 every morning) and sent to the user's device.

[1681] In addition, the server receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response after a certain number of attempts. Furthermore, if there is no response for a long period of time, it retries by voice and eventually notifies the relatives.

[1682] User's device

[1683] The user's device is equipped with speech synthesis technology (e.g., Google Text-to-Speech) and speech recognition technology (e.g., Google Speech-to-Text). This device has the ability to convey information received from the server to the user by voice. For example, at 7:00 in the morning, the device might say, "Good morning. It's a sunny day today. The temperature is 22 degrees. How is the book you've been reading recently?"

[1684] When the user responds to this question, the device uses voice recognition technology to convert the user's response into text data and sends it to the server. The server then generates an appropriate response based on the received data and sends the result to the device. The device then uses voice synthesis technology to respond to the user, saying, "That's interesting. What era are you talking about?"

[1685] Relatives' devices

[1686] Relatives can use devices such as smartphones and PCs to send messages to elderly people from remote locations. Using a dedicated application, the relative inputs the message content and desired sending time, and sends it to the server. The server then sends the message to the user's device at the specified time and conveys it to the user by voice. For example, the user's device might say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[1687] Emergency call function

[1688] In an emergency, if the user says "help" or presses the emergency button, the device will recognize the emergency command and automatically call 119 or 110. At the same time, the user's location information will also be sent to the emergency services. An emergency notification will also be sent to relatives.

[1689] Online shopping integration function

[1690] When a user says to the terminal, "I would like to order daily necessities," the terminal recognizes the voice command and sends a request to the server. The server obtains a list of candidate products through the online shopping system's API (for example, YAHOO Shopping API) and sends it to the terminal. The terminal presents the candidate product list to the user by voice, recognizes the user's selection, and sends it back to the server. The server executes the ordering process based on the user's selection and returns order confirmation information to the terminal. The terminal then notifies the user by voice that the order has been completed.

[1691] The system will enable elderly people to maintain daily information and communication, allow relatives to easily check on their safety, and facilitate rapid response in emergencies and the smooth purchase of daily necessities.

[1692] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1693] Step 1:

[1694] Server: Collects weather information, news information, and user hobbies and preferences every morning at 6:50.

[1695] Inputs: OpenWeatherMap API, NewsAPI, Hobby Database

[1696] Data processing and calculation: Use API to get current weather and latest news. Get the latest information relevant to the user from the hobby database.

[1697] Output: Generates a set of weather, news, and interest information and prepares it for transmission to the user's device.

[1698] Step 2:

[1699] Terminal: Information received at 7:00 a.m. is analyzed using voice synthesis technology and provided to the user.

[1700] Input: A set of weather information, news information, and hobby / preference information received from the server

[1701] Data processing and calculation: Convert received information into a voice message using Google Text-to-Speech.

[1702] Output: Provide the user with a voice message saying, "Good morning. It's a sunny day today. The temperature is 72 degrees. How's your reading going?"

[1703] Step 3:

[1704] User: Responds to the device by saying, "Good morning. I've been reading historical novels lately."

[1705] Input: Voice response to terminal

[1706] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[1707] Output: Generate the converted text data "Good morning. The book I'm reading recently is a historical novel." and prepare to send it to the server.

[1708] Step 4:

[1709] Terminal: Sends the user's voice response to the server.

[1710] Input: User's speech converted to text

[1711] Data processing and calculation: Converts text data into a protocol for sending to the server.

[1712] Output: Text data sent to the server

[1713] Step 5:

[1714] Server: Analyzes the user's response and generates an appropriate response.

[1715] Input: Text data sent by the user

[1716] Data processing and computation: Analyzing text data using natural language processing techniques to generate appropriate responses. For example, a generative AI model could be used to generate the response, "That's interesting. What era are you talking about?"

[1717] Output: Sends the generated reply to the terminal.

[1718] Step 6:

[1719] Terminal: Provides the user with a voice response to the response received from the server.

[1720] Input: Text data sent from the server

[1721] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[1722] Output: Provides the user with a spoken message saying "That's interesting. What era are you talking about?"

[1723] Step 7:

[1724] Relatives: Use the smartphone app to enter the message content and desired delivery time, and send it to the server.

[1725] Input: Message content, desired sending time

[1726] Data processing and calculation: Data entered via the smartphone app is converted into a protocol for sending to the server.

[1727] Output: Message sent to the server and desired delivery time

[1728] Step 8:

[1729] Server: Sends messages from relatives to the user's device at the specified time.

[1730] Input: Message sent by relative and desired time of sending

[1731] Data processing and calculation: The message content is stored and sent to the user's device at the specified time.

[1732] Output: Message sent to user's device

[1733] Step 9:

[1734] Terminal: Provides the user with a voice message at the specified time.

[1735] Input: Message sent from the server

[1736] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[1737] Output: Provides a voice message saying "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready."

[1738] Step 10:

[1739] User: "Okay," replies the terminal.

[1740] Input: Voice response to terminal

[1741] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[1742] Output: Generates "I understand." as converted text data and prepares it to send to the server.

[1743] Step 11:

[1744] Terminal: Sends the user's voice response to the server.

[1745] Input: User's speech converted to text

[1746] Data processing and calculation: Converts text data into a protocol for sending to the server.

[1747] Output: Text data sent to the server

[1748] Step 12:

[1749] Server: If the user does not respond, perform a series of retries, eventually notifying the next of kin.

[1750] Input: No response status

[1751] Data processing and calculation: If there is no response, the message will be sent again after 10 minutes, and this will be repeated up to three times. If there is still no response, the next of kin will be notified.

[1752] Output: Message to notify relatives

[1753] Step 13:

[1754] User: In an emergency, say "help" or press the emergency button.

[1755] Input: Emergency voice command or button press

[1756] Data processing and calculation: Recognize emergency commands and collect necessary data (such as location information).

[1757] Output: Emergency services and immediate family notification

[1758] Step 14:

[1759] Terminal: Recognizes emergency commands and makes emergency calls.

[1760] Input: User voice command or button press

[1761] Data processing and calculation: Obtain location information and automatically call 119 or 110.

[1762] Output: Emergency call message and location information

[1763] Step 15:

[1764] Relatives: Receive emergency notifications.

[1765] Input: Urgent notification sent from the server

[1766] Data processing and calculation: Display emergency notifications.

[1767] Output: Emergency notification displayed on a relative's device

[1768] Step 16:

[1769] User: Says to the device, "I want to order some groceries."

[1770] Input: User's voice command

[1771] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[1772] Step 17:

[1773] Device: Recognizes voice commands and sends requests to the server.

[1774] Input: User's speech converted to text

[1775] Data processing and calculation: Converts text data into a protocol for sending to the server.

[1776] Output: Request sent to the server

[1777] Step 18:

[1778] Server: Obtains a list of product candidates through the API of the online shopping system.

[1779] Input: The request sent by the user

[1780] Data processing and calculation: Obtain a list of product candidates using the YAHOO Shopping API.

[1781] Output: Sends the product candidate list to the terminal.

[1782] Step 19:

[1783] Terminal: Presents a list of product candidates to the user via voice.

[1784] Input: Product candidate list sent from the server

[1785] Data processing and calculation: Use Google Text-to-Speech to convert the product candidate list into a voice message.

[1786] Output: Provide the user with a list of products as a voice message

[1787] (Application example 1)

[1788] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1789] In elderly life support systems, there is a lack of means to promote daily conversations, communicate with relatives, respond to emergencies, and support purchasing daily necessities. Furthermore, conventional systems rely on the devices used by the elderly, and the operation of the devices is complicated, making it difficult for the elderly to use them. Furthermore, there is a lack of a mechanism that allows relatives in remote locations to easily understand the status of the elderly.

[1790] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1791] In this invention, the server includes a means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice; a means for a relative to remotely send a message to the terminal and deliver the message by voice to the user at a specified time; and a means for generating appropriate responses to the user's questions using a generative AI model. This allows elderly people to easily enjoy conversations and smoothly communicate with their relatives. Furthermore, the use of a generative AI model enables more advanced and flexible responses, allowing the elderly to receive the information they need in a timely manner.

[1792] "Weather information" refers to meteorological data such as the climate conditions, temperature, and probability of precipitation in the elderly person's place of residence.

[1793] "News information" refers to the latest news data on domestic and international events, incidents, culture, economy, etc.

[1794] "Hobbies and Preference Data" refers to data that compiles information related to the user's interests and areas of interest.

[1795] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to provide appropriate answers and generate conversations in response to questions and utterances from users.

[1796] "Providing audio" refers to a computer using speech synthesis technology to audibly convey information to a user.

[1797] "Relatives" refers to family members and blood relatives of the user.

[1798] "Sending a message from a remote location" refers to a relative sending a message to the user's terminal from a physically distant location via the Internet.

[1799] The "designated time" refers to a specific time set in advance by the relative at which the message should be conveyed to the user.

[1800] "Voice command" refers to an instruction or command given by a user through a microphone.

[1801] "Online shopping site" refers to a website for purchasing products over the Internet.

[1802] "User location information" refers to data indicating the user's current physical location.

[1803] A "central processing unit" refers to a computer that functions as the main computing resource of a server, etc., and is responsible for controlling the entire system and processing data.

[1804] "Emergency services" refers to public agencies and services that respond to emergencies, such as fire departments, police, and emergency medical services.

[1805] "Daily necessities" refers to consumables and food items necessary for the daily lives of the elderly.

[1806] This invention is a system for supporting the lives of elderly people, and is realized mainly through cooperation between a server, a terminal used by the user, and a terminal used remotely by a relative. This system has the following configuration and functions.

[1807] 1. Server:

[1808] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. Specifically, every morning at 6:50, it collects the latest information from the weather API, news API, and user hobby database, and sends it to the user's device. It also has the function of generating appropriate responses to user questions using a generative AI model. It can also receive messages from relatives and send them to the user's device at a specified time. The server also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[1809] 2. On the user's device:

[1810] The user device uses speech synthesis technology to send information to the user, recognizes the user's voice using its speech recognition function, and sends it to the server. If the user responds, "The book I've been reading recently is a historical novel," the device recognizes the speech and sends it to the server. It receives the response from the server and uses a generative AI model to generate an appropriate response, such as, "That's interesting. What period is it set in?" In an emergency, the user can issue a voice command such as "Help" or press an emergency button, and the device will automatically make an emergency call, obtain the user's location information, and provide it to medical services. It also works with online shopping sites, recognizing the voice command "I'd like to order daily necessities" and initiating the ordering process via the server.

[1811] 3. Family members' devices:

[1812] Relatives can remotely send messages to the elderly using devices such as smartphones or PCs. When the relative enters the message and the time to send it, the data is sent to a server, and the server sends the message to the user's device at the specified time. For example, a message such as "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready" can be spoken to the user.

[1813] Hardware and software used

[1814] Speech synthesis technology: pyttsx3

[1815] Speech recognition technology: Google Speech Recognition API

[1816] Weather API and News API: Includes any external services

[1817] Generative AI model: using natural language processing techniques

[1818] Specific use cases

[1819] An elderly person is asked at 7:00 a.m. from a terminal, "Good morning. It's a sunny day today. The temperature is 22 degrees. What book have you been reading recently?" If the user responds, "The book I've been reading recently is a historical novel," the server analyzes the data and uses a generative AI model to provide an appropriate response, such as, "That's interesting. What period is it set in?"

[1820] Prompt Sentence Examples

[1821] "Today's weather is sunny and the temperature is 22 degrees. Tell us about the book you've been reading recently."

[1822] This system will enable the elderly to enjoy everyday conversations and communicate smoothly with their relatives. It will also enable emergency response and assistance with purchasing everyday items, improving the quality of life for the elderly.

[1823] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1824] Step 1:

[1825] (Input) The server collects information from the weather API, news API, and hobby database.

[1826] (Processing) Every morning at 6:50, the server sends a request to an external service to collect data related to weather, news, and hobbies.

[1827] (Output) Collected weather information, news information, and hobby-related data.

[1828] (Specific operation) The server sends a request to the API, receives weather and news information in response, and searches and collects related information from the user's hobby database.

[1829] Step 2:

[1830] (Input) Information collected by the server.

[1831] (Processing) The server sends the collected information to the user's terminal.

[1832] (Output) Weather, news, and hobby-related information sent to the user's device.

[1833] (Specific Operation) The server formats the collected data and sends it to the user's device, which receives this information.

[1834] Step 3:

[1835] (Input) Information sent to the user's device.

[1836] (Processing) The terminal speaks to the user using voice synthesis technology.

[1837] (Output) A voice message to the user.

[1838] (Specific operation) The user's device converts information about the weather, news, and hobbies using a speech synthesis engine (pyttsx3) and speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1839] Step 4:

[1840] (Input) The user's response.

[1841] The (processing) terminal uses voice recognition technology to convert the user's response into text.

[1842] (Output) The user's response in text format.

[1843] (Specific operation) When the user responds, "The book I've been reading recently is a historical novel," the device converts the speech into text using the Google Speech Recognition API.

[1844] Step 5:

[1845] (Input) The user's response in text form.

[1846] (Processing) The terminal sends the user's response to the server.

[1847] (Output) The user's response sent to the server.

[1848] (Specific operation) The terminal sends the converted text to the server via an HTTP request.

[1849] Step 6:

[1850] (Input) The user's reply text.

[1851] The (processing) server uses a generative AI model to generate an appropriate response.

[1852] (Output) The generated response.

[1853] (Specific operation) The server uses a generative AI model to generate a response such as "That's interesting. What period is it set in?" in response to "The book I've been reading lately is a historical novel."

[1854] Step 7:

[1855] (Input) The generated response.

[1856] (Processing) The server sends the generated response to the user's terminal.

[1857] (Output) The generated reply to the user's terminal.

[1858] (Specific operation) The response generated by the server is formatted for transmission to the user's terminal and sent to the terminal via an HTTP request.

[1859] Step 8:

[1860] (Input) The response sent by the server.

[1861] The (processing) terminal uses speech synthesis technology to communicate the response to the user.

[1862] (Output) A spoken response to the user.

[1863] (Specific operation) The device converts the response received from the server using a speech synthesis engine (pyttsx3) and speaks to the user, saying, "That's interesting. What era are you talking about?"

[1864] Step 9:

[1865] (Input) Message content and time sent by relatives.

[1866] (Processing) A relative uses a smartphone or computer from a remote location to send the message content and sending time to the server.

[1867] (Output) Message content and sending time stored on the server.

[1868] (Specific operation) A relative uses the application to enter the message content and sending time, and sends it to the server.

[1869] Step 10:

[1870] (Input) Message content and time sent from relatives.

[1871] (Processing) The server sends a message to the user's terminal at the specified time.

[1872] (Output) Messages from relatives to the user's terminal.

[1873] (Specific operation) The server monitors the specified time, and when the set time arrives, it sends a message to the user's terminal.

[1874] Step 11:

[1875] (Input) Message from relatives from the server.

[1876] (Processing) The terminal uses voice synthesis technology to convey the message to the user.

[1877] (Output) A voice message to the user.

[1878] (Specific operation) The terminal converts the message received from the server using a speech synthesis engine (pyttsx3) and tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[1879] Step 12:

[1880] (Input) If there is no response from the user.

[1881] (Processing) The terminal retries a certain number of times, and if there is no response, it notifies the server.

[1882] (Output) Notifies the server of the user's response status.

[1883] (Specific operation) The terminal waits for a response from the user, and if there is still no response, it retries a certain number of times and sends the message again, and if there is still no response, it notifies the server.

[1884] Step 13:

[1885] (Input) User response status notification received by the server.

[1886] (Processing) The server notifies the relatives of the user's response status.

[1887] (Output) Notify relatives of the user's response status.

[1888] (Specific operation) The server sends emails and application notifications to relatives to report the user's response status.

[1889] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1890] This invention is a system for supporting the lives of the elderly and promoting conversation. This system not only provides weather information, news information, and topics based on the user's hobbies and preferences, but also incorporates an emotion engine to recognize the user's emotions and achieve more appropriate communication. The following describes each component of this system and the program processing.

[1891] Overall system overview

[1892] 1. Server:

[1893] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time. In addition, it analyzes the user's emotional data via an emotion engine and generates feedback based on this.

[1894] 2. Terminal:

[1895] The user's device uses voice synthesis technology to send information to the user. It also has the function of recognizing the user's voice and sending response data to the server. In an emergency, an emergency call can be made using voice commands or a button. It also has a function that links with online shopping sites to assist with purchasing daily necessities. The emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the content of the conversation based on the results.

[1896] 3. Family members' devices:

[1897] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[1898] Program processing

[1899] Daily conversation promotion features

[1900] server:

[1901] Every morning, the server collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[1902] Device:

[1903] Every morning, the device uses voice synthesis technology to analyze the latest information received from the server and speaks to the user, such as, "Good morning. It's sunny today. The temperature is 22 degrees. How's the book you've been reading recently?" The emotion engine also analyzes the user's emotions at this point and adjusts the tone and content of the conversation.

[1904] User:

[1905] The user replies, "Good morning. I've been reading historical novels lately."

[1906] Device:

[1907] The device recognizes the user's response through voice recognition, performs emotional analysis using an emotion engine, and then sends the results to the server.

[1908] server:

[1909] The server analyzes the user's response and emotional data, generates appropriate response data, and sends it to the terminal.

[1910] Device:

[1911] Based on the received response data and emotion data, the device responds to the user by saying, "That's interesting. What era are you talking about?"

[1912] Message function from relatives

[1913] relatives:

[1914] The relative opens the smartphone app and enters the message content and desired delivery time.

[1915] server:

[1916] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[1917] Device:

[1918] The message will arrive at the specified time and tell the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready." At this time, the emotion engine will analyze the user's reaction and provide feedback on the emotion to the relatives.

[1919] User:

[1920] The user responds, "Okay."

[1921] Device:

[1922] The device recognizes the user's response and sends it to the server. If there is no response, it retries a certain number of times, and if there is still no response, it notifies the server.

[1923] server:

[1924] If the server does not respond for a certain period of time, it will send a notification to the next of kin.

[1925] Emergency call function

[1926] User:

[1927] In an emergency, you can either say "help" to the device or press the emergency button.

[1928] Device:

[1929] The device detects an emergency command or button trigger and automatically calls 119 or 110. At the same time, emotional data is also sent to the server, and relatives are notified.

[1930] Online shopping integration function

[1931] User:

[1932] The user speaks to the terminal saying, "I would like to order daily necessities."

[1933] Device:

[1934] The device analyzes the user's voice commands using a voice recognition engine and emotion engine, and sends a request to the server.

[1935] server:

[1936] The server initiates the order process through the online shopping API and sends the order confirmation information to the terminal.

[1937] Device:

[1938] The terminal will then notify the user that the order has been completed and analyze the user's emotions during the process, providing appropriate feedback.

[1939] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. It combines functions for daily conversations, emergency response, and online shopping support, improving the quality of life for elderly people.

[1940] The processing flow will be explained below.

[1941] Daily conversation promotion features

[1942] Server-side processing

[1943] Step 1:

[1944] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[1945] Step 2:

[1946] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[1947] Step 3:

[1948] The server transmits the acquired data to each user's terminal.

[1949] Terminal side processing

[1950] Step 1:

[1951] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[1952] Step 2:

[1953] The terminal analyzes the received information and generates content that speaks to the user.

[1954] Step 3:

[1955] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[1956] User processing

[1957] Step 1:

[1958] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[1959] Terminal side processing

[1960] Step 1:

[1961] The device recognizes the user's response through voice recognition, analyzes the emotion using an emotion engine, and then sends the results to the server.

[1962] Server-side processing

[1963] Step 1:

[1964] The server analyzes the user's response content and emotional data, generates appropriate response data, and sends it to the terminal.

[1965] Terminal side processing

[1966] Step 1:

[1967] The device analyzes the received response data and emotion data and responds to the user, "That's interesting. What era are you talking about?"

[1968] Message function from relatives

[1969] Processing for relatives

[1970] Step 1:

[1971] The relative opens the smartphone app and enters the message content and desired delivery time.

[1972] Step 2:

[1973] The relative taps the "Send" button to send the message to the server.

[1974] Server-side processing

[1975] Step 1:

[1976] The server stores the message received from the relative and the specified time in a database.

[1977] Step 2:

[1978] At the specified time, the server sends a message to the corresponding user's terminal.

[1979] Terminal side processing

[1980] Step 1:

[1981] The terminal receives a message from the server at a specified time.

[1982] Step 2:

[1983] The device then audibly conveys the received message to the user, saying, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[1984] Step 3:

[1985] The device uses an emotion engine to analyze the user's reaction and transmits the emotion data to the server.

[1986] User processing

[1987] Step 1:

[1988] The user responds, "Okay."

[1989] Terminal side processing

[1990] Step 1:

[1991] The terminal recognizes the user's response and sends it to the server.

[1992] Step 2:

[1993] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[1994] Step 3:

[1995] If the device does not receive a response after three retries, it notifies the server.

[1996] Server-side processing

[1997] Step 1:

[1998] The server detects the user's lack of response and sends a notification to the next of kin.

[1999] Emergency call function

[2000] User processing

[2001] Step 1:

[2002] In an emergency, the user can either say "help" to the device or press the emergency button.

[2003] Terminal side processing

[2004] Step 1:

[2005] The device recognizes emergency commands or button triggers.

[2006] Step 2:

[2007] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[2008] Step 3:

[2009] The device will send a notification to relatives as soon as the emergency call is completed.

[2010] Online shopping integration function

[2011] User processing

[2012] Step 1:

[2013] The user speaks to the terminal saying, "I would like to order daily necessities."

[2014] Terminal side processing

[2015] Step 1:

[2016] The device analyzes the user's voice commands using a voice recognition engine and emotion engine, and sends a request to the server.

[2017] Step 2:

[2018] The terminal presents the candidate product list received from the server to the user by voice.

[2019] Step 3:

[2020] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[2021] Server-side processing

[2022] Step 1:

[2023] The server initiates the order process through an online shopping API.

[2024] Step 2:

[2025] Once the order is complete, the server sends confirmation information to the terminal.

[2026] Terminal side processing

[2027] Step 1:

[2028] The device notifies the user by voice when the order is complete, and transmits the emotion data obtained during the process to the server to provide appropriate feedback to the relatives.

[2029] Example 2

[2030] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2031] In modern society, elderly people often feel lonely, and the limited opportunities for daily conversation and communication often have a negative impact on their mental and psychological health. It is also difficult to contact relatives who live far away, making it difficult to detect abnormalities or emergencies in the elderly in a timely manner. Furthermore, it can be difficult to purchase daily necessities, raising concerns about a decline in quality of life.

[2032] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2033] In this invention, the server includes means for generating weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice, means for a relative to send a message to the terminal from a remote location and deliver the message by voice to the user at a specified time, and means for analyzing the user's emotion data using an emotion recognition engine and adjusting the tone and content of the conversation. This allows the user to promote daily conversations, enable appropriate communication with relatives even when they are far away, and make it possible to respond to emergencies and purchase daily necessities smoothly.

[2034] "Weather information" refers to information such as temperature, precipitation, wind speed, and weather conditions provided based on meteorological data.

[2035] "News information" refers to reports and articles about the latest events and social topics.

[2036] "Hobbies and preferences" is information related to activities and topics that an individual is interested in and engages in for fun.

[2037] "Users" refer to the elderly people who use this system.

[2038] "Remote location" refers to a location that is physically separate, and typically refers to a case where the user and their relatives are in different locations.

[2039] "Message" refers to a message that a relative wants to convey to the user.

[2040] A "voice command" is a method in which a user issues instructions to a terminal using voice.

[2041] An "emotion recognition engine" is a technology for detecting and analyzing emotions from a user's voice and facial expressions.

[2042] "Emergency call" is a means of contacting a user to request help in an emergency.

[2043] An "online shopping site" is a website where you can purchase goods and services over the Internet.

[2044] "Speech synthesis technology" is a technology for converting text data into voice data.

[2045] A "voice recognition engine" is a technology for converting voice data into text data.

[2046] This invention relates to a system for supporting the lives of the elderly and promoting conversation. This system provides weather information, news information, and topics based on the user's hobbies and preferences, and also uses an emotion recognition engine to achieve more appropriate communication.

[2047] Overall system configuration

[2048] 1. Server:

[2049] We use weather APIs (e.g., OpenWeatherMap API) and news APIs (e.g., NewsAPI) to periodically collect data related to weather information, news information, and hobbies and preferences.

[2050] The system receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response for a certain period of time.

[2051] An emotion recognition engine (e.g., Affectiva, IBM Watson Tone Analyzer) is used to analyze the user's emotion data and generate feedback.

[2052] 2. Terminal:

[2053] Using speech synthesis technology (e.g., Google Text-to-Speech API), information is transmitted to the user via voice.

[2054] A speech recognition engine (e.g., Google Speech-to-Text API) is used to recognize the user's voice and send response data to the server.

[2055] In an emergency, use voice commands or press a button to make an emergency call.

[2056] It works in conjunction with online shopping sites and has the ability to assist with purchasing daily necessities using voice commands.

[2057] An emotion recognition engine is used to recognize emotions from the user's voice and facial expressions and adjust the content of the conversation.

[2058] 3. Family members' devices:

[2059] Using a smartphone or personal computer, messages can be sent to elderly people from remote locations, and notifications can be received in the event of an emergency or when safety confirmation is required.

[2060] Specific examples

[2061] Daily conversation promotion features

[2062] server:

[2063] Every morning, the server collects the latest information from weather APIs, news APIs, and the user's hobby database and sends it to the device.

[2064] Device:

[2065] Every morning, the device receives the latest information from the server, analyzes it using speech synthesis technology, and speaks to the user. Example: "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[2066] User:

[2067] The user replies, "Good morning. I've been reading historical novels lately."

[2068] Device:

[2069] The device converts the user's response into text using a voice recognition engine, analyzes the emotion using an emotion recognition engine, and sends the results to the server.

[2070] server:

[2071] The server generates an appropriate response based on the user's response and emotional data and sends it to the device. Example: "That's interesting. What era are you talking about?"

[2072] Device:

[2073] The device uses voice synthesis technology to communicate with the user based on the received response and emotional data. Example: "That's interesting. What era are you talking about?"

[2074] Message function from relatives

[2075] relatives:

[2076] The relative opens the smartphone app and enters the message and desired delivery time. For example, "Please let them know I have a doctor's appointment tomorrow at 9:00 AM."

[2077] server:

[2078] The server stores the message content and the specified time received from the relative, and sends the message to the corresponding user's terminal when the specified time arrives.

[2079] Device:

[2080] The device receives the message at the specified time and uses speech synthesis technology to convey it to the user. For example, "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready." An emotion recognition engine analyzes the user's reaction and provides feedback on that emotion to the relative.

[2081] Emergency call function

[2082] User:

[2083] In an emergency, you can either say "help" to the device or press the emergency button.

[2084] Device:

[2085] The device will detect the emergency command or button trigger and immediately make an automatic call to 119 or 110. Example: "An elderly person needs help."

[2086] The device sends the emotional data along with the report to a server, and notifies relatives.

[2087] Online shopping integration function

[2088] User:

[2089] The user speaks to the terminal saying, "I would like to order daily necessities."

[2090] Device:

[2091] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[2092] server:

[2093] The server initiates the order process through an online shopping API. Example: "Your grocery order has been initiated."

[2094] Device:

[2095] The device notifies the user of order confirmation information from the server using voice synthesis technology and provides emotional feedback. Example: "Your order has been completed."

[2096] Prompt Sentence Examples

[2097] Based on the following conversation, please suggest topics that might interest users next:

[2098] User: "The book I've been reading lately is a historical novel."

[2099] System: "That's interesting. What era are you talking about?"

[2100] User: "It's from the Meiji period."

[2101] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2102] Specific explanation of processing steps

[2103] Daily conversation promotion features

[2104] server

[2105] Step 1:

[2106] The server collects weather information from a weather API at a fixed time every morning. The specific input is the request data to the API, and the output is the retrieved weather information. It also obtains the latest news information from a news API and extracts topics based on the user's hobbies and preferences from a hobby database.

[2107] Step 2:

[2108] The server integrates the collected weather information, news information, and hobby data into a JSON format and sends this data package to the terminal. The input data is each piece of collected information, and the output is the compiled JSON data.

[2109] Terminal

[2110] Step 3:

[2111] The device analyzes the JSON data received from the server. During analysis, it uses speech synthesis technology (e.g., Google Text-to-Speech API) to convert the transmitted information into voice data. Specifically, it converts text input into voice data and speaks to the user. Example: "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[2112] User

[2113] Step 4:

[2114] The user responds to the voice guidance from the terminal. For example, they might say, "Good morning. The book I've been reading recently is a historical novel." This user's voice data becomes the input.

[2115] Terminal

[2116] Step 5:

[2117] The device converts the user's response into text data using a speech recognition engine (e.g., Google Speech-to-Text API). It then uses an emotion recognition engine to analyze emotional data from the user's voice. The input is the user's voice data, and the output is the recognized text data and emotional data.

[2118] Step 6:

[2119] The device sends the obtained text data and emotion data to the server. The input is the analysis results (text data and emotion data) on the device side, which are then sent to the server.

[2120] server

[2121] Step 7:

[2122] The server analyzes the received user response and emotional data. Specifically, it uses a natural language processing (NLP) model (e.g., OpenAI GPT-3) to generate an appropriate response. The input is the user response and emotional data, and the output is the generated response. Example: "That's interesting. What era are you talking about?"

[2123] Step 8:

[2124] The server sends the generated response along with the emotion data to the terminal. The input is the response content and emotion data, and the output is the transmitted data.

[2125] Terminal

[2126] Step 9:

[2127] The device receives the response and emotion data from the server and again uses speech synthesis technology to speak to the user. Example: "That's interesting. What era are you talking about?" The input data are the received response and emotion data, and the output is voice data.

[2128] Message function from relatives

[2129] relatives

[2130] Step 1:

[2131] The relative opens the smartphone app and inputs the message content and desired sending time. For example, "Please tell them I have a doctor's appointment tomorrow at 9:00 AM."

[2132] server

[2133] Step 2:

[2134] The server stores the message content received from the relatives and the specified sending time in a database. The input is the message content and sending time, and the output is the stored data.

[2135] Step 3:

[2136] When the designated sending time arrives, the server retrieves the stored message data and sends the message to the corresponding user's terminal. The input is the sending time and message data, and the output is the message to be sent.

[2137] Terminal

[2138] Step 4:

[2139] The device receives the message from the server at the specified time and conveys it to the user using speech synthesis technology. Example: "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready." At this time, the emotion recognition engine analyzes the user's reaction and generates emotional data. The input is the message, and the output is voice data and emotional data.

[2140] User

[2141] Step 5:

[2142] The user responds, "I understand." This voice data is the input.

[2143] Terminal

[2144] Step 6:

[2145] The device converts the user's response into text data using a speech recognition engine and sends it to the server. At the same time, emotion data is also sent to the server. The input is the user's voice data, and the output is text data and emotion data.

[2146] server

[2147] Step 7:

[2148] If there is no response for a certain period of time, the server will retry, and if there is still no response, it will send a notification to the relatives. The input is the response status, and the output is the notification message.

[2149] Emergency call function

[2150] User

[2151] Step 1:

[2152] The user can say "help" or press the emergency button on the terminal. This voice data or button down is the input.

[2153] Terminal

[2154] Step 2:

[2155] The device detects an emergency command or button trigger and immediately and automatically calls 119 or 110. Specifically, it uses voice synthesis technology to automatically generate the message content. The input is the emergency command or button trigger, and the output is the message data.

[2156] Step 3:

[2157] Along with the emergency call, the device sends emotional data to the server and notifies relatives. The input is the emergency call and emotional data, and the output is the transmitted data.

[2158] Online shopping integration function

[2159] User

[2160] Step 1:

[2161] The user speaks to the terminal, saying, "I would like to order daily necessities." This voice data is the input.

[2162] Terminal

[2163] Step 2:

[2164] The device analyzes the user's voice commands using a voice recognition engine and sends a request to the server. The input is voice data and the output is the voice recognition result.

[2165] server

[2166] Step 3:

[2167] The server initiates the order process through an online shopping API, placing the order with the necessary authentication information and product data. The input is the speech recognition result, and the output is the order confirmation data.

[2168] Step 4:

[2169] The server sends order confirmation information to the terminal. The input is the order confirmation data and the output is the transmission data.

[2170] Terminal

[2171] Step 5:

[2172] The device receives order confirmation information from the server and notifies the user using voice synthesis technology. For example, "Your order has been completed." At the same time, emotional feedback is provided. The input is order confirmation data, and the output is voice data and emotional feedback.

[2173] (Application example 2)

[2174] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2175] Isolation and lack of communication among the elderly are serious problems in modern society. It is also difficult for relatives living far away to keep track of the elderly's lifestyle and safety, which can delay emergency response. It is also difficult for elderly people to obtain timely entertainment and information that suits their emotions and moods. There is a need for a system that can solve these issues and improve the quality of life for the elderly.

[2176] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting weather information and news information and generating topics taking into account the user's hobbies and preferences, means for performing emotion analysis and recommending content suitable for the user, and means for relatives to send messages from remote locations and deliver them to the user at a specified time. This prevents the elderly from becoming isolated, enables appropriate communication to be maintained, and makes it easier for relatives to understand the elderly's condition even from remote locations. It also makes it possible to provide appropriate support and comfortable entertainment in emergencies and in daily life.

[2177] "Weather information" is data relating to the current weather conditions and forecasted weather conditions in the user's place of residence.

[2178] "News information" refers to the latest information and news reports about current events and social topics.

[2179] "User hobbies and preferences" is data about the activities and interests of the user.

[2180] "Emotion analysis" is the process of identifying and assessing a user's emotional state from their voice and facial expressions.

[2181] "Content" means digital data that provides information or entertainment, such as music, video, news, and articles.

[2182] "Relatives" refers to family members or relatives who are related by blood or legal relationship to the user.

[2183] An "emergency" is when a user is faced with a serious health or safety situation.

[2184] A "voice command" is an instruction that a user gives to a system using voice.

[2185] An "online shopping site" is a website for purchasing goods and services over the Internet.

[2186] "Server" means a computer system that collects, processes, and transmits data over a network.

[2187] A "terminal" is a device that a user can directly operate (such as a smartphone, smart glasses, a head-mounted display, or a robot).

[2188] This invention is a system for supporting the lives of the elderly and promoting conversation. This system provides weather information, news information, and topics based on the user's hobbies and preferences, and uses an emotion engine to evaluate the user's emotional state and provide appropriate content. Below, we will clearly explain the program processes and hardware and software configurations required to implement this invention.

[2189] server

[2190] A server is a computer system connected to the Internet that has the following functions:

[2191] 1. Data collection: Data is collected periodically from weather information API, news information API, and a database of user hobbies.

[2192] 2. Sentiment analysis: The user's voice data is sent to the sentiment analysis engine to evaluate the emotional state.

[2193] 3. Data transmission: Collected data and analysis results are sent to the device.

[2194] 4. Family collaboration: Receive messages and notifications from relatives and forward them to the appropriate device at the specified time.

[2195] 5. Emergency Calls: Receive emergency calls from users and contact emergency services as needed.

[2196] As a concrete example, every morning the server obtains the current weather conditions from a weather information API, collects news and topics based on the user's hobbies and preferences, and sends them to the device. At this time, the server analyzes the user's voice data from the previous day using an emotion analysis engine, and selects and provides topics that suit the user's mood.

[2197] Terminal

[2198] The device operated by the user (smartphone, smart glasses, head-mounted display, or robot) has the following functions:

[2199] 1. Voice recognition: Recognizes the user's voice instructions and sends the data to the server.

[2200] 2. Speech synthesis: The data received from the server is presented to the user as voice.

[2201] 3. Emotion analysis: Analyze the user's voice and facial expressions using the built-in emotion analysis engine.

[2202] 4. Message from relatives: A voice message from a relative will be delivered at the specified time.

[2203] 5. Emergency Call: In case of an emergency, the device receives voice commands or button inputs from the user and makes an emergency call.

[2204] As a concrete example, the device will speak to the user every morning saying, "Good morning. It's a sunny day today. How do you like the novel you've been reading recently?" and will then analyze the user's response using an emotion analysis engine to provide more appropriate topics of conversation.

[2205] User

[2206] Users can give voice instructions to the device and receive information. They can receive messages from relatives or purchase daily necessities using voice commands. In an emergency, they can also make an emergency call by saying "help" to the device.

[2207] relatives

[2208] Relatives can use smartphones or computers to send messages to the elderly and receive notifications in emergencies and regular safety checks, making it easier for relatives to keep track of the elderly's living conditions even when they are far away.

[2209] Example prompt sentence:

[2210] "Please wait as we are currently conducting an ongoing sentiment analysis."

[2211] "We recommend relaxing music to suit your mood."

[2212] "Good morning. It's a beautiful day today. How's the novel you've been reading lately?"

[2213] This will make it possible to provide an environment where elderly people can spend their days comfortably and safely.

[2214] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2215] Step 1:

[2216] Data collection (server)

[2217] The server collects the latest information from the weather information API, news information API, and user interest and preference database. Current weather data is obtained from the weather information API, and the latest news articles are collected from the news information API. Data related to the user's areas of interest is extracted from the user interest and preference database. This data is temporarily stored on the server.

[2218] Input: Weather information API, news information API, hobby and preference database

[2219] Output: Latest weather data, latest news data, hobby and preference data

[2220] Step 2:

[2221] Sentiment analysis (server)

[2222] The server sends the user's voice and text data to the emotion analysis engine to evaluate the user's emotional state. For example, if the user says "I'm tired," the emotion analysis engine will identify this as "fatigue" and return the result to the server.

[2223] Input: User voice or text data

[2224] Output: Emotion analysis results (e.g., fatigue)

[2225] Step 3:

[2226] Data transmission (server)

[2227] The server compiles the collected weather information, news information, user hobby data, and emotion analysis results and sends them to the user's device. This data becomes material for promoting daily conversations between users.

[2228] Input: Weather data, news data, hobby and preference data, emotion analysis results

[2229] Output: Consolidated data to send to the device

[2230] Step 4:

[2231] Speech synthesis and provision (terminal)

[2232] The device analyzes the data received from the server and provides it to the user using speech synthesis technology. For example, it might say something like, "Good morning. It's a sunny day today. How's the novel you've been reading recently?"

[2233] Input: Integrated data (weather data, news data, hobby and preference data, emotion analysis results)

[2234] Output: The audio message to be presented to the user

[2235] Step 5:

[2236] User response and reanalysis (terminal)

[2237] The user responds to the device, for example, saying, "The novel I've been reading recently is very interesting." The device recognizes this speech and sends it to the server. At the same time, the emotion analysis engine reanalyzes the user's emotional state.

[2238] Input: User's voice response

[2239] Output: Voice data sent to the server, reanalyzed emotional state

[2240] Step 6:

[2241] Message notification from relatives (server and terminal)

[2242] The server sends the message received from the relative to the terminal at the specified time. The terminal then delivers the message to the user by voice. For example, the terminal may say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[2243] Input: Message from relatives, specified time

[2244] Output: The audio message to be presented to the user

[2245] Step 7:

[2246] Emergency call (terminal)

[2247] In an emergency, the user can either say "help" to the device or press the emergency button, and the device will make an emergency call. The call will also include the user's location information.

[2248] Input: User's voice command or button press

[2249] Output: Emergency call with location information attached

[2250] Step 8:

[2251] Online shopping (terminal)

[2252] The user speaks to the device, saying, "I'd like to order some daily necessities." The device recognizes this command and sends it to the server. The server processes the order through the online shopping API and sends a confirmation to the device.

[2253] Input: User's voice command

[2254] Output: Online shopping order confirmation information

[2255] This will realize a system that can take into account the user's emotional state and provide appropriate information and communication.

[2256] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2257] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2258] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2259] [Fourth embodiment]

[2260] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2261] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2262] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2263] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2264] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2265] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2266] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2267] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2268] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2269] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2270] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2271] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2272] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2273] This invention is a system for supporting the lives of the elderly, aiming to promote daily conversations and facilitate communication with relatives. The system mainly includes the following elements: a server, a terminal used by the user, and a terminal used remotely by relatives.

[2274] Overall system overview

[2275] 1. Server:

[2276] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides them to the device. It also has the function of receiving messages from relatives and sending them to the user's device at a specified time. It also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[2277] 2. Terminal:

[2278] The user's device uses voice synthesis technology to send information to the user. It also has the ability to recognize the user's voice and send response data to the server. In an emergency, an emergency call can be made using voice commands or a button. The device also has a function that links with online shopping sites to help with purchasing daily necessities.

[2279] 3. Family members' devices:

[2280] Relatives can remotely send messages to the elderly using devices such as smartphones or computers, and can also receive notifications in the event of an emergency or when safety confirmation is required.

[2281] Program processing

[2282] Daily conversation promotion features

[2283] server:

[2284] At 6:50 in the morning, it collects the latest information from the weather API, news API, and user hobby database and sends it to the user's device.

[2285] Device:

[2286] At 7:00 a.m., the latest information received from the server is analyzed using voice synthesis technology, and the system speaks to the user, saying things like, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[2287] User:

[2288] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[2289] Device:

[2290] The system recognizes the user's response and sends the analysis results to the server. Based on the response received from the server, the system responds with "That's interesting. What era are you talking about?"

[2291] Message function from relatives

[2292] relatives:

[2293] The relative opens the smartphone app, enters the message content and desired sending time, and sends it to the server.

[2294] server:

[2295] The system stores messages received from relatives and the specified time, and sends the message to the corresponding user's terminal when the time comes.

[2296] Device:

[2297] A message will be received at the specified time, telling the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[2298] User:

[2299] The user responds, "Okay."

[2300] Device:

[2301] The user's response is sent to the server. If there is no response, the message is sent again after 10 minutes, and this is repeated up to three times. If there is still no response, the server is notified.

[2302] server:

[2303] If the user does not respond and there is still no response after a certain number of retries, a notification is sent to the next of kin.

[2304] Emergency call function

[2305] User:

[2306] In an emergency, you can either say "help" to the device or press the emergency button.

[2307] Device:

[2308] When an emergency command is recognized, it will automatically call 119 or 110 and simultaneously send the user's location information. It will also notify relatives.

[2309] Online shopping integration function

[2310] User:

[2311] Speak to the device and say, "I'd like to order daily necessities."

[2312] Device:

[2313] It recognizes voice commands, sends requests to the server, receives a list of product candidates from the server, and presents them to the user via voice.

[2314] server:

[2315] The ordering process is initiated through the YAHOO Shopping API and order confirmation information is sent back to the terminal.

[2316] Device:

[2317] The user is notified by voice that the order has been completed.

[2318] The system is designed to increase opportunities for elderly people to enjoy daily conversations and allow relatives to monitor them from a distance. By integrating functions for daily conversations, emergency response, and online shopping support, it contributes to improving the quality of life for elderly people.

[2319] The processing flow will be explained below.

[2320] Daily conversation promotion features

[2321] Server-side processing

[2322] Step 1:

[2323] The server connects to the weather API, news API, and user interest database every morning at 6:50.

[2324] Step 2:

[2325] The server retrieves weather information, the latest news, and data based on the user's interests and preferences.

[2326] Step 3:

[2327] The server transmits the acquired data to each user's terminal.

[2328] Terminal side processing

[2329] Step 1:

[2330] The terminal receives the latest weather information, news information, and information about the user's hobbies from the server every morning at 7:00.

[2331] Step 2:

[2332] The terminal analyzes the received information and generates content that speaks to the user.

[2333] Step 3:

[2334] Using voice synthesis technology, the device speaks to the user, saying, "Good morning. It's a sunny day today. The temperature is 22 degrees. How's the book you've been reading lately?"

[2335] User processing

[2336] Step 1:

[2337] The user responds to the device's question by saying, "Good morning. The book I've been reading recently is a historical novel."

[2338] Terminal side processing

[2339] Step 1:

[2340] The device performs voice recognition on the user's response and sends the analysis results to the server.

[2341] Step 2:

[2342] The server analyzes the user's response, generates response data, and sends it to the terminal.

[2343] Step 3:

[2344] The device analyzes the received response data and responds to the user, "That's interesting. What era are you talking about?"

[2345] Message function from relatives

[2346] Processing for relatives

[2347] Step 1:

[2348] The relative opens the smartphone app and enters the message content and desired delivery time.

[2349] Step 2:

[2350] The relative taps the "Send" button to send the message to the server.

[2351] Server-side processing

[2352] Step 1:

[2353] The server stores the message received from the relative and the specified time in a database.

[2354] Step 2:

[2355] At the specified time, the server sends a message to the corresponding user's terminal.

[2356] Terminal side processing

[2357] Step 1:

[2358] The terminal receives a message from the server at a specified time.

[2359] Step 2:

[2360] The terminal will then audibly convey the received message to the user.

[2361] Step 3:

[2362] The terminal tells the user, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so let's get ready."

[2363] User processing

[2364] Step 1:

[2365] The user responds, "Okay."

[2366] Terminal side processing

[2367] Step 1:

[2368] The terminal recognizes the user's response and sends it to the server.

[2369] Step 2:

[2370] If the user does not respond, the device will send the message again after 10 minutes, and repeat this process up to three times.

[2371] Step 3:

[2372] If the device does not receive a response after three retries, it notifies the server.

[2373] Server-side processing

[2374] Step 1:

[2375] The server detects the user's lack of response and sends a notification to the next of kin.

[2376] Emergency call function

[2377] User processing

[2378] Step 1:

[2379] In an emergency, the user can either say "help" to the device or press the emergency button.

[2380] Terminal side processing

[2381] Step 1:

[2382] The device recognizes emergency commands or button triggers.

[2383] Step 2:

[2384] The device will immediately automatically call 119 or 110 and simultaneously send the user's location information.

[2385] Step 3:

[2386] The device will send a notification to relatives as soon as the emergency call is completed.

[2387] Online shopping integration function

[2388] User processing

[2389] Step 1:

[2390] The user speaks to the terminal saying, "I would like to order daily necessities."

[2391] Terminal side processing

[2392] Step 1:

[2393] The terminal analyzes the user's voice commands using a voice recognition engine and sends a request to the server.

[2394] Step 2:

[2395] The terminal presents the candidate product list received from the server to the user by voice.

[2396] Step 3:

[2397] The terminal waits for the user's selection and confirmation, and then transmits the result to the server.

[2398] Server-side processing

[2399] Step 1:

[2400] The server initiates the order process through the YAHOO Shopping API.

[2401] Step 2:

[2402] Once the order is complete, the server sends confirmation information to the terminal.

[2403] Terminal side processing

[2404] Step 1:

[2405] The terminal will notify the user by voice that the order has been completed.

[2406] Example 1

[2407] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2408] It is becoming increasingly difficult for elderly people to maintain daily contact with society, leading to increased isolation and health risks. It is also difficult for relatives who live far away to constantly check on the safety of their elderly relatives. Furthermore, it is difficult for elderly people to find a way to respond quickly in an emergency or to purchase daily necessities.

[2409] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2410] In this invention, the server includes: means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user via voice; means for a relative to remotely send a message to the terminal and deliver the message via voice to the user at a specified time; means for retrying a certain number of times if there is no response from the user and notifying the relative if there is still no response; means for retrying via voice if there is no response for a long period of time and notifying the relative if there is still no response; means for the user to make an emergency call using a voice command or button in an emergency; means for connecting with an online shopping system to purchase daily necessities using voice commands; means for collecting various information at a specified time and transmitting it to the terminal; means for voice recognition of the user's response, transmitting it to the server, analyzing the response from the server, and responding; and means for notifying the relative if there is no conversation for a certain period of time. This allows the elderly to maintain daily information and communication, allowing relatives to easily check the elderly's safety. It also enables quick response in emergencies and smooth purchasing of daily necessities.

[2411] "Weather information" is data about current and forecast atmospheric conditions, such as weather, temperature, precipitation, and wind speed.

[2412] "News information" refers to the latest reports on events and topics in various fields, including society, politics, economics, and culture.

[2413] "User's hobbies and preferences" refers to data and information related to the user's interests, favorite activities, favorite items, and the like.

[2414] "Means of providing by voice" is a function that uses voice synthesis technology to convey text or data to the user as voice output.

[2415] "Relatives" refers to family members or close friends of the user who are interested in the user's life and well-being.

[2416] A "message" is a word or content conveyed in the form of a message, and is information sent primarily from relatives to the user.

[2417] The "means for retrying if there is no response" is a function for sending the same message again if the user does not respond to a specified message.

[2418] "Means to call emergency services" is the ability for a user to contact emergency services via voice command or physical button in the event of an emergency.

[2419] An "online shopping system" is a system that allows you to search for and select products and complete the purchasing process via the Internet.

[2420] "Means of collecting various information" refers to the function of obtaining weather information, news information, hobby-related information, etc. from external data sources.

[2421] The "means for recognizing voice and responding" is a function that converts the user's voice into text data, sends it to the server, and generates an appropriate voice message based on the analysis results.

[2422] The "means for notifying when there is no conversation for a certain period of time" is a function that notifies relatives of the situation when the user does not talk to the system for a certain period of time.

[2423] This invention is a system for supporting the daily lives of elderly people, and aims to promote daily conversations and facilitate communication with relatives. The system includes a server, a terminal used by the user, and a terminal used by relatives remotely.

[2424] server

[2425] The server is responsible for periodically collecting information from multiple external data sources and providing it to the user's device. Specifically, it uses the OpenWeatherMap API as a weather API to obtain weather information, and the News API as a news API to obtain news information. In addition, information based on the user's hobbies and preferences is obtained from the user's hobby database. This information is collected at a specified time (for example, 6:50 every morning) and sent to the user's device.

[2426] In addition, the server receives messages from relatives and sends them to the user's device at a specified time. It also monitors the user's response status and notifies the relatives if there is no response after a certain number of attempts. Furthermore, if there is no response for a long period of time, it retries by voice and eventually notifies the relatives.

[2427] User's device

[2428] The user's device is equipped with speech synthesis technology (e.g., Google Text-to-Speech) and speech recognition technology (e.g., Google Speech-to-Text). This device has the ability to convey information received from the server to the user by voice. For example, at 7:00 in the morning, the device might say, "Good morning. It's a sunny day today. The temperature is 22 degrees. How is the book you've been reading recently?"

[2429] When the user responds to this question, the device uses voice recognition technology to convert the user's response into text data and sends it to the server. The server then generates an appropriate response based on the received data and sends the result to the device. The device then uses voice synthesis technology to respond to the user, saying, "That's interesting. What era are you talking about?"

[2430] Relatives' devices

[2431] Relatives can use devices such as smartphones and PCs to send messages to elderly people from remote locations. Using a dedicated application, the relative inputs the message content and desired sending time, and sends it to the server. The server then sends the message to the user's device at the specified time and conveys it to the user by voice. For example, the user's device might say, "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready."

[2432] Emergency call function

[2433] In an emergency, if the user says "help" or presses the emergency button, the device will recognize the emergency command and automatically call 119 or 110. At the same time, the user's location information will also be sent to the emergency services. An emergency notification will also be sent to relatives.

[2434] Online shopping integration function

[2435] When a user says to the terminal, "I would like to order daily necessities," the terminal recognizes the voice command and sends a request to the server. The server obtains a list of candidate products through the online shopping system's API (for example, YAHOO Shopping API) and sends it to the terminal. The terminal presents the candidate product list to the user by voice, recognizes the user's selection, and sends it back to the server. The server executes the ordering process based on the user's selection and returns order confirmation information to the terminal. The terminal then notifies the user by voice that the order has been completed.

[2436] The system will enable elderly people to maintain daily information and communication, allow relatives to easily check on their safety, and facilitate rapid response in emergencies and the smooth purchase of daily necessities.

[2437] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2438] Step 1:

[2439] Server: Collects weather information, news information, and user hobbies and preferences every morning at 6:50.

[2440] Inputs: OpenWeatherMap API, NewsAPI, Hobby Database

[2441] Data processing and calculation: Use API to get current weather and latest news. Get the latest information relevant to the user from the hobby database.

[2442] Output: Generates a set of weather, news, and interest information and prepares it for transmission to the user's device.

[2443] Step 2:

[2444] Terminal: Information received at 7:00 a.m. is analyzed using voice synthesis technology and provided to the user.

[2445] Input: A set of weather information, news information, and hobby / preference information received from the server

[2446] Data processing and calculation: Convert received information into a voice message using Google Text-to-Speech.

[2447] Output: Provide the user with a voice message saying, "Good morning. It's a sunny day today. The temperature is 72 degrees. How's your reading going?"

[2448] Step 3:

[2449] User: Responds to the device by saying, "Good morning. I've been reading historical novels lately."

[2450] Input: Voice response to terminal

[2451] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[2452] Output: Generate the converted text data "Good morning. The book I'm reading recently is a historical novel." and prepare to send it to the server.

[2453] Step 4:

[2454] Terminal: Sends the user's voice response to the server.

[2455] Input: User's speech converted to text

[2456] Data processing and calculation: Converts text data into a protocol for sending to the server.

[2457] Output: Text data sent to the server

[2458] Step 5:

[2459] Server: Analyzes the user's response and generates an appropriate response.

[2460] Input: Text data sent by the user

[2461] Data processing and computation: Analyzing text data using natural language processing techniques to generate appropriate responses. For example, a generative AI model could be used to generate the response, "That's interesting. What era are you talking about?"

[2462] Output: Sends the generated reply to the terminal.

[2463] Step 6:

[2464] Terminal: Provides the user with a voice response to the response received from the server.

[2465] Input: Text data sent from the server

[2466] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[2467] Output: Provides the user with a spoken message saying "That's interesting. What era are you talking about?"

[2468] Step 7:

[2469] Relatives: Use the smartphone app to enter the message content and desired delivery time, and send it to the server.

[2470] Input: Message content, desired sending time

[2471] Data processing and calculation: Data entered via the smartphone app is converted into a protocol for sending to the server.

[2472] Output: Message sent to the server and desired delivery time

[2473] Step 8:

[2474] Server: Sends messages from relatives to the user's device at the specified time.

[2475] Input: Message sent by relative and desired time of sending

[2476] Data processing and calculation: The message content is stored and sent to the user's device at the specified time.

[2477] Output: Message sent to user's device

[2478] Step 9:

[2479] Terminal: Provides the user with a voice message at the specified time.

[2480] Input: Message sent from the server

[2481] Data processing and calculation: Convert text data into voice messages using Google Text-to-Speech.

[2482] Output: Provides a voice message saying "Good morning. This is a message from your grandchild. You have a doctor's appointment today, so let's get ready."

[2483] Step 10:

[2484] User: "Okay," replies the terminal.

[2485] Input: Voice response to terminal

[2486] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[2487] Output: Generates "I understand." as converted text data and prepares it to send to the server.

[2488] Step 11:

[2489] Terminal: Sends the user's voice response to the server.

[2490] Input: User's speech converted to text

[2491] Data processing and calculation: Converts text data into a protocol for sending to the server.

[2492] Output: Text data sent to the server

[2493] Step 12:

[2494] Server: If the user does not respond, perform a series of retries, eventually notifying the next of kin.

[2495] Input: No response status

[2496] Data processing and calculation: If there is no response, the message will be sent again after 10 minutes, and this will be repeated up to three times. If there is still no response, the next of kin will be notified.

[2497] Output: Message to notify relatives

[2498] Step 13:

[2499] User: In an emergency, say "help" or press the emergency button.

[2500] Input: Emergency voice command or button press

[2501] Data processing and calculation: Recognize emergency commands and collect necessary data (such as location information).

[2502] Output: Emergency services and immediate family notification

[2503] Step 14:

[2504] Terminal: Recognizes emergency commands and makes emergency calls.

[2505] Input: User voice command or button press

[2506] Data processing and calculation: Obtain location information and automatically call 119 or 110.

[2507] Output: Emergency call message and location information

[2508] Step 15:

[2509] Relatives: Receive emergency notifications.

[2510] Input: Urgent notification sent from the server

[2511] Data processing and calculation: Display emergency notifications.

[2512] Output: Emergency notification displayed on a relative's device

[2513] Step 16:

[2514] User: Says to the device, "I want to order some groceries."

[2515] Input: User's voice command

[2516] Data processing and calculation: The user's voice is captured using the device's microphone and converted into text data using Google Speech-to-Text.

[2517] Step 17:

[2518] Device: Recognizes voice commands and sends requests to the server.

[2519] Input: User's speech converted to text

[2520] Data processing and calculation: Converts text data into a protocol for sending to the server.

[2521] Output: Request sent to the server

[2522] Step 18:

[2523] Server: Obtains a list of product candidates through the API of the online shopping system.

[2524] Input: The request sent by the user

[2525] Data processing and calculation: Obtain a list of product candidates using the YAHOO Shopping API.

[2526] Output: Sends the product candidate list to the terminal.

[2527] Step 19:

[2528] Terminal: Presents a list of product candidates to the user via voice.

[2529] Input: Product candidate list sent from the server

[2530] Data processing and calculation: Use Google Text-to-Speech to convert the product candidate list into a voice message.

[2531] Output: Provide the user with a list of products as a voice message

[2532] (Application example 1)

[2533] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2534] In elderly life support systems, there is a lack of means to promote daily conversations, communicate with relatives, respond to emergencies, and support purchasing daily necessities. Furthermore, conventional systems rely on the devices used by the elderly, and the operation of the devices is complicated, making it difficult for the elderly to use them. Furthermore, there is a lack of a mechanism that allows relatives in remote locations to easily understand the status of the elderly.

[2535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2536] In this invention, the server includes a means for generating the day's weather information, news information, and topics based on the user's hobbies and preferences and providing them to the user by voice; a means for a relative to remotely send a message to the terminal and deliver the message by voice to the user at a specified time; and a means for generating appropriate responses to the user's questions using a generative AI model. This allows elderly people to easily enjoy conversations and smoothly communicate with their relatives. Furthermore, the use of a generative AI model enables more advanced and flexible responses, allowing the elderly to receive the information they need in a timely manner.

[2537] "Weather information" refers to meteorological data such as the climate conditions, temperature, and probability of precipitation in the elderly person's place of residence.

[2538] "News information" refers to the latest news data on domestic and international events, incidents, culture, economy, etc.

[2539] "Hobbies and Preference Data" refers to data that compiles information related to the user's interests and areas of interest.

[2540] A "generative AI model" refers to an artificial intelligence model that uses natural language processing technology to provide appropriate answers and generate conversations in response to questions and utterances from users.

[2541] "Providing audio" refers to a computer using speech synthesis technology to audibly convey information to a user.

[2542] "Relatives" refers to family members and blood relatives of the user.

[2543] "Sending a message from a remote location" refers to a relative sending a message to the user's terminal from a physically distant location via the Internet.

[2544] The "designated time" refers to a specific time set in advance by the relative at which the message should be conveyed to the user.

[2545] "Voice command" refers to an instruction or command given by a user through a microphone.

[2546] "Online shopping site" refers to a website for purchasing products over the Internet.

[2547] "User location information" refers to data indicating the user's current physical location.

[2548] A "central processing unit" refers to a computer that functions as the main computing resource of a server, etc., and is responsible for controlling the entire system and processing data.

[2549] "Emergency services" refers to public agencies and services that respond to emergencies, such as fire departments, police, and emergency medical services.

[2550] "Daily necessities" refers to consumables and food items necessary for the daily lives of the elderly.

[2551] This invention is a system for supporting the lives of elderly people, and is realized mainly through cooperation between a server, a terminal used by the user, and a terminal used remotely by a relative. This system has the following configuration and functions.

[2552] 1. Server:

[2553] The server periodically collects weather information, news information, and data related to the user's hobbies and preferences, and provides it to the device. Specifically, every morning at 6:50, it collects the latest information from the weather API, news API, and user hobby database, and sends it to the user's device. It also has the function of generating appropriate responses to user questions using a generative AI model. It can also receive messages from relatives and send them to the user's device at a specified time. The server also monitors the user's response status and notifies relatives if there is no response for a certain period of time.

[2554] 2. On the user's device:

[2555] The user device uses speech synthesis technology to send information to the user, recognizes the user's voice using its speech recognition function, and sends it to the server. If the user responds, "The book I've been reading recently is a historical novel," the device recognizes the speech and sends it to the server. It receives the response from the server and uses a generative AI model to generate an appropriate response, such as, "That's interesting. What period is it set in?" In an emergency, the user can issue a voice command such as "Help" or press an emergency button, and the device will automatically make an emergency call, obtain the user's location information, and provide it to medical services. It also works with online shopping sites, recognizing the voice command "I'd like to order daily necessities" and initiating the ordering process via the server.

[2556] 3. Family members' devices:

[2557] Relatives can remotely send messages to the elderly using devices such as smartphones or PCs. When the relative enters the message and the time to send it, the data is sent to a server, and the server sends the message to the user's device at the specified time. For example, a message such as "Good morning. This is a message from your grandchild. You have a hospital appointment today, so please get ready" can be spoken to the user.

[2558] Hardware and software used

[2559] Speech synthesis technology: pyttsx3

[2560] Speech recognition technology: Google Speech Recognition API

[2561] Weather API and News API: Includes any external services

[2562] Generative AI model: using natural language processing techniques

[2563] Specific use cases

[2564] An elderly person is asked at 7:00 a.m. from a terminal, "Good morning. It's a sunny day today. The temperature is 22 degrees. What book have you been reading recently?" If the user responds, "The book I've been reading recently is a historical novel," the server analyzes the data and uses a generative AI model to provide an appropriate response, such as, "That's interesting. What period is it set in?"

[2565] Prompt Sentence Examples

[2566] "Today's weather is sunny and the temperature is 22 degrees. Tell us about the book you've been reading recently."

[2567] This system will enable the elderly to enjoy everyday conversations and communicate smoothly with their relatives. It will also enable emergency response and assistance with purchasing everyday items, improving the quality of life for the elderly.

[2568] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2569] Step 1:

[2570] (Input) The server collects information from the weather API, news API, and hobby database.

[2571] (Processing) Every morning at 6:50, the server sends a request to an external service to collect data related to weather, news, and hobbies.

[2572] (Output) Collected weather information, news information, and hobby-related data.

[2573] (Specific operation) The server sends a request to the API, receives weather and news information in response, and searches and collects related information from the user's hobby database.

[2574] Step 2:

[2575] (Input) Information collected by the server.

[2576] (Processing) The server sends the collected information to the user's terminal.

[2577] (Output) Weather...

Claims

1. A system for promoting daily conversation among elderly people, comprising: means for generating weather information, news information, and topics based on the user's hobbies and preferences for the day and providing them to the user by voice; A means for a relative to remotely send a message to the terminal and deliver the message to the user by voice at a specified time; If there is no response from the user, a certain number of retries are made, and if there is still no response, a means for notifying the relatives; A means of notifying relatives if there has been no communication for a certain period of time; a means for the user to call emergency services using a voice command or a button in the event of an emergency; It will be linked to online shopping sites, allowing you to purchase everyday items using voice commands, A system including:

2. 2. The system according to claim 1, wherein the emergency notification means has a function of automatically obtaining the user's location information and providing it to emergency services.

3. 2. The system according to claim 1, wherein said conversation promoting means has a function of recognizing the user's answer by voice, transmitting the answer to a server, analyzing the response from the server, and providing an appropriate reply to the user.

Images