System

Abstract

A system is provided.SOLUTION: A system comprising: means for analyzing a daily life pattern of a user using generative artificial intelligence and generating a voice message for prompting execution of a task based on the extracted daily life pattern; means for reproducing the generated voice message in a voice that is easy for the user to hear via a speaker or a hearing aid; and means for receiving a reaction of the user by voice input and feeding back the reaction to the generative artificial intelligence.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Dementia and forgetfulness have become serious problems among the elderly, and the number of cases where this makes daily life difficult is increasing. Frequent forgetfulness makes it difficult for elderly people to live independently, and the additional stress can accelerate the progression of dementia. Existing support systems lack flexibility to respond to the individual lifestyle patterns of elderly people, making it difficult to provide effective support for their lives. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. We propose a system that includes: a means for analyzing a user's daily life patterns using a generative artificial intelligence and generating a voice message to encourage task execution based on the extracted daily life patterns; a means for playing the generated voice message in a voice that is easy for the user to hear via a speaker or hearing aid; and a means for receiving the user's response via voice input and feeding it back to the generative artificial intelligence. The system also includes a means for receiving and storing user behavioral data in a database, and includes a generative artificial intelligence that generates voice that is easy to hear even for users with age-related hearing loss, thereby providing flexible support tailored to the lifestyle patterns of each elderly person. This can reduce forgetfulness among the elderly and improve the quality of their daily lives.

[0006] "Generative AI" is AI that has the ability to analyze user behavioral data and generate voice messages with appropriate timing and content from the generated data.

[0007] "User" refers to individuals, primarily seniors, who use the system.

[0008] "Daily life patterns" refers to a series of behavioral patterns such as a user's daily actions, habits, and schedules.

[0009] A "task" refers to a specific action or task that the user must perform; examples include taking medicine or eating.

[0010] "Voice message" refers to a voice notification generated by artificial intelligence to encourage the user to take a specific action.

[0011] "Speaker" refers to an audio output device used to play audio messages.

[0012] "Hearing aid" refers to a hearing aid device used by a user to clearly hear audio messages.

[0013] "Voice input" refers to the voice data that is used when a user provides the system with voice feedback on the results of their actions, etc.

[0014] "Feedback" refers to data that the system collects based on the user's voice input and reflects in future notifications and lifestyle pattern analysis.

[0015] "Database" refers to a system for centrally managing and storing information such as user behavioral data and voice input.

[0016] "Age-related hearing loss" refers to a decrease in hearing ability caused by aging.

[0017] "Flexible support" refers to providing support at appropriate times and with appropriate content that suits the individual lifestyle patterns of users.

[0018] "Quality of life" refers to the overall living environment and conditions that enable users to live healthy and satisfying lives. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system for supporting the daily lives of elderly people. The specific program processing of this system will be explained below in natural language.

[0041] (Server-side processing)

[0042] The server first receives the user's daily behavioral data from the device. This data includes wake-up times, meal times, and medication times. The received data is stored in a database. The server then uses generative artificial intelligence to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0043] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[0044] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, as 10 o'clock approaches it will generate a voice message such as "It's 10 o'clock. It's time to take your medicine" and send this to the terminal.

[0045] (Processing on the terminal side)

[0046] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[0047] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server.

[0048] (User side processing)

[0049] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. It's time to take your medicine," the user performs the task (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions.

[0050] Specifically, the user receives the notification, takes the medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database and uses it for future notifications.

[0051] Through these processes, the present invention can support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness, and improve their quality of life.

[0052] The processing flow will be explained below.

[0053] Server-side processing

[0054] Step 1:

[0055] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[0056] Step 2:

[0057] The server stores the received data in a database.

[0058] Step 3:

[0059] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[0060] Step 4:

[0061] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[0062] Step 5:

[0063] Based on the learning results, the server creates a task schedule to notify the user.

[0064] Step 6:

[0065] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[0066] Step 7:

[0067] The generated voice message is sent to the user's terminal at an appropriate time.

[0068] Terminal side processing

[0069] Step 8:

[0070] The terminal receives the voice message sent from the server.

[0071] Step 9:

[0072] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[0073] Step 10:

[0074] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[0075] Step 11:

[0076] The received voice input is converted into text data and sent to the server.

[0077] User-side processing

[0078] Step 12:

[0079] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[0080] Step 13:

[0081] The user who receives the notification performs the instructed task (e.g., taking medicine).

[0082] Step 14:

[0083] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[0084] Processing Feedback

[0085] Step 15:

[0086] The server receives the user's reaction data transmitted from the terminal.

[0087] Step 16:

[0088] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[0089] Through these steps, the system of the present invention can support the elderly in their daily lives, reduce the difficulties of life caused by forgetfulness, and improve their quality of life.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] In modern society, elderly people often forget to take their medicine or eat at the right time, which can negatively impact their health and quality of life. Age-related hearing loss also often makes voice notifications ineffective. To address these issues, a system is needed that analyzes the daily life patterns of elderly people and notifies them of tasks at the appropriate time. However, current systems lack notification methods that take into account the hearing impairments of elderly people and a means to accurately provide feedback on user responses.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes means for analyzing the user's daily life patterns and generating voice messages to encourage task execution based on the extracted daily life patterns, means for playing the generated voice messages in a voice that is easy for the user to hear via an audio device, means for receiving the user's responses via voice input, converting them into text data through voice recognition, and feeding the text data back to the generating AI, and means for storing the user's task execution status in a database and using it for future notifications. This makes it possible to efficiently support the daily lives of the elderly, reduce the difficulties of life caused by forgetfulness and hearing impairment, and improve their quality of life.

[0095] "Generative AI" is a technology that uses machine learning algorithms and neural networks to analyze a user's daily life patterns and generate voice messages that encourage appropriate task execution.

[0096] "Daily life patterns" refer to the regularities and habits of daily life extracted from the user's daily behavioral data, including wake-up time, meal times, and medication times.

[0097] An "acoustic device" is a device that reproduces audio messages in a way that is easy for the user to hear. Examples include speakers and hearing aids.

[0098] "Voice input" refers to input made by voice from a user. This system is used to receive the user's voice response and convert it into digital data.

[0099] "Speech recognition" is a technology that converts received voice input into digital data (text data), making the content of the voice usable as specific behavioral data.

[0100] A "database" is a digital data storage system for storing and managing received behavioral data and task execution history. Examples include MySQL and PostgreSQL.

[0101] "Feedback" is the process of communicating the tasks performed and responses of the user to the system, allowing the system to further refine its learning of the user's behavioral patterns.

[0102] A "task schedule" is a timetable that prompts users to perform tasks at appropriate times based on daily life patterns analyzed by the generative artificial intelligence.

[0103] The present invention is a system for supporting the daily lives of elderly people, and is specifically implemented as follows.

[0104] Server Features

[0105] The server first receives the user's daily life data from the device. This data includes wake-up time, meal times, and medication times. Communication protocols used include HTTP and MQTT. The received data is stored in a database such as MySQL or PostgreSQL. Next, based on the data stored in the database, the server analyzes the user's daily life patterns using a generative AI model (e.g., TensorFlow or PyTorch). Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks. Based on these predictions, the server creates a task schedule to notify the user. Next, a voice message (e.g., "It's 10 o'clock. It's time to take your medicine") is generated according to the task schedule and sent to the device.

[0106] Device Features

[0107] The device receives the voice message sent from the server. After receiving it, the device plays the voice message to the user using an audio device (speaker or hearing aid). At this time, the device plays the voice message in an easy-to-listen voice, taking into consideration whether the user is using a speaker or hearing aid. After the user receives the notification, the device receives the user's response as voice input. The received voice input is converted into text data using voice recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The converted text data is sent to the server, which stores the data in a database and uses it for future notifications.

[0108] User Roles

[0109] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine") and performs the task. After completing the task, the user notifies the device by voice (e.g., "Yes, I have taken my medicine"). This information is received by the device as voice input, converted into text data, and sent to the server.

[0110] Specific examples

[0111] For example, if a user has a pattern of taking medicine at 10 o'clock every day, the server analyzes this pattern and notifies the user at 9:59, "It's 10 o'clock. Time to take your medicine." Upon receiving this notification, the user takes their medicine and reports to their device, "Yes, I have taken my medicine." This report is converted into text data and sent to the server, and is used for future notifications.

[0112] Prompt Sentence Examples

[0113] "Design an algorithm to analyze users' daily patterns and notify them of tasks at the appropriate time."

[0114] "Please explain in detail how the medication alert system for seniors works."

[0115] Through these processes, the present invention can efficiently support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness and hearing impairment, and improve their quality of life.

[0116] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0117] Step 1: Receiving and storing data

[0118] The server receives the user's daily life data (e.g., wake-up time, meal times, medication times) from the device. This data is sent to the server via the communication protocol used (e.g., HTTP, MQTT). The received data is stored in a database such as MySQL or PostgreSQL. The input is the behavioral data received from the device, and the output is the data stored in the database.

[0119] Step 2: Analyze the data

[0120] The server analyzes lifestyle patterns using a generative AI model (e.g., TensorFlow, PyTorch) based on the daily life data stored in the database. In this analysis process, the stored behavioral data is used as input, and the analyzed lifestyle patterns are obtained as output. Specifically, the server extracts the user's behavioral patterns (e.g., taking medicine at 10 o'clock every day) based on past data.

[0121] Step 3: Generate a task schedule

[0122] The server learns the user's lifestyle patterns based on the analysis results and predicts the timing of required tasks. Based on this, the server generates a task schedule to notify the user. The analyzed lifestyle patterns are used as input, and the generated task schedule is obtained as output. Specifically, the timing of a voice message such as "It's 10 o'clock. Time to take your medicine" is set.

[0123] Step 4: Generate a voice message

[0124] The server generates a voice message according to the generated task schedule. This uses speech synthesis technology (e.g., Google Text-to-Speech, Microsoft Azure Speech). The task schedule is used as input, and the voice message is generated as output. For example, a voice message such as "It's 10 o'clock. Time to take your medicine" is generated.

[0125] Step 5: Send a voice message

[0126] The server sends the generated voice message to the terminal. The generated voice message is the input, and the message is sent to the terminal as the output. The communication protocol used is HTTP, MQTT, etc.

[0127] Step 6: Receive and play voice messages

[0128] The terminal receives the voice message sent from the server. After receiving it, the terminal uses an acoustic device (e.g., speaker, hearing aid) to play the voice message in a way that is easy for the user to hear. The input is the voice message from the server, and the output is the played voice message. Specifically, a message such as "It's 10 o'clock. Time to take your medicine" is played to the user.

[0129] Step 7: Receiving and sending user responses

[0130] The user receives a voice message (e.g., "It's 10 o'clock. Time to take your medicine") and performs the notified task. Then, they say to the device, "Yes, I have taken my medicine." The device receives the user's voice input and converts it into text data using speech recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The user's voice response is the input, and text data is obtained as the output. The converted text data is then sent to the server.

[0131] Step 8: Store and use your feedback

[0132] The server receives the user's reaction data sent from the device. After receiving it, it stores it in a database and uses it for future notifications. The input is the user's reaction data (text), and the output is saved feedback data. Specifically, it is saved as data to be used for the next similar task notification.

[0133] (Application example 1)

[0134] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0135] In factories, robots and workers are required to perform tasks efficiently and in a timely manner, but conventional systems have problems with inefficient schedule management and feedback. In particular, in an environment where multiple tasks occur simultaneously, it is difficult to determine which task should be prioritized. It is also necessary to understand the task execution status and appropriately instruct the next task. This is necessary to improve work efficiency in factories and reduce errors.

[0136] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0137] In this invention, the server includes means for analyzing the user's daily life patterns using a generative AI model and generating a voice or text message to encourage task execution based on the extracted daily life patterns, means for notifying the user of the generated voice or text message via a voice output device or a display device, means for receiving the user's response by voice or text input and feeding it back to the generative AI model, and means for saving the results of the user's actions in a database and reflecting them in future task schedules. This enables robots and workers to efficiently execute tasks, provide appropriate feedback, and manage future schedules.

[0138] A "generative AI model" is a model that uses artificial intelligence to analyze data and make predictions or generate results based on the learning results.

[0139] "Daily life patterns" refers to the regularities and trends obtained by analyzing data on the user's daily actions and habits.

[0140] A "task" refers to an operation or action performed to achieve a specific purpose.

[0141] "Voice message" refers to audible notifications or instructions sent from a machine or system to a user.

[0142] "Text message" refers to notifications or instructions in the form of text.

[0143] "Audio output device" means a device for outputting an audio message in an audible form, such as a speaker.

[0144] "Display device" refers to a device for visually conveying information, including, for example, a display.

[0145] "Feedback" refers to returning user responses and results to the system.

[0146] A "database" refers to a system for systematically storing and managing data.

[0147] "Schedule management" refers to planning the timing of task execution and monitoring and adjusting its progress.

[0148] A "robot" refers to a mechanical device that operates autonomously based on programmed instructions.

[0149] 1. System Program Overview

[0150] This invention is a robot work scheduling system that aims to improve work efficiency and reduce errors in factories. The system consists of three main components: a server, a robot, and a user.

[0151] 2. Server-side program processing

[0152] The server uses a generative AI model to analyze daily task patterns. The analysis is performed using a cloud platform such as AWS or Google Cloud, and the data is stored in a relational database such as MySQL. Based on the obtained daily life patterns, a voice or text message is generated to prompt the user or robot to perform the task, providing a means to notify the user or robot at the appropriate time.

[0153] 3. Program processing on the robot side

[0154] The robot receives notifications from the server through an audio output device or a display device and executes the specified task. The specific robot hardware used is an industrial robot (e.g., ABB's YuMi or Fanuc's robot arm). When the robot completes a task, it feeds back the work results to the server, which are reflected in the next task schedule.

[0155] 4. Feedback and Data Collection

[0156] The results of tasks completed by users or robots are fed back to the server, which then stores the data in a database. This allows for more precise adjustments to the task schedule for future tasks.

[0157] 5. Specific Examples

[0158] For example, if a robot is assigned the task of assembling part A at 10 o'clock, the server will notify it 10 minutes before the task execution, saying, "Please start assembling part A at 10 o'clock." The robot will then start the work in accordance with the notification, and upon completion, it will report to the server, "Work completed, part A has been assembled." This allows for smooth task execution and appropriate feedback.

[0159] 6. Examples of prompts

[0160] prompt:

[0161] "Write a program that generates a task schedule for a factory robot to perform its set tasks on time each day. In particular, include the following information:

[0162] Work start time

[0163] Work content

[0164] Work completion notification

[0165] Additionally, when you complete a task, you can send feedback to the server and use that data to plan future tasks.

[0166] The embodiment of the present invention is configured as described above and provides a system for improving the work efficiency of a robot and reducing errors.

[0167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0168] Step 1:

[0169] The server receives behavioral data from robots and workers via terminals. This data includes the start time of work, the completion time of work, and the work content. The robot's sensor data and work logs are sent to the server as input data. The server stores this data in a database. The output is the behavioral data stored in the database.

[0170] Step 2:

[0171] The server uses the generative AI model to analyze the behavioral data received in step 1. The stored behavioral data is passed to the generative AI model as input data. The generative AI model learns patterns from past data and makes task predictions. The output is a predicted task schedule based on the analysis.

[0172] Step 3:

[0173] The server generates voice or text messages to notify robots and workers based on the predicted task schedule. The predicted task schedule is passed to the message generation system as input data. The generated messages are notified through a voice output device or a display device. The output is a message that is notified at the appropriate time.

[0174] Step 4:

[0175] The user (robot or worker) executes the task based on the notified message. Specifically, the robot assembles parts, and the worker checks the parts. The notified message is passed to the user as input data. The output is the result of the task execution.

[0176] Step 5:

[0177] When a robot or worker completes a task, the result is sent to the server as feedback. Specifically, the robot generates a text message such as "Task completed, part A assembled" and sends it to the server. The task completion result is sent to the server as input data. The output is feedback data stored in the database.

[0178] Step 6:

[0179] The server stores the feedback data in the database again and reflects it in the next and subsequent task schedules. Specifically, it analyzes the feedback data and adjusts the next task schedule more precisely. The feedback data is passed to the server as input data. The output is the adjusted task schedule for the next and subsequent tasks.

[0180] As described above, this system can improve work efficiency and accuracy within the factory by repeating the process of performing appropriate data processing and calculations based on input data at each step and generating output data.

[0181] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0182] This invention is a system for supporting the daily lives of elderly people by analyzing the user's daily life patterns, prompting them with voice messages to perform necessary tasks, and recognizing the user's emotions using an emotion engine. Specific program processing of this system is explained below in natural language.

[0183] (Server-side processing)

[0184] The server first receives the user's daily behavioral data from the device. This data includes wake-up time, meal times, and medication times. The received data is stored in a database. The server then uses generative AI to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0185] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[0186] The emotion engine also analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the determination, the emotion engine provides feedback to the generation AI and adjusts the content and tone of the voice message. As a result, notifications can be delivered that take the user's emotional state into consideration.

[0187] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the time approaches. The emotion engine will analyze the tone of the user's voice and, if the user is feeling stressed, it can notify them in a gentler tone.

[0188] (Processing on the terminal side)

[0189] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[0190] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and determines the user's emotional state.

[0191] (User side processing)

[0192] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[0193] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[0194] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[0195] The processing flow will be explained below.

[0196] Server-side processing

[0197] Step 1:

[0198] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[0199] Step 2:

[0200] The server stores the received data in a database.

[0201] Step 3:

[0202] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[0203] Step 4:

[0204] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[0205] Step 5:

[0206] Based on the learning results, the server creates a task schedule to notify the user.

[0207] Step 6:

[0208] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[0209] Step 7:

[0210] The generated voice message is sent to the user's terminal at an appropriate time.

[0211] Step 8:

[0212] The server uses an emotion engine to analyze the user's voice data received from the terminal and determine the user's emotion.

[0213] Step 9:

[0214] Based on the results of the assessment, the emotion engine adjusts the tone and content of the voice message and reflects this in the next notification.

[0215] Terminal side processing

[0216] Step 10:

[0217] The terminal receives the voice message sent from the server.

[0218] Step 11:

[0219] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[0220] Step 12:

[0221] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[0222] Step 13:

[0223] The received voice input is converted into text data and sent to the server.

[0224] User-side processing

[0225] Step 14:

[0226] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[0227] Step 15:

[0228] The user who receives the notification performs the instructed task (e.g., taking medicine).

[0229] Step 16:

[0230] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[0231] Processing Feedback

[0232] Step 17:

[0233] The server receives the user's reaction data transmitted from the terminal.

[0234] Step 18:

[0235] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[0236] Step 19:

[0237] The emotion engine analyzes the received response data and determines the user's emotional state.

[0238] Step 20:

[0239] The emotion engine feeds the results of the discrimination back to the generation AI, which then reflects them in subsequent notification messages.

[0240] Through these steps, the system of the present invention can support the daily lives of elderly people, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions through its emotion engine.

[0241] Example 2

[0242] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0243] To improve the quality of life for the elderly, timely task notifications based on the user's daily life patterns are required. However, current systems do not take the user's emotional state into consideration when providing notifications, making it difficult to prompt them to perform tasks at the appropriate time. Furthermore, for elderly people with age-related hearing loss, voice notifications tend to be unclear, resulting in the problem of not being able to understand the notification content. A system that can solve these issues is needed.

[0244] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0245] In this invention, the server includes means for analyzing the user's daily life patterns and generating a voice message for encouraging the user to perform a task based on the extracted daily life patterns, means for playing the generated voice message in a voice that is easy for the user to hear, means for receiving the user's response by voice input and feeding it back to the analysis result, and means for analyzing the user's emotional data and adjusting the content and tone of the voice message. This enables task notification at an appropriate time while taking into account the user's emotional state and hearing impairment.

[0246] The "user's daily life pattern" refers to the chronological trends and regularities of the user's daily activities.

[0247] "Task prompt" refers to an audio notification generated to prompt a user to perform a specific action (e.g., take medicine, eat, exercise, etc.).

[0248] "Easy-to-listen voice" refers to voice that has been appropriately adjusted in tone, volume, speed, word choice, etc. to make it easy for the user to understand.

[0249] "Receiving by voice input" refers to receiving the user's voice using a device such as a microphone and transmitting the content to the system.

[0250] "Feedback on the analysis results" refers to the process of reviewing the output of the generative AI model based on the received voice data and emotional data to improve the accuracy and adaptability of the system.

[0251] "Emotional data" refers to information about a user's emotional state, identified based on the user's tone of voice, facial expressions, etc.

[0252] "Adjusting the content and tone of the voice message" refers to a process of changing the information and voice quality of the message to be notified in accordance with the emotional state of the user.

[0253] This invention is a system that analyzes the user's daily life patterns, prompts them with voice messages to perform necessary tasks, and recognizes the user's emotions using an emotion engine to support the daily lives of elderly people. This system consists of three main components: a server, a terminal, and a user. Each component works in cooperation to realize life support.

[0254] Server Roles

[0255] The server first receives the user's daily behavioral data from the device, including wake-up time, meal times, medication times, etc. The received data is then stored in a database (e.g., MySQL or PostgreSQL).

[0256] Next, the server analyzes the user's lifestyle patterns using a generative AI model (such as OpenAI's GPT-3) based on the behavioral data stored in the database. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0257] Based on the analysis results, the server creates a task schedule to notify the user. According to this schedule, it generates a voice message to prompt the user to perform the task. The generated voice message is sent to the user at the appropriate time using a speech synthesis engine (e.g., Google Text-to-Speech).

[0258] Additionally, an emotion engine (for example, IBM Watson's emotion analysis or Microsoft Azure's Emotion API) analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the analysis, the emotion engine provides feedback to the generative AI model and adjusts the content and tone of the voice message. This enables notifications to take the user's emotional state into consideration.

[0259] Specifically, if the server learns from past data that the user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the o'clock approaches. The emotion engine will analyze the tone of the user's voice, and if the user is feeling stressed, it will notify them in a gentler tone.

[0260] Device Role

[0261] The device receives the voice message sent from the server. After receiving the message, the device plays it back to the user in an easy-to-listen voice. The device also takes into consideration the use of speakers and hearing aids. For example, adjustments have been made so that notifications can be heard clearly even if the user has age-related hearing loss.

[0262] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent to the server.

[0263] Specifically, after the user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user verbally tells the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and can determine the user's emotional state.

[0264] User Roles

[0265] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[0266] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[0267] Prompt Sentence Examples

[0268] "Analyze the user's daily patterns and generate the next notification content. For example, if you have learned from past data that the user takes their medicine at 10 o'clock, when it gets close to 10 o'clock, notify them with, 'It's 10 o'clock. It's time to take your medicine,' and adjust the tone according to the user's emotional state."

[0269] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[0270] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0271] Server-side processing

[0272] Step 1:

[0273] The server receives the user's daily activity data from the device, including wake-up time, meal times, and medication times. The input is the user's specific activity time data. The server stores this in a database. The output is an updated database.

[0274] Step 2:

[0275] The server retrieves the behavioral data stored in the database and analyzes the lifestyle patterns using the generative AI model. The input is past behavioral data. The generative AI analyzes this and extracts the user's daily life patterns. The output is the daily life patterns as the analysis results.

[0276] Step 3:

[0277] The server generates a task schedule based on the analyzed lifestyle patterns. The input is the daily life patterns as the analysis results. Based on this, the server predicts the next task (e.g., time to take medicine) and incorporates it into the schedule. The output is a task schedule.

[0278] Step 4:

[0279] The server generates a voice message using a generative AI model based on the task schedule. The input is the task schedule. The generative AI uses this to generate a voice notification such as "It's 10 o'clock. Time to take your medicine." The output is the generated voice message.

[0280] Step 5:

[0281] The server sends the generated voice message to the terminal. The input is the generated voice message. After sending, the server allows the user to perform a specified task. The output is the voice message sent to the terminal.

[0282] Step 6:

[0283] The server analyzes the user's voice input and facial expression data sent from the terminal and determines the user's emotional state. The input is the user's voice input and facial expression data. The emotion analysis engine analyzes this and identifies the user's emotional state. The output is the emotion determination result.

[0284] Step 7:

[0285] The server provides feedback to the generative AI model based on the emotion analysis results, which is reflected in the next notification. The input is the emotion analysis results. The generative AI adjusts the content and tone of the notification. The output is the adjusted content and tone of the next notification.

[0286] Terminal side processing

[0287] Step 1:

[0288] The terminal receives a voice message sent from the server. The input is the voice message sent from the server. As soon as it receives the message, it prepares to notify the user. The output is the received voice message.

[0289] Step 2:

[0290] The terminal plays the received voice message to the user. The input is the voice message received from the server. The terminal plays it through a speaker or hearing aid to clearly convey it to the user. The output is a voice message that the user can understand.

[0291] Step 3:

[0292] The device receives the user's response as voice input. The input is the user's voice response. This is detected and recorded through a microphone. The output is the recorded voice data.

[0293] Step 4:

[0294] The device converts the recorded voice data into text and sends it to the server. The input is voice data. It converts this into text data using speech recognition software. The output is the voice input as text data.

[0295] User-side processing

[0296] Step 1:

[0297] The user receives a voice notification from the terminal. The input is a voice message played from the terminal. The user listens to this and understands the next action. The output is the understood task content.

[0298] Step 2:

[0299] The user performs the task that is notified. The input is the content of the voice notification (e.g., "It's 10 o'clock. Time to take your medicine"). The user performs the task according to this instruction. The output is the performed task (e.g., take your medicine).

[0300] Step 3:

[0301] After completing a task, the user reports to the terminal. The input is a voice report about the completed task (e.g., "Yes, I took my medicine"). The user speaks this into a microphone. The output is a voice report.

[0302] By having each processing step work in conjunction with one another in this way, the system can support the daily lives of elderly people and improve their quality of life.

[0303] (Application example 2)

[0304] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0305] Existing systems and devices for supporting the daily lives of elderly people have difficulty effectively analyzing users' daily life patterns and notifying them of necessary tasks at the appropriate time. They also lack flexible responses to the user's emotional state, and therefore do not provide sufficient support for elderly people to spend their time safely and comfortably in self-driving vehicles.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's daily life patterns using a generative artificial intelligence and generating a voice message to encourage task execution based on the extracted daily life patterns, means for playing the generated voice message, means for receiving the user's responses via voice input and feeding them back to the generative artificial intelligence, means for collecting in-vehicle activity data and suggesting in-vehicle tasks based on the user's behavioral patterns, and means for analyzing the user's emotions and adjusting the tone and content of notifications based on the user's emotional state. This makes it possible to support elderly people's daily lives in self-driving vehicles, reduce difficulties caused by forgetfulness, and provide flexible support that takes emotions into consideration.

[0307] "Generative AI" is an intelligent system that analyzes the user's daily life patterns and automatically generates notifications and instructions to perform necessary tasks.

[0308] "Daily life patterns" refer to the actions and habits that a user repeats every day.

[0309] "Task execution" refers to the actions or activities that a user must perform at a specific time and in a specific situation.

[0310] A "voice message" is a message that conveys information or instructions to the user by voice.

[0311] A "speaker" is an output device for reproducing a voice message as sound.

[0312] "User response" refers to the response or action taken by the user after receiving the voice message.

[0313] "Feedback" is the process of gathering user responses and adjusting the system's behavior based on that information.

[0314] "In-vehicle activity data" refers to data relating to the user's behavior and situation within an autonomous vehicle.

[0315] A "behavior pattern" is a combination of actions that a user repeatedly performs under specific circumstances.

[0316] "Task Suggestion" refers to recommending or informing the user about a specific action or activity to perform.

[0317] "Emotion analysis" is the process of recognizing and analyzing emotions from the user's voice, facial expressions, etc.

[0318] "Tone of notification" refers to the sound quality and speaking characteristics of a voice message, and is an element that is adjusted according to the user's emotional state.

[0319] An embodiment of the present invention will be described.

[0320] server:

[0321] The server first receives daily behavioral data from the vehicle and the user's device, including boarding and alighting times and in-vehicle activities (e.g., reading, eating, resting, etc.). This data is then stored in a database.

[0322] The server then uses generative AI to analyze the user's lifestyle patterns based on the stored behavioral data. For example, the server uses the KMeans clustering algorithm to classify daily life patterns and determine the time of day when certain activities occur.

[0323] Based on the analysis results, the server creates a task schedule to notify the user, and generates a voice message prompting the user to perform the task according to the schedule. The generated voice message is sent to the user's device at the appropriate time.

[0324] Furthermore, the emotion analysis engine analyzes the user's voice input and facial expression data to determine their emotions. Based on the results of this analysis, the generation AI adjusts the content and tone of the voice message to deliver notifications that take their emotions into consideration.

[0325] Device:

[0326] The terminal receives the voice message sent from the server and then plays the voice message to the user through the speaker, taking into consideration that the voice message is easy to hear even if the user is wearing a hearing aid.

[0327] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent back to the server. An emotion analysis engine analyzes the voice data to identify the user's emotional state.

[0328] User:

[0329] The user receives a voice notification from the device, for example, hearing a message such as "It's 10 o'clock. Time to take your medicine." In response, the user performs a task (e.g., taking medicine) and reports the completion to the device by voice. The device receives voice feedback such as "Yes, I took my medicine" and sends it to the server. This information is used for future notifications.

[0330] The server can store this feedback in a database and reflect it in the next task notification or life pattern analysis. For example, if a user always responds with a specific tone of voice, the server can adjust the next notification based on that user's emotional state.

[0331] As a concrete example, we will show a scenario in which an elderly person is riding in an autonomous vehicle. The server receives and analyzes the riding data, generates a notification to take medicine at 10 o'clock, and notifies the user in a gentle tone if the user is feeling stressed. An example of this prompt sentence is, "To help the elderly person relax in the autonomous vehicle, please generate a voice message that gently notifies them that it is time to take their medicine at 10 o'clock. Also, if the user feels stressed, please notify them in a tone that requires special attention."

[0332] This will enable elderly people to live comfortably in self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that takes their emotions into consideration.

[0333] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0334] Step 1:

[0335] The device collects in-vehicle activity data. Using sensors and cameras, it records the time of boarding and exiting the vehicle, as well as activities in the vehicle (e.g., reading, eating, resting, etc.). The collected data is temporarily stored in the device.

[0336] Step 2:

[0337] The device sends the collected data to the server. The data is transferred to the server via a secure channel and stored in a database on the server. The input data is boarding and exiting times and in-car behavior data, and the output is stored in a database on the server.

[0338] Step 3:

[0339] The server uses generative AI to analyze the user's daily life patterns based on the stored data. Algorithms used here include KMeans clustering and other methods to classify daily life patterns. The input is the behavioral data in the database, and the output is the classified daily life patterns.

[0340] Step 4:

[0341] The server creates a task schedule based on the analysis results. It predicts tasks that the user should perform at specific times (e.g., taking medicine, staying hydrated), and generates a schedule to notify users at those times. The input is the analysis results, and the output is the task schedule.

[0342] Step 5:

[0343] The server uses the generative AI model to generate a voice message prompting the user to perform the task. The server also sets the prompt sentence when generating the message and takes into account the results of sentiment analysis. The input is the task schedule and emotion data, and the output is the voice message.

[0344] Step 6:

[0345] The server sends the generated voice message to the terminal. The terminal plays the received message to the user through the speaker and adjusts the voice settings to make it easy to hear even if the user is using a hearing aid. The input is the voice message and the output is a notification to the user.

[0346] Step 7:

[0347] The user receives a voice message and performs the instructed task. For example, if the user receives a notification that "It's 10 o'clock. It's time to take your medicine," the user will actually take the action of taking their medicine. The input is the voice message, and the output is the execution of the task.

[0348] Step 8:

[0349] The user reports task completion to the terminal by voice. The terminal converts this voice into text data and sends it to the server as feedback. The input is the user's voice and the output is text data.

[0350] Step 9:

[0351] The server receives the feedback and stores it in a database. An emotion analysis engine analyzes the user's voice data and identifies their emotional state. This information is used for future task notifications and lifestyle pattern analysis. The input is text data and emotion data, and the output is the database and analysis results.

[0352] This series of processing steps makes it possible to assist elderly people in their daily lives inside self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that is sensitive to their emotions.

[0353] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0354] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0355] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0356] [Second embodiment]

[0357] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0358] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0359] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0360] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0361] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0363] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0364] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0365] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0366] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0367] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0368] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0369] The present invention is a system for supporting the daily lives of elderly people. The specific program processing of this system will be explained below in natural language.

[0370] (Server-side processing)

[0371] The server first receives the user's daily behavioral data from the device. This data includes wake-up times, meal times, and medication times. The received data is stored in a database. The server then uses generative artificial intelligence to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0372] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[0373] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, as 10 o'clock approaches it will generate a voice message such as "It's 10 o'clock. It's time to take your medicine" and send this to the terminal.

[0374] (Processing on the terminal side)

[0375] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[0376] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server.

[0377] (User side processing)

[0378] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. It's time to take your medicine," the user performs the task (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions.

[0379] Specifically, the user receives the notification, takes the medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database and uses it for future notifications.

[0380] Through these processes, the present invention can support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness, and improve their quality of life.

[0381] The processing flow will be explained below.

[0382] Server-side processing

[0383] Step 1:

[0384] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[0385] Step 2:

[0386] The server stores the received data in a database.

[0387] Step 3:

[0388] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[0389] Step 4:

[0390] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[0391] Step 5:

[0392] Based on the learning results, the server creates a task schedule to notify the user.

[0393] Step 6:

[0394] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[0395] Step 7:

[0396] The generated voice message is sent to the user's terminal at an appropriate time.

[0397] Terminal side processing

[0398] Step 8:

[0399] The terminal receives the voice message sent from the server.

[0400] Step 9:

[0401] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[0402] Step 10:

[0403] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[0404] Step 11:

[0405] The received voice input is converted into text data and sent to the server.

[0406] User-side processing

[0407] Step 12:

[0408] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[0409] Step 13:

[0410] The user who receives the notification performs the instructed task (e.g., taking medicine).

[0411] Step 14:

[0412] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[0413] Processing Feedback

[0414] Step 15:

[0415] The server receives the user's reaction data transmitted from the terminal.

[0416] Step 16:

[0417] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[0418] Through these steps, the system of the present invention can support the elderly in their daily lives, reduce the difficulties of life caused by forgetfulness, and improve their quality of life.

[0419] Example 1

[0420] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0421] In modern society, elderly people often forget to take their medicine or eat at the right time, which can negatively impact their health and quality of life. Age-related hearing loss also often makes voice notifications ineffective. To address these issues, a system is needed that analyzes the daily life patterns of elderly people and notifies them of tasks at the appropriate time. However, current systems lack notification methods that take into account the hearing impairments of elderly people and a means to accurately provide feedback on user responses.

[0422] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0423] In this invention, the server includes means for analyzing the user's daily life patterns and generating voice messages to encourage task execution based on the extracted daily life patterns, means for playing the generated voice messages in a voice that is easy for the user to hear via an audio device, means for receiving the user's responses via voice input, converting them into text data through voice recognition, and feeding the text data back to the generating AI, and means for storing the user's task execution status in a database and using it for future notifications. This makes it possible to efficiently support the daily lives of the elderly, reduce the difficulties of life caused by forgetfulness and hearing impairment, and improve their quality of life.

[0424] "Generative AI" is a technology that uses machine learning algorithms and neural networks to analyze a user's daily life patterns and generate voice messages that encourage appropriate task execution.

[0425] "Daily life patterns" refer to the regularities and habits of daily life extracted from the user's daily behavioral data, including wake-up time, meal times, and medication times.

[0426] An "acoustic device" is a device that reproduces audio messages in a way that is easy for the user to hear. Examples include speakers and hearing aids.

[0427] "Voice input" refers to input made by voice from a user. This system is used to receive the user's voice response and convert it into digital data.

[0428] "Speech recognition" is a technology that converts received voice input into digital data (text data), making the content of the voice usable as specific behavioral data.

[0429] A "database" is a digital data storage system for storing and managing received behavioral data and task execution history. Examples include MySQL and PostgreSQL.

[0430] "Feedback" is the process of communicating the tasks performed and responses of the user to the system, allowing the system to further refine its learning of the user's behavioral patterns.

[0431] A "task schedule" is a timetable that prompts users to perform tasks at appropriate times based on daily life patterns analyzed by the generative artificial intelligence.

[0432] The present invention is a system for supporting the daily lives of elderly people, and is specifically implemented as follows.

[0433] Server Features

[0434] The server first receives the user's daily life data from the device. This data includes wake-up time, meal times, and medication times. Communication protocols used include HTTP and MQTT. The received data is stored in a database such as MySQL or PostgreSQL. Next, based on the data stored in the database, the server analyzes the user's daily life patterns using a generative AI model (e.g., TensorFlow or PyTorch). Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks. Based on these predictions, the server creates a task schedule to notify the user. Next, a voice message (e.g., "It's 10 o'clock. It's time to take your medicine") is generated according to the task schedule and sent to the device.

[0435] Device Features

[0436] The device receives the voice message sent from the server. After receiving it, the device plays the voice message to the user using an audio device (speaker or hearing aid). At this time, the device plays the voice message in an easy-to-listen voice, taking into consideration whether the user is using a speaker or hearing aid. After the user receives the notification, the device receives the user's response as voice input. The received voice input is converted into text data using voice recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The converted text data is sent to the server, which stores the data in a database and uses it for future notifications.

[0437] User Roles

[0438] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine") and performs the task. After completing the task, the user notifies the device by voice (e.g., "Yes, I have taken my medicine"). This information is received by the device as voice input, converted into text data, and sent to the server.

[0439] Specific examples

[0440] For example, if a user has a pattern of taking medicine at 10 o'clock every day, the server analyzes this pattern and notifies the user at 9:59, "It's 10 o'clock. Time to take your medicine." Upon receiving this notification, the user takes their medicine and reports to their device, "Yes, I have taken my medicine." This report is converted into text data and sent to the server, and is used for future notifications.

[0441] Prompt Sentence Examples

[0442] "Design an algorithm to analyze users' daily patterns and notify them of tasks at the appropriate time."

[0443] "Please explain in detail how the medication alert system for seniors works."

[0444] Through these processes, the present invention can efficiently support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness and hearing impairment, and improve their quality of life.

[0445] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0446] Step 1: Receiving and storing data

[0447] The server receives the user's daily life data (e.g., wake-up time, meal times, medication times) from the device. This data is sent to the server via the communication protocol used (e.g., HTTP, MQTT). The received data is stored in a database such as MySQL or PostgreSQL. The input is the behavioral data received from the device, and the output is the data stored in the database.

[0448] Step 2: Analyze the data

[0449] The server analyzes lifestyle patterns using a generative AI model (e.g., TensorFlow, PyTorch) based on the daily life data stored in the database. In this analysis process, the stored behavioral data is used as input, and the analyzed lifestyle patterns are obtained as output. Specifically, the server extracts the user's behavioral patterns (e.g., taking medicine at 10 o'clock every day) based on past data.

[0450] Step 3: Generate a task schedule

[0451] The server learns the user's lifestyle patterns based on the analysis results and predicts the timing of required tasks. Based on this, the server generates a task schedule to notify the user. The analyzed lifestyle patterns are used as input, and the generated task schedule is obtained as output. Specifically, the timing of a voice message such as "It's 10 o'clock. Time to take your medicine" is set.

[0452] Step 4: Generate a voice message

[0453] The server generates a voice message according to the generated task schedule. This uses speech synthesis technology (e.g., Google Text-to-Speech, Microsoft Azure Speech). The task schedule is used as input, and the voice message is generated as output. For example, a voice message such as "It's 10 o'clock. Time to take your medicine" is generated.

[0454] Step 5: Send a voice message

[0455] The server sends the generated voice message to the terminal. The generated voice message is the input, and the message is sent to the terminal as the output. The communication protocol used is HTTP, MQTT, etc.

[0456] Step 6: Receive and play voice messages

[0457] The terminal receives the voice message sent from the server. After receiving it, the terminal uses an acoustic device (e.g., speaker, hearing aid) to play the voice message in a way that is easy for the user to hear. The input is the voice message from the server, and the output is the played voice message. Specifically, a message such as "It's 10 o'clock. Time to take your medicine" is played to the user.

[0458] Step 7: Receiving and sending user responses

[0459] The user receives a voice message (e.g., "It's 10 o'clock. Time to take your medicine") and performs the notified task. Then, they say to the device, "Yes, I have taken my medicine." The device receives the user's voice input and converts it into text data using speech recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The user's voice response is the input, and text data is obtained as the output. The converted text data is then sent to the server.

[0460] Step 8: Store and use your feedback

[0461] The server receives the user's reaction data sent from the device. After receiving it, it stores it in a database and uses it for future notifications. The input is the user's reaction data (text), and the output is saved feedback data. Specifically, it is saved as data to be used for the next similar task notification.

[0462] (Application example 1)

[0463] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0464] In factories, robots and workers are required to perform tasks efficiently and in a timely manner, but conventional systems have problems with inefficient schedule management and feedback. In particular, in an environment where multiple tasks occur simultaneously, it is difficult to determine which task should be prioritized. It is also necessary to understand the task execution status and appropriately instruct the next task. This is necessary to improve work efficiency in factories and reduce errors.

[0465] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0466] In this invention, the server includes means for analyzing the user's daily life patterns using a generative AI model and generating a voice or text message to encourage task execution based on the extracted daily life patterns, means for notifying the user of the generated voice or text message via a voice output device or a display device, means for receiving the user's response by voice or text input and feeding it back to the generative AI model, and means for saving the results of the user's actions in a database and reflecting them in future task schedules. This enables robots and workers to efficiently execute tasks, provide appropriate feedback, and manage future schedules.

[0467] A "generative AI model" is a model that uses artificial intelligence to analyze data and make predictions or generate results based on the learning results.

[0468] "Daily life patterns" refers to the regularities and trends obtained by analyzing data on the user's daily actions and habits.

[0469] A "task" refers to an operation or action performed to achieve a specific purpose.

[0470] "Voice message" refers to audible notifications or instructions sent from a machine or system to a user.

[0471] "Text message" refers to notifications or instructions in the form of text.

[0472] "Audio output device" means a device for outputting an audio message in an audible form, such as a speaker.

[0473] "Display device" refers to a device for visually conveying information, including, for example, a display.

[0474] "Feedback" refers to returning user responses and results to the system.

[0475] A "database" refers to a system for systematically storing and managing data.

[0476] "Schedule management" refers to planning the timing of task execution and monitoring and adjusting its progress.

[0477] A "robot" refers to a mechanical device that operates autonomously based on programmed instructions.

[0478] 1. System Program Overview

[0479] This invention is a robot work scheduling system that aims to improve work efficiency and reduce errors in factories. The system consists of three main components: a server, a robot, and a user.

[0480] 2. Server-side program processing

[0481] The server uses a generative AI model to analyze daily task patterns. The analysis is performed using a cloud platform such as AWS or Google Cloud, and the data is stored in a relational database such as MySQL. Based on the obtained daily life patterns, a voice or text message is generated to prompt the user or robot to perform the task, providing a means to notify the user or robot at the appropriate time.

[0482] 3. Program processing on the robot side

[0483] The robot receives notifications from the server through an audio output device or a display device and executes the specified task. The specific robot hardware used is an industrial robot (e.g., ABB's YuMi or Fanuc's robot arm). When the robot completes a task, it feeds back the work results to the server, which are reflected in the next task schedule.

[0484] 4. Feedback and Data Collection

[0485] The results of tasks completed by users or robots are fed back to the server, which then stores the data in a database. This allows for more precise adjustments to the task schedule for future tasks.

[0486] 5. Specific Examples

[0487] For example, if a robot is assigned the task of assembling part A at 10 o'clock, the server will notify it 10 minutes before the task execution, saying, "Please start assembling part A at 10 o'clock." The robot will then start the work in accordance with the notification, and upon completion, it will report to the server, "Work completed, part A has been assembled." This allows for smooth task execution and appropriate feedback.

[0488] 6. Examples of prompts

[0489] prompt:

[0490] "Write a program that generates a task schedule for a factory robot to perform its set tasks on time each day. In particular, include the following information:

[0491] Work start time

[0492] Work content

[0493] Work completion notification

[0494] Additionally, when you complete a task, you can send feedback to the server and use that data to plan future tasks.

[0495] The embodiment of the present invention is configured as described above and provides a system for improving the work efficiency of a robot and reducing errors.

[0496] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0497] Step 1:

[0498] The server receives behavioral data from robots and workers via terminals. This data includes the start time of work, the completion time of work, and the work content. The robot's sensor data and work logs are sent to the server as input data. The server stores this data in a database. The output is the behavioral data stored in the database.

[0499] Step 2:

[0500] The server uses the generative AI model to analyze the behavioral data received in step 1. The stored behavioral data is passed to the generative AI model as input data. The generative AI model learns patterns from past data and makes task predictions. The output is a predicted task schedule based on the analysis.

[0501] Step 3:

[0502] The server generates voice or text messages to notify robots and workers based on the predicted task schedule. The predicted task schedule is passed to the message generation system as input data. The generated messages are notified through a voice output device or a display device. The output is a message that is notified at the appropriate time.

[0503] Step 4:

[0504] The user (robot or worker) executes the task based on the notified message. Specifically, the robot assembles parts, and the worker checks the parts. The notified message is passed to the user as input data. The output is the result of the task execution.

[0505] Step 5:

[0506] When a robot or worker completes a task, the result is sent to the server as feedback. Specifically, the robot generates a text message such as "Task completed, part A assembled" and sends it to the server. The task completion result is sent to the server as input data. The output is feedback data stored in the database.

[0507] Step 6:

[0508] The server stores the feedback data in the database again and reflects it in the next and subsequent task schedules. Specifically, it analyzes the feedback data and adjusts the next task schedule more precisely. The feedback data is passed to the server as input data. The output is the adjusted task schedule for the next and subsequent tasks.

[0509] As described above, this system can improve work efficiency and accuracy within the factory by repeating the process of performing appropriate data processing and calculations based on input data at each step and generating output data.

[0510] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0511] This invention is a system for supporting the daily lives of elderly people by analyzing the user's daily life patterns, prompting them with voice messages to perform necessary tasks, and recognizing the user's emotions using an emotion engine. Specific program processing of this system is explained below in natural language.

[0512] (Server-side processing)

[0513] The server first receives the user's daily behavioral data from the device. This data includes wake-up time, meal times, and medication times. The received data is stored in a database. The server then uses generative AI to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0514] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[0515] The emotion engine also analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the determination, the emotion engine provides feedback to the generation AI and adjusts the content and tone of the voice message. As a result, notifications can be delivered that take the user's emotional state into consideration.

[0516] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the time approaches. The emotion engine will analyze the tone of the user's voice and, if the user is feeling stressed, it can notify them in a gentler tone.

[0517] (Processing on the terminal side)

[0518] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[0519] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and determines the user's emotional state.

[0520] (User side processing)

[0521] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[0522] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[0523] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[0524] The processing flow will be explained below.

[0525] Server-side processing

[0526] Step 1:

[0527] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[0528] Step 2:

[0529] The server stores the received data in a database.

[0530] Step 3:

[0531] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[0532] Step 4:

[0533] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[0534] Step 5:

[0535] Based on the learning results, the server creates a task schedule to notify the user.

[0536] Step 6:

[0537] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[0538] Step 7:

[0539] The generated voice message is sent to the user's terminal at an appropriate time.

[0540] Step 8:

[0541] The server uses an emotion engine to analyze the user's voice data received from the terminal and determine the user's emotion.

[0542] Step 9:

[0543] Based on the results of the assessment, the emotion engine adjusts the tone and content of the voice message and reflects this in the next notification.

[0544] Terminal side processing

[0545] Step 10:

[0546] The terminal receives the voice message sent from the server.

[0547] Step 11:

[0548] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[0549] Step 12:

[0550] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[0551] Step 13:

[0552] The received voice input is converted into text data and sent to the server.

[0553] User-side processing

[0554] Step 14:

[0555] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[0556] Step 15:

[0557] The user who receives the notification performs the instructed task (e.g., taking medicine).

[0558] Step 16:

[0559] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[0560] Processing Feedback

[0561] Step 17:

[0562] The server receives the user's reaction data transmitted from the terminal.

[0563] Step 18:

[0564] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[0565] Step 19:

[0566] The emotion engine analyzes the received response data and determines the user's emotional state.

[0567] Step 20:

[0568] The emotion engine feeds the results of the discrimination back to the generation AI, which then reflects them in subsequent notification messages.

[0569] Through these steps, the system of the present invention can support the daily lives of elderly people, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions through its emotion engine.

[0570] Example 2

[0571] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] To improve the quality of life for the elderly, timely task notifications based on the user's daily life patterns are required. However, current systems do not take the user's emotional state into consideration when providing notifications, making it difficult to prompt them to perform tasks at the appropriate time. Furthermore, for elderly people with age-related hearing loss, voice notifications tend to be unclear, resulting in the problem of not being able to understand the notification content. A system that can solve these issues is needed.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0574] In this invention, the server includes means for analyzing the user's daily life patterns and generating a voice message for encouraging the user to perform a task based on the extracted daily life patterns, means for playing the generated voice message in a voice that is easy for the user to hear, means for receiving the user's response by voice input and feeding it back to the analysis result, and means for analyzing the user's emotional data and adjusting the content and tone of the voice message. This enables task notification at an appropriate time while taking into account the user's emotional state and hearing impairment.

[0575] The "user's daily life pattern" refers to the chronological trends and regularities of the user's daily activities.

[0576] "Task prompt" refers to an audio notification generated to prompt a user to perform a specific action (e.g., take medicine, eat, exercise, etc.).

[0577] "Easy-to-listen voice" refers to voice that has been appropriately adjusted in tone, volume, speed, word choice, etc. to make it easy for the user to understand.

[0578] "Receiving by voice input" refers to receiving the user's voice using a device such as a microphone and transmitting the content to the system.

[0579] "Feedback on the analysis results" refers to the process of reviewing the output of the generative AI model based on the received voice data and emotional data to improve the accuracy and adaptability of the system.

[0580] "Emotional data" refers to information about a user's emotional state, identified based on the user's tone of voice, facial expressions, etc.

[0581] "Adjusting the content and tone of the voice message" refers to a process of changing the information and voice quality of the message to be notified in accordance with the emotional state of the user.

[0582] This invention is a system that analyzes the user's daily life patterns, prompts them with voice messages to perform necessary tasks, and recognizes the user's emotions using an emotion engine to support the daily lives of elderly people. This system consists of three main components: a server, a terminal, and a user. Each component works in cooperation to realize life support.

[0583] Server Roles

[0584] The server first receives the user's daily behavioral data from the device, including wake-up time, meal times, medication times, etc. The received data is then stored in a database (e.g., MySQL or PostgreSQL).

[0585] Next, the server analyzes the user's lifestyle patterns using a generative AI model (such as OpenAI's GPT-3) based on the behavioral data stored in the database. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0586] Based on the analysis results, the server creates a task schedule to notify the user. According to this schedule, it generates a voice message to prompt the user to perform the task. The generated voice message is sent to the user at the appropriate time using a speech synthesis engine (e.g., Google Text-to-Speech).

[0587] Additionally, an emotion engine (for example, IBM Watson's emotion analysis or Microsoft Azure's Emotion API) analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the analysis, the emotion engine provides feedback to the generative AI model and adjusts the content and tone of the voice message. This enables notifications to take the user's emotional state into consideration.

[0588] Specifically, if the server learns from past data that the user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the o'clock approaches. The emotion engine will analyze the tone of the user's voice, and if the user is feeling stressed, it will notify them in a gentler tone.

[0589] Device Role

[0590] The device receives the voice message sent from the server. After receiving the message, the device plays it back to the user in an easy-to-listen voice. The device also takes into consideration the use of speakers and hearing aids. For example, adjustments have been made so that notifications can be heard clearly even if the user has age-related hearing loss.

[0591] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent to the server.

[0592] Specifically, after the user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user verbally tells the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and can determine the user's emotional state.

[0593] User Roles

[0594] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[0595] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[0596] Prompt Sentence Examples

[0597] "Analyze the user's daily patterns and generate the next notification content. For example, if you have learned from past data that the user takes their medicine at 10 o'clock, when it gets close to 10 o'clock, notify them with, 'It's 10 o'clock. It's time to take your medicine,' and adjust the tone according to the user's emotional state."

[0598] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[0599] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0600] Server-side processing

[0601] Step 1:

[0602] The server receives the user's daily activity data from the device, including wake-up time, meal times, and medication times. The input is the user's specific activity time data. The server stores this in a database. The output is an updated database.

[0603] Step 2:

[0604] The server retrieves the behavioral data stored in the database and analyzes the lifestyle patterns using the generative AI model. The input is past behavioral data. The generative AI analyzes this and extracts the user's daily life patterns. The output is the daily life patterns as the analysis results.

[0605] Step 3:

[0606] The server generates a task schedule based on the analyzed lifestyle patterns. The input is the daily life patterns as the analysis results. Based on this, the server predicts the next task (e.g., time to take medicine) and incorporates it into the schedule. The output is a task schedule.

[0607] Step 4:

[0608] The server generates a voice message using a generative AI model based on the task schedule. The input is the task schedule. The generative AI uses this to generate a voice notification such as "It's 10 o'clock. Time to take your medicine." The output is the generated voice message.

[0609] Step 5:

[0610] The server sends the generated voice message to the terminal. The input is the generated voice message. After sending, the server allows the user to perform a specified task. The output is the voice message sent to the terminal.

[0611] Step 6:

[0612] The server analyzes the user's voice input and facial expression data sent from the terminal and determines the user's emotional state. The input is the user's voice input and facial expression data. The emotion analysis engine analyzes this and identifies the user's emotional state. The output is the emotion determination result.

[0613] Step 7:

[0614] The server provides feedback to the generative AI model based on the emotion analysis results, which is reflected in the next notification. The input is the emotion analysis results. The generative AI adjusts the content and tone of the notification. The output is the adjusted content and tone of the next notification.

[0615] Terminal side processing

[0616] Step 1:

[0617] The terminal receives a voice message sent from the server. The input is the voice message sent from the server. As soon as it receives the message, it prepares to notify the user. The output is the received voice message.

[0618] Step 2:

[0619] The terminal plays the received voice message to the user. The input is the voice message received from the server. The terminal plays it through a speaker or hearing aid to clearly convey it to the user. The output is a voice message that the user can understand.

[0620] Step 3:

[0621] The device receives the user's response as voice input. The input is the user's voice response. This is detected and recorded through a microphone. The output is the recorded voice data.

[0622] Step 4:

[0623] The device converts the recorded voice data into text and sends it to the server. The input is voice data. It converts this into text data using speech recognition software. The output is the voice input as text data.

[0624] User-side processing

[0625] Step 1:

[0626] The user receives a voice notification from the terminal. The input is a voice message played from the terminal. The user listens to this and understands the next action. The output is the understood task content.

[0627] Step 2:

[0628] The user performs the task that is notified. The input is the content of the voice notification (e.g., "It's 10 o'clock. Time to take your medicine"). The user performs the task according to this instruction. The output is the performed task (e.g., take your medicine).

[0629] Step 3:

[0630] After completing a task, the user reports to the terminal. The input is a voice report about the completed task (e.g., "Yes, I took my medicine"). The user speaks this into a microphone. The output is a voice report.

[0631] By having each processing step work in conjunction with one another in this way, the system can support the daily lives of elderly people and improve their quality of life.

[0632] (Application example 2)

[0633] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0634] Existing systems and devices for supporting the daily lives of elderly people have difficulty effectively analyzing users' daily life patterns and notifying them of necessary tasks at the appropriate time. They also lack flexible responses to the user's emotional state, and therefore do not provide sufficient support for elderly people to spend their time safely and comfortably in self-driving vehicles.

[0635] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's daily life patterns using a generative artificial intelligence and generating a voice message to encourage task execution based on the extracted daily life patterns, means for playing the generated voice message, means for receiving the user's responses via voice input and feeding them back to the generative artificial intelligence, means for collecting in-vehicle activity data and suggesting in-vehicle tasks based on the user's behavioral patterns, and means for analyzing the user's emotions and adjusting the tone and content of notifications based on the user's emotional state. This makes it possible to support elderly people's daily lives in self-driving vehicles, reduce difficulties caused by forgetfulness, and provide flexible support that takes emotions into consideration.

[0636] "Generative AI" is an intelligent system that analyzes the user's daily life patterns and automatically generates notifications and instructions to perform necessary tasks.

[0637] "Daily life patterns" refer to the actions and habits that a user repeats every day.

[0638] "Task execution" refers to the actions or activities that a user must perform at a specific time and in a specific situation.

[0639] A "voice message" is a message that conveys information or instructions to the user by voice.

[0640] A "speaker" is an output device for reproducing a voice message as sound.

[0641] "User response" refers to the response or action taken by the user after receiving the voice message.

[0642] "Feedback" is the process of gathering user responses and adjusting the system's behavior based on that information.

[0643] "In-vehicle activity data" refers to data relating to the user's behavior and situation within an autonomous vehicle.

[0644] A "behavior pattern" is a combination of actions that a user repeatedly performs under specific circumstances.

[0645] "Task Suggestion" refers to recommending or informing the user about a specific action or activity to perform.

[0646] "Emotion analysis" is the process of recognizing and analyzing emotions from the user's voice, facial expressions, etc.

[0647] "Tone of notification" refers to the sound quality and speaking characteristics of a voice message, and is an element that is adjusted according to the user's emotional state.

[0648] An embodiment of the present invention will be described.

[0649] server:

[0650] The server first receives daily behavioral data from the vehicle and the user's device, including boarding and alighting times and in-vehicle activities (e.g., reading, eating, resting, etc.). This data is then stored in a database.

[0651] The server then uses generative AI to analyze the user's lifestyle patterns based on the stored behavioral data. For example, the server uses the KMeans clustering algorithm to classify daily life patterns and determine the time of day when certain activities occur.

[0652] Based on the analysis results, the server creates a task schedule to notify the user, and generates a voice message prompting the user to perform the task according to the schedule. The generated voice message is sent to the user's device at the appropriate time.

[0653] Furthermore, the emotion analysis engine analyzes the user's voice input and facial expression data to determine their emotions. Based on the results of this analysis, the generation AI adjusts the content and tone of the voice message to deliver notifications that take their emotions into consideration.

[0654] Device:

[0655] The terminal receives the voice message sent from the server and then plays the voice message to the user through the speaker, taking into consideration that the voice message is easy to hear even if the user is wearing a hearing aid.

[0656] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent back to the server. An emotion analysis engine analyzes the voice data to identify the user's emotional state.

[0657] User:

[0658] The user receives a voice notification from the device, for example, hearing a message such as "It's 10 o'clock. Time to take your medicine." In response, the user performs a task (e.g., taking medicine) and reports the completion to the device by voice. The device receives voice feedback such as "Yes, I took my medicine" and sends it to the server. This information is used for future notifications.

[0659] The server can store this feedback in a database and reflect it in the next task notification or life pattern analysis. For example, if a user always responds with a specific tone of voice, the server can adjust the next notification based on that user's emotional state.

[0660] As a concrete example, we will show a scenario in which an elderly person is riding in an autonomous vehicle. The server receives and analyzes the riding data, generates a notification to take medicine at 10 o'clock, and notifies the user in a gentle tone if the user is feeling stressed. An example of this prompt sentence is, "To help the elderly person relax in the autonomous vehicle, please generate a voice message that gently notifies them that it is time to take their medicine at 10 o'clock. Also, if the user feels stressed, please notify them in a tone that requires special attention."

[0661] This will enable elderly people to live comfortably in self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that takes their emotions into consideration.

[0662] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0663] Step 1:

[0664] The device collects in-vehicle activity data. Using sensors and cameras, it records the time of boarding and exiting the vehicle, as well as activities in the vehicle (e.g., reading, eating, resting, etc.). The collected data is temporarily stored in the device.

[0665] Step 2:

[0666] The device sends the collected data to the server. The data is transferred to the server via a secure channel and stored in a database on the server. The input data is boarding and exiting times and in-car behavior data, and the output is stored in a database on the server.

[0667] Step 3:

[0668] The server uses generative AI to analyze the user's daily life patterns based on the stored data. Algorithms used here include KMeans clustering and other methods to classify daily life patterns. The input is the behavioral data in the database, and the output is the classified daily life patterns.

[0669] Step 4:

[0670] The server creates a task schedule based on the analysis results. It predicts tasks that the user should perform at specific times (e.g., taking medicine, staying hydrated), and generates a schedule to notify users at those times. The input is the analysis results, and the output is the task schedule.

[0671] Step 5:

[0672] The server uses the generative AI model to generate a voice message prompting the user to perform the task. The server also sets the prompt sentence when generating the message and takes into account the results of sentiment analysis. The input is the task schedule and emotion data, and the output is the voice message.

[0673] Step 6:

[0674] The server sends the generated voice message to the terminal. The terminal plays the received message to the user through the speaker and adjusts the voice settings to make it easy to hear even if the user is using a hearing aid. The input is the voice message and the output is a notification to the user.

[0675] Step 7:

[0676] The user receives a voice message and performs the instructed task. For example, if the user receives a notification that "It's 10 o'clock. It's time to take your medicine," the user will actually take the action of taking their medicine. The input is the voice message, and the output is the execution of the task.

[0677] Step 8:

[0678] The user reports task completion to the terminal by voice. The terminal converts this voice into text data and sends it to the server as feedback. The input is the user's voice and the output is text data.

[0679] Step 9:

[0680] The server receives the feedback and stores it in a database. An emotion analysis engine analyzes the user's voice data and identifies their emotional state. This information is used for future task notifications and lifestyle pattern analysis. The input is text data and emotion data, and the output is the database and analysis results.

[0681] This series of processing steps makes it possible to assist elderly people in their daily lives inside self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that is sensitive to their emotions.

[0682] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0683] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0684] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0685] [Third embodiment]

[0686] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0687] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0688] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0689] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0690] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0691] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0692] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0693] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0694] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0695] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0696] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0697] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0698] The present invention is a system for supporting the daily lives of elderly people. The specific program processing of this system will be explained below in natural language.

[0699] (Server-side processing)

[0700] The server first receives the user's daily behavioral data from the device. This data includes wake-up times, meal times, and medication times. The received data is stored in a database. The server then uses generative artificial intelligence to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0701] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[0702] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, as 10 o'clock approaches it will generate a voice message such as "It's 10 o'clock. It's time to take your medicine" and send this to the terminal.

[0703] (Processing on the terminal side)

[0704] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[0705] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server.

[0706] (User side processing)

[0707] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. It's time to take your medicine," the user performs the task (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions.

[0708] Specifically, the user receives the notification, takes the medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database and uses it for future notifications.

[0709] Through these processes, the present invention can support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness, and improve their quality of life.

[0710] The processing flow will be explained below.

[0711] Server-side processing

[0712] Step 1:

[0713] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[0714] Step 2:

[0715] The server stores the received data in a database.

[0716] Step 3:

[0717] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[0718] Step 4:

[0719] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[0720] Step 5:

[0721] Based on the learning results, the server creates a task schedule to notify the user.

[0722] Step 6:

[0723] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[0724] Step 7:

[0725] The generated voice message is sent to the user's terminal at an appropriate time.

[0726] Terminal side processing

[0727] Step 8:

[0728] The terminal receives the voice message sent from the server.

[0729] Step 9:

[0730] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[0731] Step 10:

[0732] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[0733] Step 11:

[0734] The received voice input is converted into text data and sent to the server.

[0735] User-side processing

[0736] Step 12:

[0737] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[0738] Step 13:

[0739] The user who receives the notification performs the instructed task (e.g., taking medicine).

[0740] Step 14:

[0741] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[0742] Processing Feedback

[0743] Step 15:

[0744] The server receives the user's reaction data transmitted from the terminal.

[0745] Step 16:

[0746] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[0747] Through these steps, the system of the present invention can support the elderly in their daily lives, reduce the difficulties of life caused by forgetfulness, and improve their quality of life.

[0748] Example 1

[0749] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0750] In modern society, elderly people often forget to take their medicine or eat at the right time, which can negatively impact their health and quality of life. Age-related hearing loss also often makes voice notifications ineffective. To address these issues, a system is needed that analyzes the daily life patterns of elderly people and notifies them of tasks at the appropriate time. However, current systems lack notification methods that take into account the hearing impairments of elderly people and a means to accurately provide feedback on user responses.

[0751] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0752] In this invention, the server includes means for analyzing the user's daily life patterns and generating voice messages to encourage task execution based on the extracted daily life patterns, means for playing the generated voice messages in a voice that is easy for the user to hear via an audio device, means for receiving the user's responses via voice input, converting them into text data through voice recognition, and feeding the text data back to the generating AI, and means for storing the user's task execution status in a database and using it for future notifications. This makes it possible to efficiently support the daily lives of the elderly, reduce the difficulties of life caused by forgetfulness and hearing impairment, and improve their quality of life.

[0753] "Generative AI" is a technology that uses machine learning algorithms and neural networks to analyze a user's daily life patterns and generate voice messages that encourage appropriate task execution.

[0754] "Daily life patterns" refer to the regularities and habits of daily life extracted from the user's daily behavioral data, including wake-up time, meal times, and medication times.

[0755] An "acoustic device" is a device that reproduces audio messages in a way that is easy for the user to hear. Examples include speakers and hearing aids.

[0756] "Voice input" refers to input made by voice from a user. This system is used to receive the user's voice response and convert it into digital data.

[0757] "Speech recognition" is a technology that converts received voice input into digital data (text data), making the content of the voice usable as specific behavioral data.

[0758] A "database" is a digital data storage system for storing and managing received behavioral data and task execution history. Examples include MySQL and PostgreSQL.

[0759] "Feedback" is the process of communicating the tasks performed and responses of the user to the system, allowing the system to further refine its learning of the user's behavioral patterns.

[0760] A "task schedule" is a timetable that prompts users to perform tasks at appropriate times based on daily life patterns analyzed by the generative artificial intelligence.

[0761] The present invention is a system for supporting the daily lives of elderly people, and is specifically implemented as follows.

[0762] Server Features

[0763] The server first receives the user's daily life data from the device. This data includes wake-up time, meal times, and medication times. Communication protocols used include HTTP and MQTT. The received data is stored in a database such as MySQL or PostgreSQL. Next, based on the data stored in the database, the server analyzes the user's daily life patterns using a generative AI model (e.g., TensorFlow or PyTorch). Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks. Based on these predictions, the server creates a task schedule to notify the user. Next, a voice message (e.g., "It's 10 o'clock. It's time to take your medicine") is generated according to the task schedule and sent to the device.

[0764] Device Features

[0765] The device receives the voice message sent from the server. After receiving it, the device plays the voice message to the user using an audio device (speaker or hearing aid). At this time, the device plays the voice message in an easy-to-listen voice, taking into consideration whether the user is using a speaker or hearing aid. After the user receives the notification, the device receives the user's response as voice input. The received voice input is converted into text data using voice recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The converted text data is sent to the server, which stores the data in a database and uses it for future notifications.

[0766] User Roles

[0767] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine") and performs the task. After completing the task, the user notifies the device by voice (e.g., "Yes, I have taken my medicine"). This information is received by the device as voice input, converted into text data, and sent to the server.

[0768] Specific examples

[0769] For example, if a user has a pattern of taking medicine at 10 o'clock every day, the server analyzes this pattern and notifies the user at 9:59, "It's 10 o'clock. Time to take your medicine." Upon receiving this notification, the user takes their medicine and reports to their device, "Yes, I have taken my medicine." This report is converted into text data and sent to the server, and is used for future notifications.

[0770] Prompt Sentence Examples

[0771] "Design an algorithm to analyze users' daily patterns and notify them of tasks at the appropriate time."

[0772] "Please explain in detail how the medication alert system for seniors works."

[0773] Through these processes, the present invention can efficiently support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness and hearing impairment, and improve their quality of life.

[0774] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0775] Step 1: Receiving and storing data

[0776] The server receives the user's daily life data (e.g., wake-up time, meal times, medication times) from the device. This data is sent to the server via the communication protocol used (e.g., HTTP, MQTT). The received data is stored in a database such as MySQL or PostgreSQL. The input is the behavioral data received from the device, and the output is the data stored in the database.

[0777] Step 2: Analyze the data

[0778] The server analyzes lifestyle patterns using a generative AI model (e.g., TensorFlow, PyTorch) based on the daily life data stored in the database. In this analysis process, the stored behavioral data is used as input, and the analyzed lifestyle patterns are obtained as output. Specifically, the server extracts the user's behavioral patterns (e.g., taking medicine at 10 o'clock every day) based on past data.

[0779] Step 3: Generate a task schedule

[0780] The server learns the user's lifestyle patterns based on the analysis results and predicts the timing of required tasks. Based on this, the server generates a task schedule to notify the user. The analyzed lifestyle patterns are used as input, and the generated task schedule is obtained as output. Specifically, the timing of a voice message such as "It's 10 o'clock. Time to take your medicine" is set.

[0781] Step 4: Generate a voice message

[0782] The server generates a voice message according to the generated task schedule. This uses speech synthesis technology (e.g., Google Text-to-Speech, Microsoft Azure Speech). The task schedule is used as input, and the voice message is generated as output. For example, a voice message such as "It's 10 o'clock. Time to take your medicine" is generated.

[0783] Step 5: Send a voice message

[0784] The server sends the generated voice message to the terminal. The generated voice message is the input, and the message is sent to the terminal as the output. The communication protocol used is HTTP, MQTT, etc.

[0785] Step 6: Receive and play voice messages

[0786] The terminal receives the voice message sent from the server. After receiving it, the terminal uses an acoustic device (e.g., speaker, hearing aid) to play the voice message in a way that is easy for the user to hear. The input is the voice message from the server, and the output is the played voice message. Specifically, a message such as "It's 10 o'clock. Time to take your medicine" is played to the user.

[0787] Step 7: Receiving and sending user responses

[0788] The user receives a voice message (e.g., "It's 10 o'clock. Time to take your medicine") and performs the notified task. Then, they say to the device, "Yes, I have taken my medicine." The device receives the user's voice input and converts it into text data using speech recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The user's voice response is the input, and text data is obtained as the output. The converted text data is then sent to the server.

[0789] Step 8: Store and use your feedback

[0790] The server receives the user's reaction data sent from the device. After receiving it, it stores it in a database and uses it for future notifications. The input is the user's reaction data (text), and the output is saved feedback data. Specifically, it is saved as data to be used for the next similar task notification.

[0791] (Application example 1)

[0792] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0793] In factories, robots and workers are required to perform tasks efficiently and in a timely manner, but conventional systems have problems with inefficient schedule management and feedback. In particular, in an environment where multiple tasks occur simultaneously, it is difficult to determine which task should be prioritized. It is also necessary to understand the task execution status and appropriately instruct the next task. This is necessary to improve work efficiency in factories and reduce errors.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0795] In this invention, the server includes means for analyzing the user's daily life patterns using a generative AI model and generating a voice or text message to encourage task execution based on the extracted daily life patterns, means for notifying the user of the generated voice or text message via a voice output device or a display device, means for receiving the user's response by voice or text input and feeding it back to the generative AI model, and means for saving the results of the user's actions in a database and reflecting them in future task schedules. This enables robots and workers to efficiently execute tasks, provide appropriate feedback, and manage future schedules.

[0796] A "generative AI model" is a model that uses artificial intelligence to analyze data and make predictions or generate results based on the learning results.

[0797] "Daily life patterns" refers to the regularities and trends obtained by analyzing data on the user's daily actions and habits.

[0798] A "task" refers to an operation or action performed to achieve a specific purpose.

[0799] "Voice message" refers to audible notifications or instructions sent from a machine or system to a user.

[0800] "Text message" refers to notifications or instructions in the form of text.

[0801] "Audio output device" means a device for outputting an audio message in an audible form, such as a speaker.

[0802] "Display device" refers to a device for visually conveying information, including, for example, a display.

[0803] "Feedback" refers to returning user responses and results to the system.

[0804] A "database" refers to a system for systematically storing and managing data.

[0805] "Schedule management" refers to planning the timing of task execution and monitoring and adjusting its progress.

[0806] A "robot" refers to a mechanical device that operates autonomously based on programmed instructions.

[0807] 1. System Program Overview

[0808] This invention is a robot work scheduling system that aims to improve work efficiency and reduce errors in factories. The system consists of three main components: a server, a robot, and a user.

[0809] 2. Server-side program processing

[0810] The server uses a generative AI model to analyze daily task patterns. The analysis is performed using a cloud platform such as AWS or Google Cloud, and the data is stored in a relational database such as MySQL. Based on the obtained daily life patterns, a voice or text message is generated to prompt the user or robot to perform the task, providing a means to notify the user or robot at the appropriate time.

[0811] 3. Program processing on the robot side

[0812] The robot receives notifications from the server through an audio output device or a display device and executes the specified task. The specific robot hardware used is an industrial robot (e.g., ABB's YuMi or Fanuc's robot arm). When the robot completes a task, it feeds back the work results to the server, which are reflected in the next task schedule.

[0813] 4. Feedback and Data Collection

[0814] The results of tasks completed by users or robots are fed back to the server, which then stores the data in a database. This allows for more precise adjustments to the task schedule for future tasks.

[0815] 5. Specific Examples

[0816] For example, if a robot is assigned the task of assembling part A at 10 o'clock, the server will notify it 10 minutes before the task execution, saying, "Please start assembling part A at 10 o'clock." The robot will then start the work in accordance with the notification, and upon completion, it will report to the server, "Work completed, part A has been assembled." This allows for smooth task execution and appropriate feedback.

[0817] 6. Examples of prompts

[0818] prompt:

[0819] "Write a program that generates a task schedule for a factory robot to perform its set tasks on time each day. In particular, include the following information:

[0820] Work start time

[0821] Work content

[0822] Work completion notification

[0823] Additionally, when you complete a task, you can send feedback to the server and use that data to plan future tasks.

[0824] The embodiment of the present invention is configured as described above and provides a system for improving the work efficiency of a robot and reducing errors.

[0825] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0826] Step 1:

[0827] The server receives behavioral data from robots and workers via terminals. This data includes the start time of work, the completion time of work, and the work content. The robot's sensor data and work logs are sent to the server as input data. The server stores this data in a database. The output is the behavioral data stored in the database.

[0828] Step 2:

[0829] The server uses the generative AI model to analyze the behavioral data received in step 1. The stored behavioral data is passed to the generative AI model as input data. The generative AI model learns patterns from past data and makes task predictions. The output is a predicted task schedule based on the analysis.

[0830] Step 3:

[0831] The server generates voice or text messages to notify robots and workers based on the predicted task schedule. The predicted task schedule is passed to the message generation system as input data. The generated messages are notified through a voice output device or a display device. The output is a message that is notified at the appropriate time.

[0832] Step 4:

[0833] The user (robot or worker) executes the task based on the notified message. Specifically, the robot assembles parts, and the worker checks the parts. The notified message is passed to the user as input data. The output is the result of the task execution.

[0834] Step 5:

[0835] When a robot or worker completes a task, the result is sent to the server as feedback. Specifically, the robot generates a text message such as "Task completed, part A assembled" and sends it to the server. The task completion result is sent to the server as input data. The output is feedback data stored in the database.

[0836] Step 6:

[0837] The server stores the feedback data in the database again and reflects it in the next and subsequent task schedules. Specifically, it analyzes the feedback data and adjusts the next task schedule more precisely. The feedback data is passed to the server as input data. The output is the adjusted task schedule for the next and subsequent tasks.

[0838] As described above, this system can improve work efficiency and accuracy within the factory by repeating the process of performing appropriate data processing and calculations based on input data at each step and generating output data.

[0839] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0840] This invention is a system for supporting the daily lives of elderly people by analyzing the user's daily life patterns, prompting them with voice messages to perform necessary tasks, and recognizing the user's emotions using an emotion engine. Specific program processing of this system is explained below in natural language.

[0841] (Server-side processing)

[0842] The server first receives the user's daily behavioral data from the device. This data includes wake-up time, meal times, and medication times. The received data is stored in a database. The server then uses generative AI to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0843] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[0844] The emotion engine also analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the determination, the emotion engine provides feedback to the generation AI and adjusts the content and tone of the voice message. As a result, notifications can be delivered that take the user's emotional state into consideration.

[0845] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the time approaches. The emotion engine will analyze the tone of the user's voice and, if the user is feeling stressed, it can notify them in a gentler tone.

[0846] (Processing on the terminal side)

[0847] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[0848] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and determines the user's emotional state.

[0849] (User side processing)

[0850] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[0851] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[0852] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[0853] The processing flow will be explained below.

[0854] Server-side processing

[0855] Step 1:

[0856] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[0857] Step 2:

[0858] The server stores the received data in a database.

[0859] Step 3:

[0860] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[0861] Step 4:

[0862] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[0863] Step 5:

[0864] Based on the learning results, the server creates a task schedule to notify the user.

[0865] Step 6:

[0866] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[0867] Step 7:

[0868] The generated voice message is sent to the user's terminal at an appropriate time.

[0869] Step 8:

[0870] The server uses an emotion engine to analyze the user's voice data received from the terminal and determine the user's emotion.

[0871] Step 9:

[0872] Based on the results of the assessment, the emotion engine adjusts the tone and content of the voice message and reflects this in the next notification.

[0873] Terminal side processing

[0874] Step 10:

[0875] The terminal receives the voice message sent from the server.

[0876] Step 11:

[0877] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[0878] Step 12:

[0879] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[0880] Step 13:

[0881] The received voice input is converted into text data and sent to the server.

[0882] User-side processing

[0883] Step 14:

[0884] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[0885] Step 15:

[0886] The user who receives the notification performs the instructed task (e.g., taking medicine).

[0887] Step 16:

[0888] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[0889] Processing Feedback

[0890] Step 17:

[0891] The server receives the user's reaction data transmitted from the terminal.

[0892] Step 18:

[0893] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[0894] Step 19:

[0895] The emotion engine analyzes the received response data and determines the user's emotional state.

[0896] Step 20:

[0897] The emotion engine feeds the results of the discrimination back to the generation AI, which then reflects them in subsequent notification messages.

[0898] Through these steps, the system of the present invention can support the daily lives of elderly people, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions through its emotion engine.

[0899] Example 2

[0900] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0901] To improve the quality of life for the elderly, timely task notifications based on the user's daily life patterns are required. However, current systems do not take the user's emotional state into consideration when providing notifications, making it difficult to prompt them to perform tasks at the appropriate time. Furthermore, for elderly people with age-related hearing loss, voice notifications tend to be unclear, resulting in the problem of not being able to understand the notification content. A system that can solve these issues is needed.

[0902] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0903] In this invention, the server includes means for analyzing the user's daily life patterns and generating a voice message for encouraging the user to perform a task based on the extracted daily life patterns, means for playing the generated voice message in a voice that is easy for the user to hear, means for receiving the user's response by voice input and feeding it back to the analysis result, and means for analyzing the user's emotional data and adjusting the content and tone of the voice message. This enables task notification at an appropriate time while taking into account the user's emotional state and hearing impairment.

[0904] The "user's daily life pattern" refers to the chronological trends and regularities of the user's daily activities.

[0905] "Task prompt" refers to an audio notification generated to prompt a user to perform a specific action (e.g., take medicine, eat, exercise, etc.).

[0906] "Easy-to-listen voice" refers to voice that has been appropriately adjusted in tone, volume, speed, word choice, etc. to make it easy for the user to understand.

[0907] "Receiving by voice input" refers to receiving the user's voice using a device such as a microphone and transmitting the content to the system.

[0908] "Feedback on the analysis results" refers to the process of reviewing the output of the generative AI model based on the received voice data and emotional data to improve the accuracy and adaptability of the system.

[0909] "Emotional data" refers to information about a user's emotional state, identified based on the user's tone of voice, facial expressions, etc.

[0910] "Adjusting the content and tone of the voice message" refers to a process of changing the information and voice quality of the message to be notified in accordance with the emotional state of the user.

[0911] This invention is a system that analyzes the user's daily life patterns, prompts them with voice messages to perform necessary tasks, and recognizes the user's emotions using an emotion engine to support the daily lives of elderly people. This system consists of three main components: a server, a terminal, and a user. Each component works in cooperation to realize life support.

[0912] Server Roles

[0913] The server first receives the user's daily behavioral data from the device, including wake-up time, meal times, medication times, etc. The received data is then stored in a database (e.g., MySQL or PostgreSQL).

[0914] Next, the server analyzes the user's lifestyle patterns using a generative AI model (such as OpenAI's GPT-3) based on the behavioral data stored in the database. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[0915] Based on the analysis results, the server creates a task schedule to notify the user. According to this schedule, it generates a voice message to prompt the user to perform the task. The generated voice message is sent to the user at the appropriate time using a speech synthesis engine (e.g., Google Text-to-Speech).

[0916] Additionally, an emotion engine (for example, IBM Watson's emotion analysis or Microsoft Azure's Emotion API) analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the analysis, the emotion engine provides feedback to the generative AI model and adjusts the content and tone of the voice message. This enables notifications to take the user's emotional state into consideration.

[0917] Specifically, if the server learns from past data that the user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the o'clock approaches. The emotion engine will analyze the tone of the user's voice, and if the user is feeling stressed, it will notify them in a gentler tone.

[0918] Device Role

[0919] The device receives the voice message sent from the server. After receiving the message, the device plays it back to the user in an easy-to-listen voice. The device also takes into consideration the use of speakers and hearing aids. For example, adjustments have been made so that notifications can be heard clearly even if the user has age-related hearing loss.

[0920] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent to the server.

[0921] Specifically, after the user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user verbally tells the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and can determine the user's emotional state.

[0922] User Roles

[0923] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[0924] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[0925] Prompt Sentence Examples

[0926] "Analyze the user's daily patterns and generate the next notification content. For example, if you have learned from past data that the user takes their medicine at 10 o'clock, when it gets close to 10 o'clock, notify them with, 'It's 10 o'clock. It's time to take your medicine,' and adjust the tone according to the user's emotional state."

[0927] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[0928] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0929] Server-side processing

[0930] Step 1:

[0931] The server receives the user's daily activity data from the device, including wake-up time, meal times, and medication times. The input is the user's specific activity time data. The server stores this in a database. The output is an updated database.

[0932] Step 2:

[0933] The server retrieves the behavioral data stored in the database and analyzes the lifestyle patterns using the generative AI model. The input is past behavioral data. The generative AI analyzes this and extracts the user's daily life patterns. The output is the daily life patterns as the analysis results.

[0934] Step 3:

[0935] The server generates a task schedule based on the analyzed lifestyle patterns. The input is the daily life patterns as the analysis results. Based on this, the server predicts the next task (e.g., time to take medicine) and incorporates it into the schedule. The output is a task schedule.

[0936] Step 4:

[0937] The server generates a voice message using a generative AI model based on the task schedule. The input is the task schedule. The generative AI uses this to generate a voice notification such as "It's 10 o'clock. Time to take your medicine." The output is the generated voice message.

[0938] Step 5:

[0939] The server sends the generated voice message to the terminal. The input is the generated voice message. After sending, the server allows the user to perform a specified task. The output is the voice message sent to the terminal.

[0940] Step 6:

[0941] The server analyzes the user's voice input and facial expression data sent from the terminal and determines the user's emotional state. The input is the user's voice input and facial expression data. The emotion analysis engine analyzes this and identifies the user's emotional state. The output is the emotion determination result.

[0942] Step 7:

[0943] The server provides feedback to the generative AI model based on the emotion analysis results, which is reflected in the next notification. The input is the emotion analysis results. The generative AI adjusts the content and tone of the notification. The output is the adjusted content and tone of the next notification.

[0944] Terminal side processing

[0945] Step 1:

[0946] The terminal receives a voice message sent from the server. The input is the voice message sent from the server. As soon as it receives the message, it prepares to notify the user. The output is the received voice message.

[0947] Step 2:

[0948] The terminal plays the received voice message to the user. The input is the voice message received from the server. The terminal plays it through a speaker or hearing aid to clearly convey it to the user. The output is a voice message that the user can understand.

[0949] Step 3:

[0950] The device receives the user's response as voice input. The input is the user's voice response. This is detected and recorded through a microphone. The output is the recorded voice data.

[0951] Step 4:

[0952] The device converts the recorded voice data into text and sends it to the server. The input is voice data. It converts this into text data using speech recognition software. The output is the voice input as text data.

[0953] User-side processing

[0954] Step 1:

[0955] The user receives a voice notification from the terminal. The input is a voice message played from the terminal. The user listens to this and understands the next action. The output is the understood task content.

[0956] Step 2:

[0957] The user performs the task that is notified. The input is the content of the voice notification (e.g., "It's 10 o'clock. Time to take your medicine"). The user performs the task according to this instruction. The output is the performed task (e.g., take your medicine).

[0958] Step 3:

[0959] After completing a task, the user reports to the terminal. The input is a voice report about the completed task (e.g., "Yes, I took my medicine"). The user speaks this into a microphone. The output is a voice report.

[0960] By having each processing step work in conjunction with one another in this way, the system can support the daily lives of elderly people and improve their quality of life.

[0961] (Application example 2)

[0962] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0963] Existing systems and devices for supporting the daily lives of elderly people have difficulty effectively analyzing users' daily life patterns and notifying them of necessary tasks at the appropriate time. They also lack flexible responses to the user's emotional state, and therefore do not provide sufficient support for elderly people to spend their time safely and comfortably in self-driving vehicles.

[0964] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's daily life patterns using a generative artificial intelligence and generating a voice message to encourage task execution based on the extracted daily life patterns, means for playing the generated voice message, means for receiving the user's responses via voice input and feeding them back to the generative artificial intelligence, means for collecting in-vehicle activity data and suggesting in-vehicle tasks based on the user's behavioral patterns, and means for analyzing the user's emotions and adjusting the tone and content of notifications based on the user's emotional state. This makes it possible to support elderly people's daily lives in self-driving vehicles, reduce difficulties caused by forgetfulness, and provide flexible support that takes emotions into consideration.

[0965] "Generative AI" is an intelligent system that analyzes the user's daily life patterns and automatically generates notifications and instructions to perform necessary tasks.

[0966] "Daily life patterns" refer to the actions and habits that a user repeats every day.

[0967] "Task execution" refers to the actions or activities that a user must perform at a specific time and in a specific situation.

[0968] A "voice message" is a message that conveys information or instructions to the user by voice.

[0969] A "speaker" is an output device for reproducing a voice message as sound.

[0970] "User response" refers to the response or action taken by the user after receiving the voice message.

[0971] "Feedback" is the process of gathering user responses and adjusting the system's behavior based on that information.

[0972] "In-vehicle activity data" refers to data relating to the user's behavior and situation within an autonomous vehicle.

[0973] A "behavior pattern" is a combination of actions that a user repeatedly performs under specific circumstances.

[0974] "Task Suggestion" refers to recommending or informing the user about a specific action or activity to perform.

[0975] "Emotion analysis" is the process of recognizing and analyzing emotions from the user's voice, facial expressions, etc.

[0976] "Tone of notification" refers to the sound quality and speaking characteristics of a voice message, and is an element that is adjusted according to the user's emotional state.

[0977] An embodiment of the present invention will be described.

[0978] server:

[0979] The server first receives daily behavioral data from the vehicle and the user's device, including boarding and alighting times and in-vehicle activities (e.g., reading, eating, resting, etc.). This data is then stored in a database.

[0980] The server then uses generative AI to analyze the user's lifestyle patterns based on the stored behavioral data. For example, the server uses the KMeans clustering algorithm to classify daily life patterns and determine the time of day when certain activities occur.

[0981] Based on the analysis results, the server creates a task schedule to notify the user, and generates a voice message prompting the user to perform the task according to the schedule. The generated voice message is sent to the user's device at the appropriate time.

[0982] Furthermore, the emotion analysis engine analyzes the user's voice input and facial expression data to determine their emotions. Based on the results of this analysis, the generation AI adjusts the content and tone of the voice message to deliver notifications that take their emotions into consideration.

[0983] Device:

[0984] The terminal receives the voice message sent from the server and then plays the voice message to the user through the speaker, taking into consideration that the voice message is easy to hear even if the user is wearing a hearing aid.

[0985] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent back to the server. An emotion analysis engine analyzes the voice data to identify the user's emotional state.

[0986] User:

[0987] The user receives a voice notification from the device, for example, hearing a message such as "It's 10 o'clock. Time to take your medicine." In response, the user performs a task (e.g., taking medicine) and reports the completion to the device by voice. The device receives voice feedback such as "Yes, I took my medicine" and sends it to the server. This information is used for future notifications.

[0988] The server can store this feedback in a database and reflect it in the next task notification or life pattern analysis. For example, if a user always responds with a specific tone of voice, the server can adjust the next notification based on that user's emotional state.

[0989] As a concrete example, we will show a scenario in which an elderly person is riding in an autonomous vehicle. The server receives and analyzes the riding data, generates a notification to take medicine at 10 o'clock, and notifies the user in a gentle tone if the user is feeling stressed. An example of this prompt sentence is, "To help the elderly person relax in the autonomous vehicle, please generate a voice message that gently notifies them that it is time to take their medicine at 10 o'clock. Also, if the user feels stressed, please notify them in a tone that requires special attention."

[0990] This will enable elderly people to live comfortably in self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that takes their emotions into consideration.

[0991] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0992] Step 1:

[0993] The device collects in-vehicle activity data. Using sensors and cameras, it records the time of boarding and exiting the vehicle, as well as activities in the vehicle (e.g., reading, eating, resting, etc.). The collected data is temporarily stored in the device.

[0994] Step 2:

[0995] The device sends the collected data to the server. The data is transferred to the server via a secure channel and stored in a database on the server. The input data is boarding and exiting times and in-car behavior data, and the output is stored in a database on the server.

[0996] Step 3:

[0997] The server uses generative AI to analyze the user's daily life patterns based on the stored data. Algorithms used here include KMeans clustering and other methods to classify daily life patterns. The input is the behavioral data in the database, and the output is the classified daily life patterns.

[0998] Step 4:

[0999] The server creates a task schedule based on the analysis results. It predicts tasks that the user should perform at specific times (e.g., taking medicine, staying hydrated), and generates a schedule to notify users at those times. The input is the analysis results, and the output is the task schedule.

[1000] Step 5:

[1001] The server uses the generative AI model to generate a voice message prompting the user to perform the task. The server also sets the prompt sentence when generating the message and takes into account the results of sentiment analysis. The input is the task schedule and emotion data, and the output is the voice message.

[1002] Step 6:

[1003] The server sends the generated voice message to the terminal. The terminal plays the received message to the user through the speaker and adjusts the voice settings to make it easy to hear even if the user is using a hearing aid. The input is the voice message and the output is a notification to the user.

[1004] Step 7:

[1005] The user receives a voice message and performs the instructed task. For example, if the user receives a notification that "It's 10 o'clock. It's time to take your medicine," the user will actually take the action of taking their medicine. The input is the voice message, and the output is the execution of the task.

[1006] Step 8:

[1007] The user reports task completion to the terminal by voice. The terminal converts this voice into text data and sends it to the server as feedback. The input is the user's voice and the output is text data.

[1008] Step 9:

[1009] The server receives the feedback and stores it in a database. An emotion analysis engine analyzes the user's voice data and identifies their emotional state. This information is used for future task notifications and lifestyle pattern analysis. The input is text data and emotion data, and the output is the database and analysis results.

[1010] This series of processing steps makes it possible to assist elderly people in their daily lives inside self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that is sensitive to their emotions.

[1011] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1012] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1013] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1014] [Fourth embodiment]

[1015] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1016] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1018] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1019] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1020] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1022] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1023] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1024] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1026] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1027] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1028] The present invention is a system for supporting the daily lives of elderly people. The specific program processing of this system will be explained below in natural language.

[1029] (Server-side processing)

[1030] The server first receives the user's daily behavioral data from the device. This data includes wake-up times, meal times, and medication times. The received data is stored in a database. The server then uses generative artificial intelligence to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[1031] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[1032] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, as 10 o'clock approaches it will generate a voice message such as "It's 10 o'clock. It's time to take your medicine" and send this to the terminal.

[1033] (Processing on the terminal side)

[1034] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[1035] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server.

[1036] (User side processing)

[1037] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. It's time to take your medicine," the user performs the task (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions.

[1038] Specifically, the user receives the notification, takes the medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database and uses it for future notifications.

[1039] Through these processes, the present invention can support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness, and improve their quality of life.

[1040] The processing flow will be explained below.

[1041] Server-side processing

[1042] Step 1:

[1043] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[1044] Step 2:

[1045] The server stores the received data in a database.

[1046] Step 3:

[1047] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[1048] Step 4:

[1049] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[1050] Step 5:

[1051] Based on the learning results, the server creates a task schedule to notify the user.

[1052] Step 6:

[1053] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[1054] Step 7:

[1055] The generated voice message is sent to the user's terminal at an appropriate time.

[1056] Terminal side processing

[1057] Step 8:

[1058] The terminal receives the voice message sent from the server.

[1059] Step 9:

[1060] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[1061] Step 10:

[1062] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[1063] Step 11:

[1064] The received voice input is converted into text data and sent to the server.

[1065] User-side processing

[1066] Step 12:

[1067] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[1068] Step 13:

[1069] The user who receives the notification performs the instructed task (e.g., taking medicine).

[1070] Step 14:

[1071] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[1072] Processing Feedback

[1073] Step 15:

[1074] The server receives the user's reaction data transmitted from the terminal.

[1075] Step 16:

[1076] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[1077] Through these steps, the system of the present invention can support the elderly in their daily lives, reduce the difficulties of life caused by forgetfulness, and improve their quality of life.

[1078] Example 1

[1079] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1080] In modern society, elderly people often forget to take their medicine or eat at the right time, which can negatively impact their health and quality of life. Age-related hearing loss also often makes voice notifications ineffective. To address these issues, a system is needed that analyzes the daily life patterns of elderly people and notifies them of tasks at the appropriate time. However, current systems lack notification methods that take into account the hearing impairments of elderly people and a means to accurately provide feedback on user responses.

[1081] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1082] In this invention, the server includes means for analyzing the user's daily life patterns and generating voice messages to encourage task execution based on the extracted daily life patterns, means for playing the generated voice messages in a voice that is easy for the user to hear via an audio device, means for receiving the user's responses via voice input, converting them into text data through voice recognition, and feeding the text data back to the generating AI, and means for storing the user's task execution status in a database and using it for future notifications. This makes it possible to efficiently support the daily lives of the elderly, reduce the difficulties of life caused by forgetfulness and hearing impairment, and improve their quality of life.

[1083] "Generative AI" is a technology that uses machine learning algorithms and neural networks to analyze a user's daily life patterns and generate voice messages that encourage appropriate task execution.

[1084] "Daily life patterns" refer to the regularities and habits of daily life extracted from the user's daily behavioral data, including wake-up time, meal times, and medication times.

[1085] An "acoustic device" is a device that reproduces audio messages in a way that is easy for the user to hear. Examples include speakers and hearing aids.

[1086] "Voice input" refers to input made by voice from a user. This system is used to receive the user's voice response and convert it into digital data.

[1087] "Speech recognition" is a technology that converts received voice input into digital data (text data), making the content of the voice usable as specific behavioral data.

[1088] A "database" is a digital data storage system for storing and managing received behavioral data and task execution history. Examples include MySQL and PostgreSQL.

[1089] "Feedback" is the process of communicating the tasks performed and responses of the user to the system, allowing the system to further refine its learning of the user's behavioral patterns.

[1090] A "task schedule" is a timetable that prompts users to perform tasks at appropriate times based on daily life patterns analyzed by the generative artificial intelligence.

[1091] The present invention is a system for supporting the daily lives of elderly people, and is specifically implemented as follows.

[1092] Server Features

[1093] The server first receives the user's daily life data from the device. This data includes wake-up time, meal times, and medication times. Communication protocols used include HTTP and MQTT. The received data is stored in a database such as MySQL or PostgreSQL. Next, based on the data stored in the database, the server analyzes the user's daily life patterns using a generative AI model (e.g., TensorFlow or PyTorch). Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks. Based on these predictions, the server creates a task schedule to notify the user. Next, a voice message (e.g., "It's 10 o'clock. It's time to take your medicine") is generated according to the task schedule and sent to the device.

[1094] Device Features

[1095] The device receives the voice message sent from the server. After receiving it, the device plays the voice message to the user using an audio device (speaker or hearing aid). At this time, the device plays the voice message in an easy-to-listen voice, taking into consideration whether the user is using a speaker or hearing aid. After the user receives the notification, the device receives the user's response as voice input. The received voice input is converted into text data using voice recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The converted text data is sent to the server, which stores the data in a database and uses it for future notifications.

[1096] User Roles

[1097] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine") and performs the task. After completing the task, the user notifies the device by voice (e.g., "Yes, I have taken my medicine"). This information is received by the device as voice input, converted into text data, and sent to the server.

[1098] Specific examples

[1099] For example, if a user has a pattern of taking medicine at 10 o'clock every day, the server analyzes this pattern and notifies the user at 9:59, "It's 10 o'clock. Time to take your medicine." Upon receiving this notification, the user takes their medicine and reports to their device, "Yes, I have taken my medicine." This report is converted into text data and sent to the server, and is used for future notifications.

[1100] Prompt Sentence Examples

[1101] "Design an algorithm to analyze users' daily patterns and notify them of tasks at the appropriate time."

[1102] "Please explain in detail how the medication alert system for seniors works."

[1103] Through these processes, the present invention can efficiently support the daily lives of elderly people, reduce the difficulties they face due to forgetfulness and hearing impairment, and improve their quality of life.

[1104] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1105] Step 1: Receiving and storing data

[1106] The server receives the user's daily life data (e.g., wake-up time, meal times, medication times) from the device. This data is sent to the server via the communication protocol used (e.g., HTTP, MQTT). The received data is stored in a database such as MySQL or PostgreSQL. The input is the behavioral data received from the device, and the output is the data stored in the database.

[1107] Step 2: Analyze the data

[1108] The server analyzes lifestyle patterns using a generative AI model (e.g., TensorFlow, PyTorch) based on the daily life data stored in the database. In this analysis process, the stored behavioral data is used as input, and the analyzed lifestyle patterns are obtained as output. Specifically, the server extracts the user's behavioral patterns (e.g., taking medicine at 10 o'clock every day) based on past data.

[1109] Step 3: Generate a task schedule

[1110] The server learns the user's lifestyle patterns based on the analysis results and predicts the timing of required tasks. Based on this, the server generates a task schedule to notify the user. The analyzed lifestyle patterns are used as input, and the generated task schedule is obtained as output. Specifically, the timing of a voice message such as "It's 10 o'clock. Time to take your medicine" is set.

[1111] Step 4: Generate a voice message

[1112] The server generates a voice message according to the generated task schedule. This uses speech synthesis technology (e.g., Google Text-to-Speech, Microsoft Azure Speech). The task schedule is used as input, and the voice message is generated as output. For example, a voice message such as "It's 10 o'clock. Time to take your medicine" is generated.

[1113] Step 5: Send a voice message

[1114] The server sends the generated voice message to the terminal. The generated voice message is the input, and the message is sent to the terminal as the output. The communication protocol used is HTTP, MQTT, etc.

[1115] Step 6: Receive and play voice messages

[1116] The terminal receives the voice message sent from the server. After receiving it, the terminal uses an acoustic device (e.g., speaker, hearing aid) to play the voice message in a way that is easy for the user to hear. The input is the voice message from the server, and the output is the played voice message. Specifically, a message such as "It's 10 o'clock. Time to take your medicine" is played to the user.

[1117] Step 7: Receiving and sending user responses

[1118] The user receives a voice message (e.g., "It's 10 o'clock. Time to take your medicine") and performs the notified task. Then, they say to the device, "Yes, I have taken my medicine." The device receives the user's voice input and converts it into text data using speech recognition technology (e.g., Google Speech-to-Text, Microsoft Azure Speech). The user's voice response is the input, and text data is obtained as the output. The converted text data is then sent to the server.

[1119] Step 8: Store and use your feedback

[1120] The server receives the user's reaction data sent from the device. After receiving it, it stores it in a database and uses it for future notifications. The input is the user's reaction data (text), and the output is saved feedback data. Specifically, it is saved as data to be used for the next similar task notification.

[1121] (Application example 1)

[1122] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1123] In factories, robots and workers are required to perform tasks efficiently and in a timely manner, but conventional systems have problems with inefficient schedule management and feedback. In particular, in an environment where multiple tasks occur simultaneously, it is difficult to determine which task should be prioritized. It is also necessary to understand the task execution status and appropriately instruct the next task. This is necessary to improve work efficiency in factories and reduce errors.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1125] In this invention, the server includes means for analyzing the user's daily life patterns using a generative AI model and generating a voice or text message to encourage task execution based on the extracted daily life patterns, means for notifying the user of the generated voice or text message via a voice output device or a display device, means for receiving the user's response by voice or text input and feeding it back to the generative AI model, and means for saving the results of the user's actions in a database and reflecting them in future task schedules. This enables robots and workers to efficiently execute tasks, provide appropriate feedback, and manage future schedules.

[1126] A "generative AI model" is a model that uses artificial intelligence to analyze data and make predictions or generate results based on the learning results.

[1127] "Daily life patterns" refers to the regularities and trends obtained by analyzing data on the user's daily actions and habits.

[1128] A "task" refers to an operation or action performed to achieve a specific purpose.

[1129] "Voice message" refers to audible notifications or instructions sent from a machine or system to a user.

[1130] "Text message" refers to notifications or instructions in the form of text.

[1131] "Audio output device" means a device for outputting an audio message in an audible form, such as a speaker.

[1132] "Display device" refers to a device for visually conveying information, including, for example, a display.

[1133] "Feedback" refers to returning user responses and results to the system.

[1134] A "database" refers to a system for systematically storing and managing data.

[1135] "Schedule management" refers to planning the timing of task execution and monitoring and adjusting its progress.

[1136] A "robot" refers to a mechanical device that operates autonomously based on programmed instructions.

[1137] 1. System Program Overview

[1138] This invention is a robot work scheduling system that aims to improve work efficiency and reduce errors in factories. The system consists of three main components: a server, a robot, and a user.

[1139] 2. Server-side program processing

[1140] The server uses a generative AI model to analyze daily task patterns. The analysis is performed using a cloud platform such as AWS or Google Cloud, and the data is stored in a relational database such as MySQL. Based on the obtained daily life patterns, a voice or text message is generated to prompt the user or robot to perform the task, providing a means to notify the user or robot at the appropriate time.

[1141] 3. Program processing on the robot side

[1142] The robot receives notifications from the server through an audio output device or a display device and executes the specified task. The specific robot hardware used is an industrial robot (e.g., ABB's YuMi or Fanuc's robot arm). When the robot completes a task, it feeds back the work results to the server, which are reflected in the next task schedule.

[1143] 4. Feedback and Data Collection

[1144] The results of tasks completed by users or robots are fed back to the server, which then stores the data in a database. This allows for more precise adjustments to the task schedule for future tasks.

[1145] 5. Specific Examples

[1146] For example, if a robot is assigned the task of assembling part A at 10 o'clock, the server will notify it 10 minutes before the task execution, saying, "Please start assembling part A at 10 o'clock." The robot will then start the work in accordance with the notification, and upon completion, it will report to the server, "Work completed, part A has been assembled." This allows for smooth task execution and appropriate feedback.

[1147] 6. Examples of prompts

[1148] prompt:

[1149] "Write a program that generates a task schedule for a factory robot to perform its set tasks on time each day. In particular, include the following information:

[1150] Work start time

[1151] Work content

[1152] Work completion notification

[1153] Additionally, when you complete a task, you can send feedback to the server and use that data to plan future tasks.

[1154] The embodiment of the present invention is configured as described above and provides a system for improving the work efficiency of a robot and reducing errors.

[1155] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1156] Step 1:

[1157] The server receives behavioral data from robots and workers via terminals. This data includes the start time of work, the completion time of work, and the work content. The robot's sensor data and work logs are sent to the server as input data. The server stores this data in a database. The output is the behavioral data stored in the database.

[1158] Step 2:

[1159] The server uses the generative AI model to analyze the behavioral data received in step 1. The stored behavioral data is passed to the generative AI model as input data. The generative AI model learns patterns from past data and makes task predictions. The output is a predicted task schedule based on the analysis.

[1160] Step 3:

[1161] The server generates voice or text messages to notify robots and workers based on the predicted task schedule. The predicted task schedule is passed to the message generation system as input data. The generated messages are notified through a voice output device or a display device. The output is a message that is notified at the appropriate time.

[1162] Step 4:

[1163] The user (robot or worker) executes the task based on the notified message. Specifically, the robot assembles parts, and the worker checks the parts. The notified message is passed to the user as input data. The output is the result of the task execution.

[1164] Step 5:

[1165] When a robot or worker completes a task, the result is sent to the server as feedback. Specifically, the robot generates a text message such as "Task completed, part A assembled" and sends it to the server. The task completion result is sent to the server as input data. The output is feedback data stored in the database.

[1166] Step 6:

[1167] The server stores the feedback data in the database again and reflects it in the next and subsequent task schedules. Specifically, it analyzes the feedback data and adjusts the next task schedule more precisely. The feedback data is passed to the server as input data. The output is the adjusted task schedule for the next and subsequent tasks.

[1168] As described above, this system can improve work efficiency and accuracy within the factory by repeating the process of performing appropriate data processing and calculations based on input data at each step and generating output data.

[1169] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1170] This invention is a system for supporting the daily lives of elderly people by analyzing the user's daily life patterns, prompting them with voice messages to perform necessary tasks, and recognizing the user's emotions using an emotion engine. Specific program processing of this system is explained below in natural language.

[1171] (Server-side processing)

[1172] The server first receives the user's daily behavioral data from the device. This data includes wake-up time, meal times, and medication times. The received data is stored in a database. The server then uses generative AI to analyze the user's lifestyle patterns based on the stored data. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[1173] Next, the server creates a task schedule based on the learning results to notify the user. According to this schedule, it generates a voice message prompting the user to perform the task. The generated voice message is sent to the user at the appropriate time.

[1174] The emotion engine also analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the determination, the emotion engine provides feedback to the generation AI and adjusts the content and tone of the voice message. As a result, notifications can be delivered that take the user's emotional state into consideration.

[1175] For example, if the server learns from past data that a user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the time approaches. The emotion engine will analyze the tone of the user's voice and, if the user is feeling stressed, it can notify them in a gentler tone.

[1176] (Processing on the terminal side)

[1177] The device receives the voice message sent from the server. After receiving the voice message, the device plays it back to the user in a voice that is easy to hear, taking into account whether the user is using a speaker or a hearing aid. After the user receives the voice message, the device receives the user's response as voice input. The received voice input is converted into text data and sent to the server.

[1178] For example, after a user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user can say to the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and determines the user's emotional state.

[1179] (User side processing)

[1180] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[1181] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[1182] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[1183] The processing flow will be explained below.

[1184] Server-side processing

[1185] Step 1:

[1186] The server receives the user's daily behavior data from the device, including wake-up time, meal times, and medication times.

[1187] Step 2:

[1188] The server stores the received data in a database.

[1189] Step 3:

[1190] The server uses generative AI to analyze the user's lifestyle patterns based on the saved behavioral data.

[1191] Step 4:

[1192] Based on the analysis results, the server learns the user's daily life patterns and predicts the timing of necessary tasks.

[1193] Step 5:

[1194] Based on the learning results, the server creates a task schedule to notify the user.

[1195] Step 6:

[1196] The server generates a voice message to prompt the user to perform the task (e.g., "It's 10 o'clock. Time to take your medicine.").

[1197] Step 7:

[1198] The generated voice message is sent to the user's terminal at an appropriate time.

[1199] Step 8:

[1200] The server uses an emotion engine to analyze the user's voice data received from the terminal and determine the user's emotion.

[1201] Step 9:

[1202] Based on the results of the assessment, the emotion engine adjusts the tone and content of the voice message and reflects this in the next notification.

[1203] Terminal side processing

[1204] Step 10:

[1205] The terminal receives the voice message sent from the server.

[1206] Step 11:

[1207] The device then plays the received message in an easy-to-listen voice to the user, either through a speaker or a hearing aid.

[1208] Step 12:

[1209] The device receives the user's response to the voice message as voice input (e.g., "Yes, I took my medicine").

[1210] Step 13:

[1211] The received voice input is converted into text data and sent to the server.

[1212] User-side processing

[1213] Step 14:

[1214] The user receives a voice notification from the device (e.g., "It's 10 o'clock. Time to take your medicine").

[1215] Step 15:

[1216] The user who receives the notification performs the instructed task (e.g., taking medicine).

[1217] Step 16:

[1218] After completing the task, the user verbally informs the device that the task is complete (e.g., "Yes, I have taken my medicine").

[1219] Processing Feedback

[1220] Step 17:

[1221] The server receives the user's reaction data transmitted from the terminal.

[1222] Step 18:

[1223] The server stores the received response data in a database and uses it to analyze future lifestyle patterns and generate notification schedules.

[1224] Step 19:

[1225] The emotion engine analyzes the received response data and determines the user's emotional state.

[1226] Step 20:

[1227] The emotion engine feeds the results of the discrimination back to the generation AI, which then reflects them in subsequent notification messages.

[1228] Through these steps, the system of the present invention can support the daily lives of elderly people, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions through its emotion engine.

[1229] Example 2

[1230] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1231] To improve the quality of life for the elderly, timely task notifications based on the user's daily life patterns are required. However, current systems do not take the user's emotional state into consideration when providing notifications, making it difficult to prompt them to perform tasks at the appropriate time. Furthermore, for elderly people with age-related hearing loss, voice notifications tend to be unclear, resulting in the problem of not being able to understand the notification content. A system that can solve these issues is needed.

[1232] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1233] In this invention, the server includes means for analyzing the user's daily life patterns and generating a voice message for encouraging the user to perform a task based on the extracted daily life patterns, means for playing the generated voice message in a voice that is easy for the user to hear, means for receiving the user's response by voice input and feeding it back to the analysis result, and means for analyzing the user's emotional data and adjusting the content and tone of the voice message. This enables task notification at an appropriate time while taking into account the user's emotional state and hearing impairment.

[1234] The "user's daily life pattern" refers to the chronological trends and regularities of the user's daily activities.

[1235] "Task prompt" refers to an audio notification generated to prompt a user to perform a specific action (e.g., take medicine, eat, exercise, etc.).

[1236] "Easy-to-listen voice" refers to voice that has been appropriately adjusted in tone, volume, speed, word choice, etc. to make it easy for the user to understand.

[1237] "Receiving by voice input" refers to receiving the user's voice using a device such as a microphone and transmitting the content to the system.

[1238] "Feedback on the analysis results" refers to the process of reviewing the output of the generative AI model based on the received voice data and emotional data to improve the accuracy and adaptability of the system.

[1239] "Emotional data" refers to information about a user's emotional state, identified based on the user's tone of voice, facial expressions, etc.

[1240] "Adjusting the content and tone of the voice message" refers to a process of changing the information and voice quality of the message to be notified in accordance with the emotional state of the user.

[1241] This invention is a system that analyzes the user's daily life patterns, prompts them with voice messages to perform necessary tasks, and recognizes the user's emotions using an emotion engine to support the daily lives of elderly people. This system consists of three main components: a server, a terminal, and a user. Each component works in cooperation to realize life support.

[1242] Server Roles

[1243] The server first receives the user's daily behavioral data from the device, including wake-up time, meal times, medication times, etc. The received data is then stored in a database (e.g., MySQL or PostgreSQL).

[1244] Next, the server analyzes the user's lifestyle patterns using a generative AI model (such as OpenAI's GPT-3) based on the behavioral data stored in the database. Based on the analysis results, the server learns the user's lifestyle patterns and predicts the timing of necessary tasks.

[1245] Based on the analysis results, the server creates a task schedule to notify the user. According to this schedule, it generates a voice message to prompt the user to perform the task. The generated voice message is sent to the user at the appropriate time using a speech synthesis engine (e.g., Google Text-to-Speech).

[1246] Additionally, an emotion engine (for example, IBM Watson's emotion analysis or Microsoft Azure's Emotion API) analyzes the user's voice input and facial expression data to determine the user's emotions. Based on the results of the analysis, the emotion engine provides feedback to the generative AI model and adjusts the content and tone of the voice message. This enables notifications to take the user's emotional state into consideration.

[1247] Specifically, if the server learns from past data that the user takes their medicine at 10 o'clock every day, it will generate a voice message such as "It's 10 o'clock. Time to take your medicine" and send it to the device as the o'clock approaches. The emotion engine will analyze the tone of the user's voice, and if the user is feeling stressed, it will notify them in a gentler tone.

[1248] Device Role

[1249] The device receives the voice message sent from the server. After receiving the message, the device plays it back to the user in an easy-to-listen voice. The device also takes into consideration the use of speakers and hearing aids. For example, adjustments have been made so that notifications can be heard clearly even if the user has age-related hearing loss.

[1250] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent to the server.

[1251] Specifically, after the user receives a notification from the device saying, "It's 10 o'clock. It's time to take your medicine," the user verbally tells the device, "Yes, I have taken my medicine." This voice input is received by the device, converted into text data, and sent to the server. The emotion engine analyzes this voice data and can determine the user's emotional state.

[1252] User Roles

[1253] The user receives a voice notification from the device. For example, when the user receives a notification saying, "It's 10 o'clock. Time to take your medicine," the user performs the task indicated by the notification (e.g., take your medicine). After completing the task, the user notifies the device by voice. This provides feedback to the system about the user's actions. At the same time, the user's emotional data is collected and reflected in the next notification.

[1254] Specifically, the user receives the notification, takes their medicine, and reports this to the device by saying, "Yes, I took my medicine." This information is sent from the device to the server, which stores it in a database for use in future notifications. If the emotion engine analyzes the user's voice response and determines that the user appears relieved, this will be reflected in the content of the next notification.

[1255] Prompt Sentence Examples

[1256] "Analyze the user's daily patterns and generate the next notification content. For example, if you have learned from past data that the user takes their medicine at 10 o'clock, when it gets close to 10 o'clock, notify them with, 'It's 10 o'clock. It's time to take your medicine,' and adjust the tone according to the user's emotional state."

[1257] Through these processes, the present invention can support the daily lives of the elderly, reduce the difficulties caused by forgetfulness, and improve their quality of life by providing flexible support that takes into consideration the user's emotions using an emotion engine.

[1258] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1259] Server-side processing

[1260] Step 1:

[1261] The server receives the user's daily activity data from the device, including wake-up time, meal times, and medication times. The input is the user's specific activity time data. The server stores this in a database. The output is an updated database.

[1262] Step 2:

[1263] The server retrieves the behavioral data stored in the database and analyzes the lifestyle patterns using the generative AI model. The input is past behavioral data. The generative AI analyzes this and extracts the user's daily life patterns. The output is the daily life patterns as the analysis results.

[1264] Step 3:

[1265] The server generates a task schedule based on the analyzed lifestyle patterns. The input is the daily life patterns as the analysis results. Based on this, the server predicts the next task (e.g., time to take medicine) and incorporates it into the schedule. The output is a task schedule.

[1266] Step 4:

[1267] The server generates a voice message using a generative AI model based on the task schedule. The input is the task schedule. The generative AI uses this to generate a voice notification such as "It's 10 o'clock. Time to take your medicine." The output is the generated voice message.

[1268] Step 5:

[1269] The server sends the generated voice message to the terminal. The input is the generated voice message. After sending, the server allows the user to perform a specified task. The output is the voice message sent to the terminal.

[1270] Step 6:

[1271] The server analyzes the user's voice input and facial expression data sent from the terminal and determines the user's emotional state. The input is the user's voice input and facial expression data. The emotion analysis engine analyzes this and identifies the user's emotional state. The output is the emotion determination result.

[1272] Step 7:

[1273] The server provides feedback to the generative AI model based on the emotion analysis results, which is reflected in the next notification. The input is the emotion analysis results. The generative AI adjusts the content and tone of the notification. The output is the adjusted content and tone of the next notification.

[1274] Terminal side processing

[1275] Step 1:

[1276] The terminal receives a voice message sent from the server. The input is the voice message sent from the server. As soon as it receives the message, it prepares to notify the user. The output is the received voice message.

[1277] Step 2:

[1278] The terminal plays the received voice message to the user. The input is the voice message received from the server. The terminal plays it through a speaker or hearing aid to clearly convey it to the user. The output is a voice message that the user can understand.

[1279] Step 3:

[1280] The device receives the user's response as voice input. The input is the user's voice response. This is detected and recorded through a microphone. The output is the recorded voice data.

[1281] Step 4:

[1282] The device converts the recorded voice data into text and sends it to the server. The input is voice data. It converts this into text data using speech recognition software. The output is the voice input as text data.

[1283] User-side processing

[1284] Step 1:

[1285] The user receives a voice notification from the terminal. The input is a voice message played from the terminal. The user listens to this and understands the next action. The output is the understood task content.

[1286] Step 2:

[1287] The user performs the task that is notified. The input is the content of the voice notification (e.g., "It's 10 o'clock. Time to take your medicine"). The user performs the task according to this instruction. The output is the performed task (e.g., take your medicine).

[1288] Step 3:

[1289] After completing a task, the user reports to the terminal. The input is a voice report about the completed task (e.g., "Yes, I took my medicine"). The user speaks this into a microphone. The output is a voice report.

[1290] By having each processing step work in conjunction with one another in this way, the system can support the daily lives of elderly people and improve their quality of life.

[1291] (Application example 2)

[1292] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1293] Existing systems and devices for supporting the daily lives of elderly people have difficulty effectively analyzing users' daily life patterns and notifying them of necessary tasks at the appropriate time. They also lack flexible responses to the user's emotional state, and therefore do not provide sufficient support for elderly people to spend their time safely and comfortably in self-driving vehicles.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's daily life patterns using a generative artificial intelligence and generating a voice message to encourage task execution based on the extracted daily life patterns, means for playing the generated voice message, means for receiving the user's responses via voice input and feeding them back to the generative artificial intelligence, means for collecting in-vehicle activity data and suggesting in-vehicle tasks based on the user's behavioral patterns, and means for analyzing the user's emotions and adjusting the tone and content of notifications based on the user's emotional state. This makes it possible to support elderly people's daily lives in self-driving vehicles, reduce difficulties caused by forgetfulness, and provide flexible support that takes emotions into consideration.

[1295] "Generative AI" is an intelligent system that analyzes the user's daily life patterns and automatically generates notifications and instructions to perform necessary tasks.

[1296] "Daily life patterns" refer to the actions and habits that a user repeats every day.

[1297] "Task execution" refers to the actions or activities that a user must perform at a specific time and in a specific situation.

[1298] A "voice message" is a message that conveys information or instructions to the user by voice.

[1299] A "speaker" is an output device for reproducing a voice message as sound.

[1300] "User response" refers to the response or action taken by the user after receiving the voice message.

[1301] "Feedback" is the process of gathering user responses and adjusting the system's behavior based on that information.

[1302] "In-vehicle activity data" refers to data relating to the user's behavior and situation within an autonomous vehicle.

[1303] A "behavior pattern" is a combination of actions that a user repeatedly performs under specific circumstances.

[1304] "Task Suggestion" refers to recommending or informing the user about a specific action or activity to perform.

[1305] "Emotion analysis" is the process of recognizing and analyzing emotions from the user's voice, facial expressions, etc.

[1306] "Tone of notification" refers to the sound quality and speaking characteristics of a voice message, and is an element that is adjusted according to the user's emotional state.

[1307] An embodiment of the present invention will be described.

[1308] server:

[1309] The server first receives daily behavioral data from the vehicle and the user's device, including boarding and alighting times and in-vehicle activities (e.g., reading, eating, resting, etc.). This data is then stored in a database.

[1310] The server then uses generative AI to analyze the user's lifestyle patterns based on the stored behavioral data. For example, the server uses the KMeans clustering algorithm to classify daily life patterns and determine the time of day when certain activities occur.

[1311] Based on the analysis results, the server creates a task schedule to notify the user, and generates a voice message prompting the user to perform the task according to the schedule. The generated voice message is sent to the user's device at the appropriate time.

[1312] Furthermore, the emotion analysis engine analyzes the user's voice input and facial expression data to determine their emotions. Based on the results of this analysis, the generation AI adjusts the content and tone of the voice message to deliver notifications that take their emotions into consideration.

[1313] Device:

[1314] The terminal receives the voice message sent from the server and then plays the voice message to the user through the speaker, taking into consideration that the voice message is easy to hear even if the user is wearing a hearing aid.

[1315] After the user receives the voice message, the device receives the user's response as voice input, which is converted into text data and sent back to the server. An emotion analysis engine analyzes the voice data to identify the user's emotional state.

[1316] User:

[1317] The user receives a voice notification from the device, for example, hearing a message such as "It's 10 o'clock. Time to take your medicine." In response, the user performs a task (e.g., taking medicine) and reports the completion to the device by voice. The device receives voice feedback such as "Yes, I took my medicine" and sends it to the server. This information is used for future notifications.

[1318] The server can store this feedback in a database and reflect it in the next task notification or life pattern analysis. For example, if a user always responds with a specific tone of voice, the server can adjust the next notification based on that user's emotional state.

[1319] As a concrete example, we will show a scenario in which an elderly person is riding in an autonomous vehicle. The server receives and analyzes the riding data, generates a notification to take medicine at 10 o'clock, and notifies the user in a gentle tone if the user is feeling stressed. An example of this prompt sentence is, "To help the elderly person relax in the autonomous vehicle, please generate a voice message that gently notifies them that it is time to take their medicine at 10 o'clock. Also, if the user feels stressed, please notify them in a tone that requires special attention."

[1320] This will enable elderly people to live comfortably in self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that takes their emotions into consideration.

[1321] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1322] Step 1:

[1323] The device collects in-vehicle activity data. Using sensors and cameras, it records the time of boarding and exiting the vehicle, as well as activities in the vehicle (e.g., reading, eating, resting, etc.). The collected data is temporarily stored in the device.

[1324] Step 2:

[1325] The device sends the collected data to the server. The data is transferred to the server via a secure channel and stored in a database on the server. The input data is boarding and exiting times and in-car behavior data, and the output is stored in a database on the server.

[1326] Step 3:

[1327] The server uses generative AI to analyze the user's daily life patterns based on the stored data. Algorithms used here include KMeans clustering and other methods to classify daily life patterns. The input is the behavioral data in the database, and the output is the classified daily life patterns.

[1328] Step 4:

[1329] The server creates a task schedule based on the analysis results. It predicts tasks that the user should perform at specific times (e.g., taking medicine, staying hydrated), and generates a schedule to notify users at those times. The input is the analysis results, and the output is the task schedule.

[1330] Step 5:

[1331] The server uses the generative AI model to generate a voice message prompting the user to perform the task. The server also sets the prompt sentence when generating the message and takes into account the results of sentiment analysis. The input is the task schedule and emotion data, and the output is the voice message.

[1332] Step 6:

[1333] The server sends the generated voice message to the terminal. The terminal plays the received message to the user through the speaker and adjusts the voice settings to make it easy to hear even if the user is using a hearing aid. The input is the voice message and the output is a notification to the user.

[1334] Step 7:

[1335] The user receives a voice message and performs the instructed task. For example, if the user receives a notification that "It's 10 o'clock. It's time to take your medicine," the user will actually take the action of taking their medicine. The input is the voice message, and the output is the execution of the task.

[1336] Step 8:

[1337] The user reports task completion to the terminal by voice. The terminal converts this voice into text data and sends it to the server as feedback. The input is the user's voice and the output is text data.

[1338] Step 9:

[1339] The server receives the feedback and stores it in a database. An emotion analysis engine analyzes the user's voice data and identifies their emotional state. This information is used for future task notifications and lifestyle pattern analysis. The input is text data and emotion data, and the output is the database and analysis results.

[1340] This series of processing steps makes it possible to assist elderly people in their daily lives inside self-driving vehicles, reduce the difficulties they face due to forgetfulness, and provide flexible support that is sensitive to their emotions.

[1341] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1342] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1343] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1344] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1345] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1346] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1347] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1348] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1349] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1350] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1351] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1352] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1353] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1354] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1355] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1356] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1357] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1358] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1359] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1360] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1361] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1362] The following is further disclosed regarding the above embodiment.

[1363] (Claim 1)

[1364] a means for analyzing a user's daily life pattern using a generation artificial intelligence and generating a voice message to prompt the user to perform a task based on the extracted daily life pattern;

[1365] means for reproducing the generated voice message in a voice that is easy for the user to hear through a speaker or a hearing aid;

[1366] a means for receiving a user's response by voice input and feeding it back to the generating artificial intelligence;

[1367] A system including:

[1368] (Claim 2)

[1369] 10. The system of claim 1, further comprising means for receiving and storing user behavior data in a database.

[1370] (Claim 3)

[1371] The system according to claim 1, further comprising a generation artificial intelligence that generates voices that are easy to hear even when the user has age-related hearing loss.

[1372] "Example 1"

[1373] (Claim 1)

[1374] a means for analyzing a user's daily life pattern using a generation artificial intelligence and generating a voice message to prompt the user to perform a task based on the extracted daily life pattern;

[1375] means for reproducing the generated voice message in a voice that is easy for the user to hear via an acoustic device;

[1376] a means for receiving a user's response by voice input, converting the response into text data through voice recognition, and feeding the text data back to the generating artificial intelligence;

[1377] A means to save the user's task execution status in a database and use it for future notifications,

[1378] A system including:

[1379] (Claim 2)

[1380] 10. The system of claim 1, further comprising means for receiving the user's daily activity data and storing the data in a database.

[1381] (Claim 3)

[1382] The system according to claim 1, further comprising a generation artificial intelligence that generates voices that are easy to hear even when the user has a hearing impairment.

[1383] "Application Example 1"

[1384] (Claim 1)

[1385] A means for analyzing a user's daily life patterns using a generative AI model and generating a voice message or a text message to prompt the user to perform a task based on the extracted daily life patterns;

[1386] means for notifying a user of the generated voice or text message via an audio output device or a display device;

[1387] means for receiving a user's response by voice input or text input and feeding it back to the generative AI model;

[1388] A method to save the results of the user's actions in a database and reflect them in the next task schedule.

[1389] A system including:

[1390] (Claim 2)

[1391] 10. The system according to claim 1, wherein the system receives the user's behavioral data and work results and stores them in a database.

[1392] (Claim 3)

[1393] 10. The system of claim 1, including a generative AI model that causes a robot placed in a work environment to be prompted to perform a task at regular intervals.

[1394] "Example 2: Combining Emotion Engines"

[1395] (Claim 1)

[1396] means for analyzing a user's daily life pattern and generating a voice message to prompt the user to perform a task based on the extracted daily life pattern;

[1397] means for reproducing the generated voice message in a voice that is easy for the user to hear;

[1398] a means for receiving a user's response by voice input and feeding it back to the analysis result;

[1399] A means for analyzing the user's emotional data and adjusting the content and tone of the voice message;

[1400] A system including:

[1401] (Claim 2)

[1402] 10. The system of claim 1, further comprising means for receiving and storing user behavior data in a database.

[1403] (Claim 3)

[1404] 2. The system according to claim 1, further comprising means for generating sounds that are easy to hear even when the user has age-related hearing loss.

[1405] "Application example 2 when combining emotion engines"

[1406] (Claim 1)

[1407] a means for analyzing a user's daily life pattern using a generation artificial intelligence and generating a voice message to prompt the user to perform a task based on the extracted daily life pattern;

[1408] means for playing the generated voice message;

[1409] a means for receiving a user's response by voice input and feeding it back to the generating artificial intelligence;

[1410] means for collecting in-vehicle activity data and suggesting in-vehicle tasks based on behavioral patterns;

[1411] means for analyzing a user's emotions and adjusting the tone and content of notifications based on the user's emotional state;

[1412] A system including:

[1413] (Claim 2)

[1414] 10. The system of claim 1, further comprising means for receiving and storing user behavior data in a database.

[1415] (Claim 3)

[1416] The system according to claim 1, further comprising a generation artificial intelligence that generates voices that are easy to hear even when the user has age-related hearing loss. [Explanation of symbols]

[1417] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for analyzing a user's daily life pattern using a generation artificial intelligence and generating a voice message to prompt the user to perform a task based on the extracted daily life pattern; means for reproducing the generated voice message in a voice that is easy for the user to hear through a speaker or a hearing aid; a means for receiving a user's response by voice input and feeding it back to the generating artificial intelligence; A system including:

2. 10. The system of claim 1, further comprising means for receiving and storing user behavior data in a database.

3. The system according to claim 1, further comprising a generation artificial intelligence that generates voices that are easy to hear even when the user has age-related hearing loss.

Images