system

Abstract

Provide a system. 【Solution means】 A sensor device for monitoring the user's health condition, An artificial intelligence processing device for analyzing the acquired health condition data, A communication device for notifying the user or a third party based on the analysis result, A content generation device for generating content based on the user's preferences, A content providing device for providing the generated content, A communication control device for managing two-way communication with the user, An information processing device for processing the acquired health condition data in real time, A system including.

Description

Technical Field

[0005] ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the increase in the elderly population, especially the lack of health management, communication, and entertainment faced by the elderly living alone in their daily lives has become a problem. Support that can effectively solve these problems for the elderly is necessary, but it is often difficult for current technologies to flexibly respond to individual needs. The present invention aims to develop a system that provides more individualized support for these problems.

Means for Solving the Problems

[0005] This invention is characterized by acquiring user health information using a sensor device and analyzing that data with an artificial intelligence processing device. Furthermore, based on the analysis results, it supports the detection of abnormalities in health status and daily health management by sending appropriate notifications to the user via a communication device. In addition, a content generation device generates optimal entertainment based on the user's preferences and makes it viewable through a content provision device. Moreover, a communication control device smoothly establishes communication between the user and external parties, thereby resolving communication deficiencies. This constitutes a system that enables a comprehensive improvement in the quality of life for the elderly.

[0006] A "sensor device" is an electronic device used to acquire data about a user's physical condition.

[0007] An "artificial intelligence processing device" is a device that analyzes acquired data and processes necessary information based on factors such as health status.

[0008] A "communication device" is a device used to send notifications to users based on analysis results.

[0009] A "content generation device" is a device that generates optimal entertainment based on the user's past preferences and data.

[0010] A "content provision device" is a device that provides generated entertainment content to users in a viewable format.

[0011] A "communication control device" is a device that establishes communication between a user and an external party, and manages and controls that communication. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is a comprehensive system designed to support the lives of elderly users, providing a combination of three main functions: health management, entertainment, and communication. Specific embodiments of each function are described below.

[0034] Health management function

[0035] Users measure their daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is immediately transmitted to the device.

[0036] The device sends the measured data to the server. The server analyzes the data using advanced artificial intelligence algorithms and detects anomalies.

[0037] If an anomaly is detected, the server analyzes the results and sends a notification to the device. Upon receiving the notification, the device alerts the user through audio or visual alerts.

[0038] Entertainment features

[0039] The server analyzes the user's past content viewing history and preferences. Based on this, it generates the most suitable music, stories, or games.

[0040] The generated content is delivered to the user through the device. Users can select and enjoy the content using voice commands or touch controls.

[0041] Communication function

[0042] Users can initiate video calls with family and friends through the device's interface.

[0043] The device requests the necessary connection from the server for the video call. The server manages the connection to provide stable communication and guarantees call quality.

[0044] During a video call, the device processes video and audio in real time, providing users with smooth and high-quality communication.

[0045] For example, in a daily health check, the user uses a wearable device to take measurements, and the device automatically sends that data to a server. If the server detects an abnormal value, the device immediately issues an alert and, in some cases, also sends a notification to family members. Also, once a week, the user can have a video call with their grandchildren who live far away, during which the device provides high-quality audio and video, and the server maintains the stability of the communication.

[0046] Thus, the present invention is implemented as a system that comprehensively provides the necessary support in daily life in response to the diverse needs of the elderly.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] Users measure their health data (e.g., blood pressure and heart rate) using a wearable device. The measured data is automatically sent to the device.

[0050] Step 2:

[0051] The device transmits received health data to the server in real time. The data is transmitted using a secure communication protocol.

[0052] Step 3:

[0053] The server analyzes the received health data using artificial intelligence algorithms. It compares it with past data to detect any anomalies.

[0054] Step 4:

[0055] If an anomaly is detected, the server generates a warning message based on the result and sends it to the terminal. This message may include a recommendation to contact a medical institution.

[0056] Step 5:

[0057] The device communicates received warning messages to the user as audio and visual notifications. These notifications are delivered in a pre-configured manner.

[0058] Step 6:

[0059] Users can recognize notifications from their devices and contact medical institutions or family members as needed. They can also review past notifications.

[0060] Step 7:

[0061] The server periodically analyzes user preferences and past usage history to generate new entertainment content. This content is optimized to meet individual needs.

[0062] Step 8:

[0063] The device provides the user with newly generated content. The user can select and enjoy the content using voice commands.

[0064] Step 9:

[0065] Users initiate video calls with their family using the device's interface.

[0066] Step 10:

[0067] The terminal sends a request to the server to establish a video call connection. The server manages the call and performs the necessary controls to maintain communication stability.

[0068] Step 11:

[0069] During a video call, the device processes video and audio data in real time to provide the user with a high-quality experience.

[0070] Step 12:

[0071] After the call ends, the device uploads the call data to the server, where it is saved as reference information for future calls.

[0072] (Example 1)

[0073] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0074] For diverse users, including the elderly, to live with peace of mind, real-time monitoring of their health status and prompt notification of abnormalities are necessary. Furthermore, not only health management, but also personalized information provision to enrich daily life and smooth communication with external parties are essential. However, a system that can centrally manage all of these is not yet available, making it a challenge to improve users' quality of life.

[0075] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0076] In this invention, the server includes means for utilizing a wearable device for collecting health status data, means for utilizing a computing device for analyzing the collected health data and detecting abnormalities, and means for providing a communication device for issuing warnings to the user or others based on the analysis results. This enables real-time monitoring of the user's health status, prompt notification in the event of an abnormality, and provision of appropriate information.

[0077] A "wearable device" is a device that a user can wear and that has the function of continuously collecting health data.

[0078] A "computational device" is a device that processes and analyzes collected health data to detect abnormalities, and has the function of executing programs for data analysis.

[0079] A "communication device" is a device that sends notifications or warnings to users or others based on anomalies detected by a computing device.

[0080] An "information generation device" is a device that generates individually optimized information based on the user's preferences and has the function of creating new content, etc., by taking into account the user's past data.

[0081] An "information provision device" is a device that provides generated information to a user, enabling the user to view or manipulate that information.

[0082] A "communication management device" is a device that establishes and manages smooth communication between users and others, and has the function of ensuring call connection and quality.

[0083] This invention is a system that integrates health management, information provision, and communication functions to support the user's daily life. Specific embodiments of the system are described below.

[0084] Health management function

[0085] The user wears a wearable device, such as a wristwatch, to collect health data. This device continuously monitors the user's vital signs, such as heart rate and blood pressure, and transmits the data to a terminal via Bluetooth. The terminal then transmits this data to a server via the internet. The server analyzes this data in real time using analysis software with a generative AI model, and if an abnormality is detected, it notifies the user through the terminal.

[0086] Information provision function

[0087] The server analyzes the user's past information usage history and generates information based on the user's preferences. For example, it uses an AI model to create new movies and playlists based on the user's past movie and music viewing history. The generated information is sent to the device, and the user can view or interact with it through voice commands or touch controls.

[0088] Communication function

[0089] Users can communicate with others using the terminal's interface. When initiating a video call, the terminal requests a connection to the server. The server utilizes the functions of a communication management device to provide a stable communication environment. The server dynamically manages communication quality, and the terminal processes high-quality video and audio in real time, providing users with smooth communication.

[0090] For example, if a user's heart rate is detected to be abnormal in the morning, the server immediately sends an alert via the device and also sends information to the emergency contacts designated by the user. In the afternoon, the user can watch the latest movies through the device, and in the evening, they can enjoy video calls with their family. In this way, the service is provided to make the user's life richer and safer.

[0091] Examples of prompt messages include "Generate the latest recommended movies" and "Detect abnormal heart rate." This allows for the provision of flexible services tailored to user requests.

[0092] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0093] Step 1:

[0094] The user wears a wearable device that collects vital data such as heart rate and blood pressure in real time. This data is transmitted to a terminal via Bluetooth. The input is biometric data acquired from the wearable device, and the output is biometric data transmitted to the terminal. Specifically, the device periodically measures data and transfers it to the terminal.

[0095] Step 2:

[0096] The terminal transmits biometric data received from wearable devices to a server via the internet. The input is biometric data stored on the terminal, and the output is biometric data sent to the server. Specifically, an application running on the terminal transmits the data in the background, and SSL / TLS encrypted communication is used as the transmission protocol.

[0097] Step 3:

[0098] The server analyzes the received biometric data using a generating AI model to detect anomalies. The input is the biometric data sent to the server, and the output is the analysis results of the anomaly detection. Specifically, the AI ​​model in the server processes the data using a pattern recognition algorithm and classifies it into normal and abnormal values.

[0099] Step 4:

[0100] The server sends a notification to the terminal when an anomaly is detected. The input is the analysis result of the anomaly detection, and the output is the notification data sent to the terminal. Specifically, the server uses a push notification service to send an alert to the terminal, and when the terminal receives it, it notifies the user with a pop-up or warning sound.

[0101] Step 5:

[0102] The user receives this notification and follows the corresponding instructions. Input is the notification from the device, and output is the user's response or action. Specifically, this could involve following voice guidance to check data or consult with a doctor.

[0103] Step 6:

[0104] The server uses the user's past information usage history to generate new information (e.g., music and movie lists) based on the user's preferences, optimizing it with an AI model. The input is past data usage history, and the output is an optimized information list. Specifically, a recommendation algorithm based on historical data is in operation, generating individually personalized information.

[0105] Step 7:

[0106] The generated information is sent to the device and provided for the user to view or interact with. The input is an optimized information list sent from the server, and the output is the user's viewing or interaction with the content on their device. Specifically, access to the information is made possible through features such as information display and selection options within the app.

[0107] In this way, the system enables a process that smoothly supports users' daily health management, information utilization, and communication.

[0108] (Application Example 1)

[0109] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0110] In the lives of the elderly, there is a need to appropriately monitor their health and respond quickly when abnormalities occur. Furthermore, it is necessary to provide personalized entertainment and facilitate smooth communication with family members living far away. However, conventional systems have been unable to comprehensively provide all of these functions.

[0111] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0112] In this invention, the server includes a sensor device for monitoring the user's health status, an artificial intelligence processing device for analyzing the acquired health status data, and a communication device for notifying the user or a third party based on the analysis results. This makes it possible to monitor the user's health status in real time, quickly detect abnormalities, and notify relevant parties as needed.

[0113] "Health status" refers to information indicating the user's physical parameters, including biometric data acquired by sensor devices.

[0114] A "sensor device" is a device used to monitor a user's health status and has the function of collecting biometric data such as blood pressure and heart rate.

[0115] An "artificial intelligence processing device" is a device that analyzes acquired health status data and detects abnormalities, and it uses machine learning algorithms.

[0116] A "communication device" is a device that notifies the user or a third party based on the analysis results, and has network connectivity.

[0117] A "content generation device" is a system that generates content based on user preferences, providing personalized entertainment.

[0118] A "content delivery device" is a device that delivers generated content to users, transmitting information through sight or sound.

[0119] A "communication control device" is a device for managing two-way communication with users and for establishing real-time communication with external parties.

[0120] An "information processing device" is a device that processes acquired health status data in real time, enabling high-speed data analysis.

[0121] The system for realizing this invention has the function of monitoring the user's health status in real time and immediately notifying the user if an abnormality in their health status is detected. This system includes a sensor device for acquiring health information, an artificial intelligence processing device for analyzing the data, a communication device for sending notifications based on the analysis results, a content generation device for generating content according to the user's preferences, a content provision device for providing the generated content, a communication control device for managing two-way communication, and an information processing device for processing the acquired data in real time.

[0122] The server receives health data (e.g., blood pressure, heart rate) from sensor devices such as wearable devices and analyzes the data via an artificial intelligence processing unit that runs machine learning algorithms. If this analysis detects an abnormal health condition, the server sends an alert to the user's smartphone or care staff via a communication device. The specific hardware used includes wearable devices known as fitness trackers and smartphones, while the software uses Python's TENSORFLOW® and Firebase Cloud Messaging.

[0123] The device learns the user's preferences from past usage history and provides personalized entertainment through a content generation device that creates music and stories. A communication control device allows the user to make high-quality video calls with family members in distant locations.

[0124] As a concrete example, when a user performs a morning health check, data acquired from a wearable device is sent to a server. If an abnormality is detected, a notification is immediately sent to the user's smartphone or the caregiver's terminal. Furthermore, when making a video call with grandchildren on the weekend, a communication control device enables high-quality video and audio communication.

[0125] Examples of input prompts for a generative AI model:

[0126] "Please design a proof-of-concept system that monitors the health status of elderly individuals in real time and automatically notifies them when an abnormality is detected."

[0127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0128] Step 1:

[0129] The user wears a wearable device to measure health data (e.g., blood pressure, heart rate). The input is biometric data from the wearable device, and the output is information about the user's health status. This data is transmitted to the terminal via Bluetooth.

[0130] Step 2:

[0131] The device sends the received health data to a cloud server. In this process, the input is health status data, and the device acts as the sender of this data to the server. An internet connection is used for data transmission.

[0132] Step 3:

[0133] The server analyzes the received health status data. The input here is data sent from the terminal, and anomalies are detected using machine learning algorithms. The output is the analysis results.

[0134] Step 4:

[0135] Based on the analysis results, the server generates a notification if an abnormality is found in the health condition. The input is the analysis results, and the notification content is output. This notification is sent to the devices of care staff and family members as needed.

[0136] Step 5:

[0137] The device understands the user's preferences and generates entertainment content. The input is the user's past viewing history, and new, customized music and stories are output. A generative AI model is used for content generation.

[0138] Step 6:

[0139] The terminal establishes high-quality video calls with family members located remotely via a communication control device. The input is the user's call request, and the output is smooth audio and video. Streaming technology is used to send and receive data in real time.

[0140] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0141] This invention is an innovative system designed to support the lives of elderly users, incorporating health management, entertainment, and communication functions, aiming to realize a safe and fulfilling life. Furthermore, by incorporating an emotion engine, it is possible to provide advanced services that take into account the user's emotional state. Specific embodiments of each function are described below.

[0142] Health management and emotion recognition functions

[0143] Users collect daily health data using wearable devices. This data is transmitted to a server in real time by the device.

[0144] The server analyzes health data using artificial intelligence algorithms and processes user facial expressions and voice data acquired from the device using an emotion engine to recognize the user's emotional state. This information is then reflected in notifications and health management feedback.

[0145] Entertainment features

[0146] The server generates optimal entertainment content based on the user's emotional state and past content viewing history. This process includes data analysis by an emotion engine.

[0147] The terminal provides the user with generated content and creates an environment where the user can view or listen to selected content. The user can operate the terminal using voice commands.

[0148] Communication function

[0149] Users can initiate video calls with family and friends in remote locations through their devices.

[0150] The device sends a request to the server to establish a video call connection and ensure stable communication.

[0151] During a video call, the emotion engine assesses the user's emotional state in real time and provides advice and suggestions as needed.

[0152] For example, if a user performing a morning health check is feeling a little depressed, the emotion engine will recognize that emotion, and the server will generate and provide relaxing music or video content tailored to the user's preferences. Furthermore, if a user feels anxious during a video call, the device may immediately suggest encouraging messages or relaxation techniques. By combining these emotion engines, a system that provides even greater support for the user's overall life can be realized.

[0153] The following describes the processing flow.

[0154] Step 1:

[0155] Users collect daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is transmitted to the device via Bluetooth or Wi-Fi connection.

[0156] Step 2:

[0157] The device sends the received health data to the server. Data transmission is protected by an encryption protocol.

[0158] Step 3:

[0159] The server analyzes health data using an AI algorithm. During this process, it compares the data to reference values ​​and evaluates whether any abnormal values ​​exist.

[0160] Step 4:

[0161] The server simultaneously uses video and audio data from the terminal to recognize the user's emotional state using an emotion engine. It analyzes emotions from the user's facial expressions and tone of voice.

[0162] Step 5:

[0163] Based on the analysis of health data and the results of emotion recognition, the server generates personalized feedback and notifications. This feedback includes advice tailored to the user's current health status and emotions.

[0164] Step 6:

[0165] The device notifies the user of notifications from the server. These notifications are displayed as voice messages using a voice assistant or as visual alerts on the screen.

[0166] Step 7:

[0167] Users can check their health status based on notifications and contact a medical institution if necessary. They can also accept recommendations for relaxation methods tailored to their emotions.

[0168] Step 8:

[0169] The server analyzes the user's emotional state and content viewing history to generate optimal entertainment content (music, videos, games).

[0170] Step 9:

[0171] The device provides the user with generated entertainment content. The user can control and enjoy the content using voice commands.

[0172] Step 10:

[0173] If a user wants to make a video call with family or friends using their device, they can initiate the call with a voice command.

[0174] Step 11:

[0175] The device sends a request to the server when initiating a call. The server manages the communication network and ensures the stability of the connection.

[0176] Step 12:

[0177] During video calls, the emotion engine tracks the user's emotional state in real time and suggests comfort messages and relaxation techniques as needed.

[0178] Step 13:

[0179] After the call ends, the device uploads call-related data to a server for later analysis and service improvement.

[0180] (Example 2)

[0181] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0182] In recent years, with the advancement of an aging society, managing the health and improving the quality of life of elderly users has become a crucial issue. In particular, there is a need to provide appropriate content and communication support while simultaneously considering the user's health information and emotional state. However, conventional systems have struggled to provide these functions in an integrated manner, and have been unable to provide sufficient support for users.

[0183] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0184] In this invention, the server includes means for acquiring user health information and recognizing emotional state, means for generating content based on the user's emotional state and past viewing history using a generative AI model, and means for establishing communication between the user and a remote person, evaluating emotions in real time, and providing advice. This enables advanced health management, entertainment, and communication support tailored to the user's individual condition.

[0185] "User" refers to an individual who uses this system, and specifically to elderly people.

[0186] "Health information" refers to data about the user's physical condition, including heart rate, body temperature, and steps taken.

[0187] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices.

[0188] An "artificial intelligence processing device" refers to a system that analyzes acquired data and recognizes emotional states.

[0189] A "communication device" is a device used to provide users with notifications and feedback based on analysis results.

[0190] A "content generation device" is a system for generating entertainment content based on a user's emotional state and past viewing history.

[0191] A "content delivery device" is a device that provides generated content to users via voice commands.

[0192] A "communication control device" is a device that establishes communication between a user and a person in a remote location and monitors the user's emotional state in real time during a call.

[0193] "Emotional state" refers to information that represents the user's psychological state, and is recognized through facial expressions, tone of voice, and other factors.

[0194] A "generative AI model" refers to an artificial intelligence program that generates optimal content based on user data.

[0195] This invention provides a system that offers health management, entertainment, and communication for elderly users, and provides support that takes their emotional state into consideration. Users wear wearable devices to acquire health information such as heart rate, steps taken, and body temperature on a daily basis. These devices can be general smartwatches or fitness trackers.

[0196] User health information is sent to a device via Bluetooth. This device, a smartphone or tablet, transfers this health data to a server located in the cloud via the internet. On the server, the health data is analyzed using artificial intelligence algorithms, employing AI frameworks such as TensorFlow. Additionally, user facial expressions and voice data are collected by the device and sent to the server. This emotional data is analyzed by an emotion engine such as the Affectiva SDK.

[0197] Based on the analysis results, the server evaluates the user's health and emotional state and generates appropriate feedback. For example, if the user is feeling stressed, the server recommends relaxing music or video content and provides it through the device. In this case, external content services such as Spotify and Netflix can be used.

[0198] In the entertainment features, the server uses a generative AI model to generate optimized content based on the user's past content viewing history and emotional state. Users can select and view their desired content using voice commands. A specific example would be the prompt, "Play some relaxing music."

[0199] The communication features allow users to enjoy conversations with family and friends in remote locations through video call applications. The device manages communication for stable video calls and uses an emotion engine to assess the user's emotional state in real time during calls. If the user feels anxious, support can be provided through relaxation techniques and encouraging messages.

[0200] In this way, this system integrates health, entertainment, and communication, making it possible to provide users with advanced, emotion-based services.

[0201] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0202] Step 1:

[0203] Users acquire health information using wearable devices. This health information includes heart rate, steps taken, and body temperature. This information is transmitted via Bluetooth to devices such as smartphones and tablets. The input is real-time data from the wearable device, and the output is the reception of this data by the device.

[0204] Step 2:

[0205] The device transmits received health information to a server in the cloud via the internet. During this process, the device collects the user's facial expressions and voice data, which are also transmitted to the server. The input consists of health and emotional data stored on the device, and the output is the transmission of data to the server.

[0206] Step 3:

[0207] The server analyzes health information using AI algorithms. Specifically, this involves a process that uses machine learning models to detect anomalies and evaluate patterns. The input is the health data received in step 2, and the analyzed health status is generated as output.

[0208] Step 4:

[0209] The server processes facial and voice data using an emotion engine to recognize the user's emotional state. The emotion engine uses deep learning to calculate an emotional state label from the data. The input is the emotional data from step 2, and the output is the recognized emotional state.

[0210] Step 5:

[0211] The server generates feedback for the user based on the analysis of their health and emotional state. For example, if the user is stressed, relaxing content will be recommended. This feedback is provided to the user through data transmission to the device. The input is the analysis results from steps 3 and 4, and the output is the recommendation notification.

[0212] Step 6:

[0213] The server uses a generative AI model to generate optimal entertainment content based on the user's past viewing history and emotional state. The generated content is sent to the device and provided to the user. The input is viewing history and emotional state, and the output is optimized content.

[0214] Step 7:

[0215] The terminal provides the generated content to the user using a voice command interface. The user can select content by voice and begin playback. The input is the content data received from the server, and the output is the user starting playback of the content.

[0216] Step 8:

[0217] The user initiates a video call with a remote person using a device. The device controls the communication and sends a call request to the server. The input is the video call request, and the output is the establishment of the call connection.

[0218] Step 9:

[0219] During a video call, the device uses real-time data to assess the user's emotional state and receives appropriate responses from the server. If the user shows signs of anxiety, the device notifies them of relaxation techniques or encouraging messages. The input is emotional state data, and the output is the notified response.

[0220] (Application Example 2)

[0221] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0222] In the lives of the elderly, there are challenges in receiving appropriate support due to insufficient management of their health status and understanding of their emotions. In particular, there is a lack of daily support and entertainment that is tailored to their emotional state. Therefore, a system is needed that can provide optimal services according to each individual's emotions and health condition.

[0223] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0224] In this invention, the server includes means for acquiring user health information, means for analyzing health information and emotional state, and means for providing notifications and content based on the analysis results and emotional state. This enables the provision of individually optimized support and entertainment tailored to the user's health and emotional state.

[0225] A "user" is an individual who utilizes the system's functions and receives support for health management, entertainment, and communication.

[0226] "Health information" refers to physiological and activity data acquired through sensor devices, and is information that indicates the user's physical condition.

[0227] "Emotional state" refers to information that represents the user's psychological condition, obtained as a result of the emotion engine's analysis based on data such as the user's facial expressions and voice.

[0228] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices, etc.

[0229] An "artificial intelligence processing device" is a computer device that analyzes acquired health information and emotional states and outputs the analysis results.

[0230] A "communication device" is a device used to transmit analyzed data to a user or an external party.

[0231] A "content generation device" is a part of a system that creates entertainment content according to the user's preferences and emotional state.

[0232] A "content provision device" is a device that provides generated content to users in the form of viewing or listening.

[0233] A "communication control device" is a device that establishes communication between a user and an external person and has the function of providing emotion-based advice as needed.

[0234] The system for realizing this invention comprehensively manages and supports the user's health and emotions, and has the following configuration.

[0235] The server acquires and collects health information provided by the user via wearable devices in real time. This includes basic physiological data such as heart rate and step count. The artificial intelligence processing unit within the server analyzes emotions using this data along with the user's facial expressions and voice data. The software used incorporates machine learning algorithms, giving it the ability to accurately identify the user's health and emotional state. Facial recognition technology is commonly used for processing facial expression data. Voice analysis technology is also incorporated for emotion recognition of voice.

[0236] The analysis results are transmitted to a communication device that connects to the user's smartphone or tablet and provides feedback to the user. Through this device, the user operates using voice commands. The content generation device creates optimal entertainment content based on the acquired emotional state and past viewing history. The generated content is provided to the user's device and set up for viewing.

[0237] Furthermore, the communication control device has the functionality to enable video calls between the user and family members or caregivers located remotely. During the call, the server's emotion engine operates, providing real-time advice tailored to the user's mental state.

[0238] As a concrete example, if a user performs a routine health check and feels mild anxiety, the system will suggest music with a relaxing effect. For instance, it might send a prompt to the generative AI model such as, "If the user is analyzed as feeling depressed, suggest relaxing nature music."

[0239] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0240] Step 1:

[0241] The server acquires user health information through wearable devices. This information includes physiological data such as heart rate and step count. The server collects this data in real time and stores it in an internal database to monitor the user's health status. The input is physiological data from the wearable device, and the output is health information stored in the internal database.

[0242] Step 2:

[0243] The server receives facial expression and voice data transmitted from the user's terminal. This data is input into the emotion engine and analyzed by machine learning algorithms. The resulting emotional state reflects the user's real-time psychological state. The input is facial expression and voice data, and the output is the analyzed emotional state.

[0244] Step 3:

[0245] The server uses a generative AI model to generate prompt sentences based on health information and emotional state. These prompt sentences are generated in a format such as, "If the user is analyzed as being in a depressed mood, suggest relaxing nature music." These prompt sentences are then used as instructions for the content generation device. The input is health information and emotional state, and the output is the generated prompt sentence.

[0246] Step 4:

[0247] The terminal provides the user with entertainment content received from a content generator based on prompt messages. The user can select and control content using voice commands. Input is the generated prompt messages, and output is the provision of selectable content to the user.

[0248] Step 5:

[0249] The user initiates a video call with family or caregivers in a remote location via their device. During the call, the server monitors the user's mental state in real time and provides emotionally-based advice as needed. The inputs are the video call request and the user's emotional state, while the output is the advice provided to the user.

[0250] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0251] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0252] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0253] [Second Embodiment]

[0254] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0255] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0256] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0257] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0258] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0259] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0260] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0261] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0262] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0263] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0264] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0265] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0266] This invention is a comprehensive system designed to support the lives of elderly users, providing a combination of three main functions: health management, entertainment, and communication. Specific embodiments of each function are described below.

[0267] Health management function

[0268] Users measure their daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is immediately transmitted to the device.

[0269] The device sends the measured data to the server. The server analyzes the data using advanced artificial intelligence algorithms and detects anomalies.

[0270] If an anomaly is detected, the server analyzes the results and sends a notification to the device. Upon receiving the notification, the device alerts the user through audio or visual alerts.

[0271] Entertainment features

[0272] The server analyzes the user's past content viewing history and preferences. Based on this, it generates the most suitable music, stories, or games.

[0273] The generated content is delivered to the user through the device. Users can select and enjoy the content using voice commands or touch controls.

[0274] Communication function

[0275] Users can initiate video calls with family and friends through the device's interface.

[0276] The device requests the necessary connection from the server for the video call. The server manages the connection to provide stable communication and guarantees call quality.

[0277] During a video call, the device processes video and audio in real time, providing users with smooth and high-quality communication.

[0278] For example, in a daily health check, the user uses a wearable device to take measurements, and the device automatically sends that data to a server. If the server detects an abnormal value, the device immediately issues an alert and, in some cases, also sends a notification to family members. Also, once a week, the user can have a video call with their grandchildren who live far away, during which the device provides high-quality audio and video, and the server maintains the stability of the communication.

[0279] Thus, the present invention is implemented as a system that comprehensively provides the support necessary in daily life according to the diverse needs of the elderly.

[0280] The following describes the processing flow.

[0281] Step 1:

[0282] The user measures their health data (e.g., blood pressure and heart rate) using a wearable device. The measured data is automatically transmitted to the terminal.

[0283] Step 2:

[0284] The terminal transmits the received health data to the server in real time. The data is transmitted using a secure communication protocol.

[0285] Step 3:

[0286] The server analyzes the received health data using an artificial intelligence algorithm. It detects whether there are any outliers by comparing with past data.

[0287] Step 4:

[0288] If an abnormality is detected, the server generates a warning message based on the result and transmits it to the terminal. This message may include a recommendation to contact a medical institution.

[0289] Step 5:

[0290] The terminal conveys the received warning message to the user as an audio and visual notification. The notification is published in a pre-set manner.

[0291] Step 6:

[0292] Users can recognize notifications from their devices and contact medical institutions or family members as needed. They can also review past notifications.

[0293] Step 7:

[0294] The server periodically analyzes user preferences and past usage history to generate new entertainment content. This content is optimized to meet individual needs.

[0295] Step 8:

[0296] The device provides the user with newly generated content. The user can select and enjoy the content using voice commands.

[0297] Step 9:

[0298] Users initiate video calls with their family using the device's interface.

[0299] Step 10:

[0300] The terminal sends a request to the server to establish a video call connection. The server manages the call and performs the necessary controls to maintain communication stability.

[0301] Step 11:

[0302] During a video call, the device processes video and audio data in real time to provide the user with a high-quality experience.

[0303] Step 12:

[0304] After the call ends, the device uploads the call data to the server, where it is saved as reference information for future calls.

[0305] (Example 1)

[0306] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0307] In order for various users including the elderly to live with peace of mind, real-time monitoring of their health status and prompt notification in case of abnormalities are necessary. In addition to health management, it is also essential to provide individually optimized information to enrich daily life and to enable smooth communication with the outside world. However, since a system that can perform these functions in a unified manner has not yet been provided, it is an issue to improve the quality of users' lives.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0309] In this invention, the server includes means for using a wearable device for collecting health status data, means for using a computing device for analyzing the collected health data and detecting abnormalities, and means for providing a communication device for warning the user or others based on the analysis results. Thereby, it becomes possible to monitor the user's health status in real time, to provide prompt notification in case of abnormalities, and to provide appropriate information.

[0310] A "wearable device" is a device that a user can wear and has a function of continuously collecting health status data.

[0311] A "computing device" is a device for processing and analyzing the collected health data and detecting abnormalities, and has a function of executing a program for data analysis.

[0312] A "communication device" is a device for notifying or warning the user or others based on the abnormalities detected by the computing device.

[0313] An "information generation device" is a device that generates individually optimized information based on the user's preferences and has the function of creating new content, etc., by taking into account the user's past data.

[0314] An "information provision device" is a device that provides generated information to a user, enabling the user to view or manipulate that information.

[0315] A "communication management device" is a device that establishes and manages smooth communication between users and others, and has the function of ensuring call connection and quality.

[0316] This invention is a system that integrates health management, information provision, and communication functions to support the user's daily life. Specific embodiments of the system are described below.

[0317] Health management function

[0318] The user wears a wearable device, such as a wristwatch, to collect health data. This device continuously monitors the user's vital signs, such as heart rate and blood pressure, and transmits the data to a terminal via Bluetooth. The terminal then transmits this data to a server via the internet. The server analyzes this data in real time using analysis software with a generative AI model, and if an abnormality is detected, it notifies the user through the terminal.

[0319] Information provision function

[0320] The server analyzes the user's past information usage history and generates information based on the user's preferences. For example, it uses an AI model to create new movies and playlists based on the user's past movie and music viewing history. The generated information is sent to the device, and the user can view or interact with it through voice commands or touch controls.

[0321] Communication function

[0322] Users can communicate with others using the terminal's interface. When initiating a video call, the terminal requests a connection to the server. The server utilizes the functions of a communication management device to provide a stable communication environment. The server dynamically manages communication quality, and the terminal processes high-quality video and audio in real time, providing users with smooth communication.

[0323] For example, if a user's heart rate is detected to be abnormal in the morning, the server immediately sends an alert via the device and also sends information to the emergency contacts designated by the user. In the afternoon, the user can watch the latest movies through the device, and in the evening, they can enjoy video calls with their family. In this way, the service is provided to make the user's life richer and safer.

[0324] Examples of prompt messages include "Generate the latest recommended movies" and "Detect abnormal heart rate." This allows for the provision of flexible services tailored to user requests.

[0325] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0326] Step 1:

[0327] The user wears a wearable device that collects vital data such as heart rate and blood pressure in real time. This data is transmitted to a terminal via Bluetooth. The input is biometric data acquired from the wearable device, and the output is biometric data transmitted to the terminal. Specifically, the device periodically measures data and transfers it to the terminal.

[0328] Step 2:

[0329] The terminal transmits biometric data received from wearable devices to a server via the internet. The input is biometric data stored on the terminal, and the output is biometric data sent to the server. Specifically, an application running on the terminal transmits the data in the background, and SSL / TLS encrypted communication is used as the transmission protocol.

[0330] Step 3:

[0331] The server analyzes the received biometric data using a generating AI model to detect anomalies. The input is the biometric data sent to the server, and the output is the analysis results of the anomaly detection. Specifically, the AI ​​model in the server processes the data using a pattern recognition algorithm and classifies it into normal and abnormal values.

[0332] Step 4:

[0333] The server sends a notification to the terminal when an anomaly is detected. The input is the analysis result of the anomaly detection, and the output is the notification data sent to the terminal. Specifically, the server uses a push notification service to send an alert to the terminal, and when the terminal receives it, it notifies the user with a pop-up or warning sound.

[0334] Step 5:

[0335] The user receives this notification and follows the corresponding instructions. Input is the notification from the device, and output is the user's response or action. Specifically, this could involve following voice guidance to check data or consult with a doctor.

[0336] Step 6:

[0337] The server uses the user's past information usage history to generate new information (e.g., music and movie lists) based on the user's preferences, optimizing it with an AI model. The input is past data usage history, and the output is an optimized information list. Specifically, a recommendation algorithm based on historical data is in operation, generating individually personalized information.

[0338] Step 7:

[0339] The generated information is sent to the device and provided for the user to view or interact with. The input is an optimized information list sent from the server, and the output is the user's viewing or interaction with the content on their device. Specifically, access to the information is made possible through features such as information display and selection options within the app.

[0340] In this way, the system enables a process that smoothly supports users' daily health management, information utilization, and communication.

[0341] (Application Example 1)

[0342] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0343] In the lives of the elderly, there is a need to appropriately monitor their health and respond quickly when abnormalities occur. Furthermore, it is necessary to provide personalized entertainment and facilitate smooth communication with family members living far away. However, conventional systems have been unable to comprehensively provide all of these functions.

[0344] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0345] In this invention, the server includes a sensor device for monitoring the user's health status, an artificial intelligence processing device for analyzing the acquired health status data, and a communication device for notifying the user or a third party based on the analysis results. This makes it possible to monitor the user's health status in real time, quickly detect abnormalities, and notify relevant parties as needed.

[0346] "Health status" refers to information indicating the user's physical parameters, including biometric data acquired by sensor devices.

[0347] A "sensor device" is a device used to monitor a user's health status and has the function of collecting biometric data such as blood pressure and heart rate.

[0348] An "artificial intelligence processing device" is a device that analyzes acquired health status data and detects abnormalities, and it uses machine learning algorithms.

[0349] A "communication device" is a device that notifies the user or a third party based on the analysis results, and has network connectivity.

[0350] A "content generation device" is a system that generates content based on user preferences, providing personalized entertainment.

[0351] A "content delivery device" is a device that delivers generated content to users, transmitting information through sight or sound.

[0352] A "communication control device" is a device for managing two-way communication with users and for establishing real-time communication with external parties.

[0353] An "information processing device" is a device that processes acquired health status data in real time, enabling high-speed data analysis.

[0354] The system for realizing this invention has the function of monitoring the user's health status in real time and immediately notifying the user if an abnormality in their health status is detected. This system includes a sensor device for acquiring health information, an artificial intelligence processing device for analyzing the data, a communication device for sending notifications based on the analysis results, a content generation device for generating content according to the user's preferences, a content provision device for providing the generated content, a communication control device for managing two-way communication, and an information processing device for processing the acquired data in real time.

[0355] The server receives health data (e.g., blood pressure, heart rate) from sensor devices such as wearable devices and analyzes the data via an artificial intelligence processing unit that runs machine learning algorithms. If this analysis detects an abnormal health condition, the server sends an alert to the user's smartphone or care staff via a communication device. The specific hardware used includes wearable devices known as fitness trackers and smartphones, while the software uses Python's TensorFlow and Firebase Cloud Messaging.

[0356] The device learns the user's preferences from past usage history and provides personalized entertainment through a content generation device that creates music and stories. A communication control device allows the user to make high-quality video calls with family members in distant locations.

[0357] As a concrete example, when a user performs a morning health check, data acquired from a wearable device is sent to a server. If an abnormality is detected, a notification is immediately sent to the user's smartphone or the caregiver's terminal. Furthermore, when making a video call with grandchildren on the weekend, a communication control device enables high-quality video and audio communication.

[0358] Examples of input prompts for a generative AI model:

[0359] "Please design a proof-of-concept system that monitors the health status of elderly individuals in real time and automatically notifies them when an abnormality is detected."

[0360] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0361] Step 1:

[0362] The user wears a wearable device to measure health data (e.g., blood pressure, heart rate). The input is biometric data from the wearable device, and the output is information about the user's health status. This data is transmitted to the terminal via Bluetooth.

[0363] Step 2:

[0364] The device sends the received health data to a cloud server. In this process, the input is health status data, and the device acts as the sender of this data to the server. An internet connection is used for data transmission.

[0365] Step 3:

[0366] The server analyzes the received health status data. The input here is data sent from the terminal, and anomalies are detected using machine learning algorithms. The output is the analysis results.

[0367] Step 4:

[0368] Based on the analysis results, the server generates a notification if an abnormality is found in the health condition. The input is the analysis results, and the notification content is output. This notification is sent to the devices of care staff and family members as needed.

[0369] Step 5:

[0370] The device understands the user's preferences and generates entertainment content. The input is the user's past viewing history, and new, customized music and stories are output. A generative AI model is used for content generation.

[0371] Step 6:

[0372] The terminal establishes high-quality video calls with family members located remotely via a communication control device. The input is the user's call request, and the output is smooth audio and video. Streaming technology is used to send and receive data in real time.

[0373] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0374] This invention is an innovative system designed to support the lives of elderly users, incorporating health management, entertainment, and communication functions, aiming to realize a safe and fulfilling life. Furthermore, by incorporating an emotion engine, it is possible to provide advanced services that take into account the user's emotional state. Specific embodiments of each function are described below.

[0375] Health management and emotion recognition functions

[0376] Users collect daily health data using wearable devices. This data is transmitted to a server in real time by the device.

[0377] The server analyzes health data using artificial intelligence algorithms and processes user facial expressions and voice data acquired from the device using an emotion engine to recognize the user's emotional state. This information is then reflected in notifications and health management feedback.

[0378] Entertainment features

[0379] The server generates optimal entertainment content based on the user's emotional state and past content viewing history. This process includes data analysis by an emotion engine.

[0380] The terminal provides the user with generated content and creates an environment where the user can view or listen to selected content. The user can operate the terminal using voice commands.

[0381] Communication function

[0382] Users can initiate video calls with family and friends in remote locations through their devices.

[0383] The device sends a request to the server to establish a video call connection and ensure stable communication.

[0384] During a video call, the emotion engine assesses the user's emotional state in real time and provides advice and suggestions as needed.

[0385] For example, if a user performing a morning health check is feeling a little depressed, the emotion engine will recognize that emotion, and the server will generate and provide relaxing music or video content tailored to the user's preferences. Furthermore, if a user feels anxious during a video call, the device may immediately suggest encouraging messages or relaxation techniques. By combining these emotion engines, a system that provides even greater support for the user's overall life can be realized.

[0386] The following describes the processing flow.

[0387] Step 1:

[0388] Users collect daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is transmitted to the device via Bluetooth or Wi-Fi connection.

[0389] Step 2:

[0390] The device sends the received health data to the server. Data transmission is protected by an encryption protocol.

[0391] Step 3:

[0392] The server analyzes health data using an AI algorithm. During this process, it compares the data to reference values ​​and evaluates whether any abnormal values ​​exist.

[0393] Step 4:

[0394] The server simultaneously uses video and audio data from the terminal to recognize the user's emotional state using an emotion engine. It analyzes emotions from the user's facial expressions and tone of voice.

[0395] Step 5:

[0396] Based on the analysis of health data and the results of emotion recognition, the server generates personalized feedback and notifications. This feedback includes advice tailored to the user's current health status and emotions.

[0397] Step 6:

[0398] The device notifies the user of notifications from the server. These notifications are displayed as voice messages using a voice assistant or as visual alerts on the screen.

[0399] Step 7:

[0400] Users can check their health status based on notifications and contact a medical institution if necessary. They can also accept recommendations for relaxation methods tailored to their emotions.

[0401] Step 8:

[0402] The server analyzes the user's emotional state and content viewing history to generate optimal entertainment content (music, videos, games).

[0403] Step 9:

[0404] The device provides the user with generated entertainment content. The user can control and enjoy the content using voice commands.

[0405] Step 10:

[0406] If a user wants to make a video call with family or friends using their device, they can initiate the call with a voice command.

[0407] Step 11:

[0408] The device sends a request to the server when initiating a call. The server manages the communication network and ensures the stability of the connection.

[0409] Step 12:

[0410] During video calls, the emotion engine tracks the user's emotional state in real time and suggests comfort messages and relaxation techniques as needed.

[0411] Step 13:

[0412] After the call ends, the device uploads call-related data to a server for later analysis and service improvement.

[0413] (Example 2)

[0414] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0415] In recent years, with the advancement of an aging society, managing the health and improving the quality of life of elderly users has become a crucial issue. In particular, there is a need to provide appropriate content and communication support while simultaneously considering the user's health information and emotional state. However, conventional systems have struggled to provide these functions in an integrated manner, and have been unable to provide sufficient support for users.

[0416] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0417] In this invention, the server includes means for acquiring user health information and recognizing emotional state, means for generating content based on the user's emotional state and past viewing history using a generative AI model, and means for establishing communication between the user and a remote person, evaluating emotions in real time, and providing advice. This enables advanced health management, entertainment, and communication support tailored to the user's individual condition.

[0418] "User" refers to an individual who uses this system, and specifically to elderly people.

[0419] "Health information" refers to data about the user's physical condition, including heart rate, body temperature, and steps taken.

[0420] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices.

[0421] An "artificial intelligence processing device" refers to a system that analyzes acquired data and recognizes emotional states.

[0422] A "communication device" is a device used to provide users with notifications and feedback based on analysis results.

[0423] A "content generation device" is a system for generating entertainment content based on a user's emotional state and past viewing history.

[0424] A "content delivery device" is a device that provides generated content to users via voice commands.

[0425] A "communication control device" is a device that establishes communication between a user and a person in a remote location and monitors the user's emotional state in real time during a call.

[0426] "Emotional state" refers to information that represents the user's psychological state, and is recognized through facial expressions, tone of voice, and other factors.

[0427] A "generative AI model" refers to an artificial intelligence program that generates optimal content based on user data.

[0428] This invention provides a system that offers health management, entertainment, and communication for elderly users, and provides support that takes their emotional state into consideration. Users wear wearable devices to acquire health information such as heart rate, steps taken, and body temperature on a daily basis. These devices can be general smartwatches or fitness trackers.

[0429] User health information is sent to a device via Bluetooth. This device, a smartphone or tablet, transfers this health data to a server located in the cloud via the internet. On the server, the health data is analyzed using artificial intelligence algorithms, employing AI frameworks such as TensorFlow. Additionally, user facial expressions and voice data are collected by the device and sent to the server. This emotional data is analyzed by an emotion engine such as the Affectiva SDK.

[0430] Based on the analysis results, the server evaluates the user's health and emotional state and generates appropriate feedback. For example, if the user is feeling stressed, the server recommends relaxing music or video content and provides it through the device. In this case, external content services such as Spotify and Netflix can be used.

[0431] In the entertainment features, the server uses a generative AI model to generate optimized content based on the user's past content viewing history and emotional state. Users can select and view their desired content using voice commands. A specific example would be the prompt, "Play some relaxing music."

[0432] The communication features allow users to enjoy conversations with family and friends in remote locations through video call applications. The device manages communication for stable video calls and uses an emotion engine to assess the user's emotional state in real time during calls. If the user feels anxious, support can be provided through relaxation techniques and encouraging messages.

[0433] In this way, this system integrates health, entertainment, and communication, making it possible to provide users with advanced, emotion-based services.

[0434] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0435] Step 1:

[0436] Users acquire health information using wearable devices. This health information includes heart rate, steps taken, and body temperature. This information is transmitted via Bluetooth to devices such as smartphones and tablets. The input is real-time data from the wearable device, and the output is the reception of this data by the device.

[0437] Step 2:

[0438] The device transmits received health information to a server in the cloud via the internet. During this process, the device collects the user's facial expressions and voice data, which are also transmitted to the server. The input consists of health and emotional data stored on the device, and the output is the transmission of data to the server.

[0439] Step 3:

[0440] The server analyzes health information using AI algorithms. Specifically, this involves a process that uses machine learning models to detect anomalies and evaluate patterns. The input is the health data received in step 2, and the analyzed health status is generated as output.

[0441] Step 4:

[0442] The server processes facial and voice data using an emotion engine to recognize the user's emotional state. The emotion engine uses deep learning to calculate an emotional state label from the data. The input is the emotional data from step 2, and the output is the recognized emotional state.

[0443] Step 5:

[0444] The server generates feedback for the user based on the analysis of their health and emotional state. For example, if the user is stressed, relaxing content will be recommended. This feedback is provided to the user through data transmission to the device. The input is the analysis results from steps 3 and 4, and the output is the recommendation notification.

[0445] Step 6:

[0446] The server uses a generative AI model to generate optimal entertainment content based on the user's past viewing history and emotional state. The generated content is sent to the device and provided to the user. The input is viewing history and emotional state, and the output is optimized content.

[0447] Step 7:

[0448] The terminal provides the generated content to the user using a voice command interface. The user can select content by voice and begin playback. The input is the content data received from the server, and the output is the user starting playback of the content.

[0449] Step 8:

[0450] The user initiates a video call with a remote person using a device. The device controls the communication and sends a call request to the server. The input is the video call request, and the output is the establishment of the call connection.

[0451] Step 9:

[0452] During a video call, the device uses real-time data to assess the user's emotional state and receives appropriate responses from the server. If the user shows signs of anxiety, the device notifies them of relaxation techniques or encouraging messages. The input is emotional state data, and the output is the notified response.

[0453] (Application Example 2)

[0454] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0455] In the lives of the elderly, there are challenges in receiving appropriate support due to insufficient management of their health status and understanding of their emotions. In particular, there is a lack of daily support and entertainment that is tailored to their emotional state. Therefore, a system is needed that can provide optimal services according to each individual's emotions and health condition.

[0456] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0457] In this invention, the server includes means for acquiring user health information, means for analyzing health information and emotional state, and means for providing notifications and content based on the analysis results and emotional state. This enables the provision of individually optimized support and entertainment tailored to the user's health and emotional state.

[0458] A "user" is an individual who utilizes the system's functions and receives support for health management, entertainment, and communication.

[0459] "Health information" refers to physiological and activity data acquired through sensor devices, and is information that indicates the user's physical condition.

[0460] "Emotional state" refers to information that represents the user's psychological condition, obtained as a result of the emotion engine's analysis based on data such as the user's facial expressions and voice.

[0461] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices, etc.

[0462] An "artificial intelligence processing device" is a computer device that analyzes acquired health information and emotional states and outputs the analysis results.

[0463] A "communication device" is a device used to transmit analyzed data to a user or an external party.

[0464] A "content generation device" is a part of a system that creates entertainment content according to the user's preferences and emotional state.

[0465] A "content provision device" is a device that provides generated content to users in the form of viewing or listening.

[0466] A "communication control device" is a device that establishes communication between a user and an external person and has the function of providing emotion-based advice as needed.

[0467] The system for realizing this invention comprehensively manages and supports the user's health and emotions, and has the following configuration.

[0468] The server acquires and collects health information provided by the user via wearable devices in real time. This includes basic physiological data such as heart rate and step count. The artificial intelligence processing unit within the server analyzes emotions using this data along with the user's facial expressions and voice data. The software used incorporates machine learning algorithms, giving it the ability to accurately identify the user's health and emotional state. Facial recognition technology is commonly used for processing facial expression data. Voice analysis technology is also incorporated for emotion recognition of voice.

[0469] The analysis results are transmitted to a communication device that connects to the user's smartphone or tablet and provides feedback to the user. Through this device, the user operates using voice commands. The content generation device creates optimal entertainment content based on the acquired emotional state and past viewing history. The generated content is provided to the user's device and set up for viewing.

[0470] Furthermore, the communication control device has the functionality to enable video calls between the user and family members or caregivers located remotely. During the call, the server's emotion engine operates, providing real-time advice tailored to the user's mental state.

[0471] As a concrete example, if a user performs a routine health check and feels mild anxiety, the system will suggest music with a relaxing effect. For instance, it might send a prompt to the generative AI model such as, "If the user is analyzed as feeling depressed, suggest relaxing nature music."

[0472] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0473] Step 1:

[0474] The server acquires user health information through wearable devices. This information includes physiological data such as heart rate and step count. The server collects this data in real time and stores it in an internal database to monitor the user's health status. The input is physiological data from the wearable device, and the output is health information stored in the internal database.

[0475] Step 2:

[0476] The server receives facial expression and voice data transmitted from the user's terminal. This data is input into the emotion engine and analyzed by machine learning algorithms. The resulting emotional state reflects the user's real-time psychological state. The input is facial expression and voice data, and the output is the analyzed emotional state.

[0477] Step 3:

[0478] The server uses a generative AI model to generate prompt sentences based on health information and emotional state. These prompt sentences are generated in a format such as, "If the user is analyzed as being in a depressed mood, suggest relaxing nature music." These prompt sentences are then used as instructions for the content generation device. The input is health information and emotional state, and the output is the generated prompt sentence.

[0479] Step 4:

[0480] The terminal provides the user with entertainment content received from a content generator based on prompt messages. The user can select and control content using voice commands. Input is the generated prompt messages, and output is the provision of selectable content to the user.

[0481] Step 5:

[0482] The user initiates a video call with family or caregivers in a remote location via their device. During the call, the server monitors the user's mental state in real time and provides emotionally-based advice as needed. The inputs are the video call request and the user's emotional state, while the output is the advice provided to the user.

[0483] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0484] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0485] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0486] [Third Embodiment]

[0487] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0488] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0489] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0490] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0491] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0492] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0493] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0494] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0495] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0496] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0497] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0498] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0499] This invention is a comprehensive system designed to support the lives of elderly users, providing a combination of three main functions: health management, entertainment, and communication. Specific embodiments of each function are described below.

[0500] Health management function

[0501] Users measure their daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is immediately transmitted to the device.

[0502] The device sends the measured data to the server. The server analyzes the data using advanced artificial intelligence algorithms and detects anomalies.

[0503] If an anomaly is detected, the server analyzes the results and sends a notification to the device. Upon receiving the notification, the device alerts the user through audio or visual alerts.

[0504] Entertainment features

[0505] The server analyzes the user's past content viewing history and preferences. Based on this, it generates the most suitable music, stories, or games.

[0506] The generated content is delivered to the user through the device. Users can select and enjoy the content using voice commands or touch controls.

[0507] Communication function

[0508] Users can initiate video calls with family and friends through the device's interface.

[0509] The device requests the necessary connection from the server for the video call. The server manages the connection to provide stable communication and guarantees call quality.

[0510] During a video call, the device processes video and audio in real time, providing users with smooth and high-quality communication.

[0511] For example, in a daily health check, the user uses a wearable device to take measurements, and the device automatically sends that data to a server. If the server detects an abnormal value, the device immediately issues an alert and, in some cases, also sends a notification to family members. Also, once a week, the user can have a video call with their grandchildren who live far away, during which the device provides high-quality audio and video, and the server maintains the stability of the communication.

[0512] Thus, the present invention is implemented as a system that comprehensively provides the necessary support in daily life in response to the diverse needs of the elderly.

[0513] The following describes the processing flow.

[0514] Step 1:

[0515] Users measure their health data (e.g., blood pressure and heart rate) using a wearable device. The measured data is automatically sent to the device.

[0516] Step 2:

[0517] The device transmits received health data to the server in real time. The data is transmitted using a secure communication protocol.

[0518] Step 3:

[0519] The server analyzes the received health data using artificial intelligence algorithms. It compares it with past data to detect any anomalies.

[0520] Step 4:

[0521] If an anomaly is detected, the server generates a warning message based on the result and sends it to the terminal. This message may include a recommendation to contact a medical institution.

[0522] Step 5:

[0523] The device communicates received warning messages to the user as audio and visual notifications. These notifications are delivered in a pre-configured manner.

[0524] Step 6:

[0525] Users can recognize notifications from their devices and contact medical institutions or family members as needed. They can also review past notifications.

[0526] Step 7:

[0527] The server periodically analyzes user preferences and past usage history to generate new entertainment content. This content is optimized to meet individual needs.

[0528] Step 8:

[0529] The device provides the user with newly generated content. The user can select and enjoy the content using voice commands.

[0530] Step 9:

[0531] Users initiate video calls with their family using the device's interface.

[0532] Step 10:

[0533] The terminal sends a request to the server to establish a video call connection. The server manages the call and performs the necessary controls to maintain communication stability.

[0534] Step 11:

[0535] During a video call, the device processes video and audio data in real time to provide the user with a high-quality experience.

[0536] Step 12:

[0537] After the call ends, the device uploads the call data to the server, where it is saved as reference information for future calls.

[0538] (Example 1)

[0539] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] For diverse users, including the elderly, to live with peace of mind, real-time monitoring of their health status and prompt notification of abnormalities are necessary. Furthermore, not only health management, but also personalized information provision to enrich daily life and smooth communication with external parties are essential. However, a system that can centrally manage all of these is not yet available, making it a challenge to improve users' quality of life.

[0541] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0542] In this invention, the server includes means for utilizing a wearable device for collecting health status data, means for utilizing a computing device for analyzing the collected health data and detecting abnormalities, and means for providing a communication device for issuing warnings to the user or others based on the analysis results. This enables real-time monitoring of the user's health status, prompt notification in the event of an abnormality, and provision of appropriate information.

[0543] A "wearable device" is a device that a user can wear and that has the function of continuously collecting health data.

[0544] A "computational device" is a device that processes and analyzes collected health data to detect abnormalities, and has the function of executing programs for data analysis.

[0545] A "communication device" is a device that sends notifications or warnings to users or others based on anomalies detected by a computing device.

[0546] An "information generation device" is a device that generates individually optimized information based on the user's preferences and has the function of creating new content, etc., by taking into account the user's past data.

[0547] An "information provision device" is a device that provides generated information to a user, enabling the user to view or manipulate that information.

[0548] A "communication management device" is a device that establishes and manages smooth communication between users and others, and has the function of ensuring call connection and quality.

[0549] This invention is a system that integrates health management, information provision, and communication functions to support the user's daily life. Specific embodiments of the system are described below.

[0550] Health management function

[0551] The user wears a wearable device, such as a wristwatch, to collect health data. This device continuously monitors the user's vital signs, such as heart rate and blood pressure, and transmits the data to a terminal via Bluetooth. The terminal then transmits this data to a server via the internet. The server analyzes this data in real time using analysis software with a generative AI model, and if an abnormality is detected, it notifies the user through the terminal.

[0552] Information provision function

[0553] The server analyzes the user's past information usage history and generates information based on the user's preferences. For example, it uses an AI model to create new movies and playlists based on the user's past movie and music viewing history. The generated information is sent to the device, and the user can view or interact with it through voice commands or touch controls.

[0554] Communication function

[0555] Users can communicate with others using the terminal's interface. When initiating a video call, the terminal requests a connection to the server. The server utilizes the functions of a communication management device to provide a stable communication environment. The server dynamically manages communication quality, and the terminal processes high-quality video and audio in real time, providing users with smooth communication.

[0556] For example, if a user's heart rate is detected to be abnormal in the morning, the server immediately sends an alert via the device and also sends information to the emergency contacts designated by the user. In the afternoon, the user can watch the latest movies through the device, and in the evening, they can enjoy video calls with their family. In this way, the service is provided to make the user's life richer and safer.

[0557] Examples of prompt messages include "Generate the latest recommended movies" and "Detect abnormal heart rate." This allows for the provision of flexible services tailored to user requests.

[0558] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0559] Step 1:

[0560] The user wears a wearable device that collects vital data such as heart rate and blood pressure in real time. This data is transmitted to a terminal via Bluetooth. The input is biometric data acquired from the wearable device, and the output is biometric data transmitted to the terminal. Specifically, the device periodically measures data and transfers it to the terminal.

[0561] Step 2:

[0562] The terminal transmits biometric data received from wearable devices to a server via the internet. The input is biometric data stored on the terminal, and the output is biometric data sent to the server. Specifically, an application running on the terminal transmits the data in the background, and SSL / TLS encrypted communication is used as the transmission protocol.

[0563] Step 3:

[0564] The server analyzes the received biometric data using a generating AI model to detect anomalies. The input is the biometric data sent to the server, and the output is the analysis results of the anomaly detection. Specifically, the AI ​​model in the server processes the data using a pattern recognition algorithm and classifies it into normal and abnormal values.

[0565] Step 4:

[0566] The server sends a notification to the terminal when an anomaly is detected. The input is the analysis result of the anomaly detection, and the output is the notification data sent to the terminal. Specifically, the server uses a push notification service to send an alert to the terminal, and when the terminal receives it, it notifies the user with a pop-up or warning sound.

[0567] Step 5:

[0568] The user receives this notification and follows the corresponding instructions. Input is the notification from the device, and output is the user's response or action. Specifically, this could involve following voice guidance to check data or consult with a doctor.

[0569] Step 6:

[0570] The server uses the user's past information usage history to generate new information (e.g., music and movie lists) based on the user's preferences, optimizing it with an AI model. The input is past data usage history, and the output is an optimized information list. Specifically, a recommendation algorithm based on historical data is in operation, generating individually personalized information.

[0571] Step 7:

[0572] The generated information is sent to the device and provided for the user to view or interact with. The input is an optimized information list sent from the server, and the output is the user's viewing or interaction with the content on their device. Specifically, access to the information is made possible through features such as information display and selection options within the app.

[0573] In this way, the system enables a process that smoothly supports users' daily health management, information utilization, and communication.

[0574] (Application Example 1)

[0575] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0576] In the lives of the elderly, there is a need to appropriately monitor their health and respond quickly when abnormalities occur. Furthermore, it is necessary to provide personalized entertainment and facilitate smooth communication with family members living far away. However, conventional systems have been unable to comprehensively provide all of these functions.

[0577] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0578] In this invention, the server includes a sensor device for monitoring the user's health status, an artificial intelligence processing device for analyzing the acquired health status data, and a communication device for notifying the user or a third party based on the analysis results. This makes it possible to monitor the user's health status in real time, quickly detect abnormalities, and notify relevant parties as needed.

[0579] "Health status" refers to information indicating the user's physical parameters, including biometric data acquired by sensor devices.

[0580] A "sensor device" is a device used to monitor a user's health status and has the function of collecting biometric data such as blood pressure and heart rate.

[0581] An "artificial intelligence processing device" is a device that analyzes acquired health status data and detects abnormalities, and it uses machine learning algorithms.

[0582] A "communication device" is a device that notifies the user or a third party based on the analysis results, and has network connectivity.

[0583] A "content generation device" is a system that generates content based on user preferences, providing personalized entertainment.

[0584] A "content delivery device" is a device that delivers generated content to users, transmitting information through sight or sound.

[0585] A "communication control device" is a device for managing two-way communication with users and for establishing real-time communication with external parties.

[0586] An "information processing device" is a device that processes acquired health status data in real time, enabling high-speed data analysis.

[0587] The system for realizing this invention has the function of monitoring the user's health status in real time and immediately notifying the user if an abnormality in their health status is detected. This system includes a sensor device for acquiring health information, an artificial intelligence processing device for analyzing the data, a communication device for sending notifications based on the analysis results, a content generation device for generating content according to the user's preferences, a content provision device for providing the generated content, a communication control device for managing two-way communication, and an information processing device for processing the acquired data in real time.

[0588] The server receives health data (e.g., blood pressure, heart rate) from sensor devices such as wearable devices and analyzes the data via an artificial intelligence processing unit that runs machine learning algorithms. If this analysis detects an abnormal health condition, the server sends an alert to the user's smartphone or care staff via a communication device. The specific hardware used includes wearable devices known as fitness trackers and smartphones, while the software uses Python's TensorFlow and Firebase Cloud Messaging.

[0589] The device learns the user's preferences from past usage history and provides personalized entertainment through a content generation device that creates music and stories. A communication control device allows the user to make high-quality video calls with family members in distant locations.

[0590] As a concrete example, when a user performs a morning health check, data acquired from a wearable device is sent to a server. If an abnormality is detected, a notification is immediately sent to the user's smartphone or the caregiver's terminal. Furthermore, when making a video call with grandchildren on the weekend, a communication control device enables high-quality video and audio communication.

[0591] Examples of input prompts for a generative AI model:

[0592] "Please design a proof-of-concept system that monitors the health status of elderly individuals in real time and automatically notifies them when an abnormality is detected."

[0593] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0594] Step 1:

[0595] The user wears a wearable device to measure health data (e.g., blood pressure, heart rate). The input is biometric data from the wearable device, and the output is information about the user's health status. This data is transmitted to the terminal via Bluetooth.

[0596] Step 2:

[0597] The device sends the received health data to a cloud server. In this process, the input is health status data, and the device acts as the sender of this data to the server. An internet connection is used for data transmission.

[0598] Step 3:

[0599] The server analyzes the received health status data. The input here is data sent from the terminal, and anomalies are detected using machine learning algorithms. The output is the analysis results.

[0600] Step 4:

[0601] Based on the analysis results, the server generates a notification if an abnormality is found in the health condition. The input is the analysis results, and the notification content is output. This notification is sent to the devices of care staff and family members as needed.

[0602] Step 5:

[0603] The device understands the user's preferences and generates entertainment content. The input is the user's past viewing history, and new, customized music and stories are output. A generative AI model is used for content generation.

[0604] Step 6:

[0605] The terminal establishes high-quality video calls with family members located remotely via a communication control device. The input is the user's call request, and the output is smooth audio and video. Streaming technology is used to send and receive data in real time.

[0606] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0607] This invention is an innovative system designed to support the lives of elderly users, incorporating health management, entertainment, and communication functions, aiming to realize a safe and fulfilling life. Furthermore, by incorporating an emotion engine, it is possible to provide advanced services that take into account the user's emotional state. Specific embodiments of each function are described below.

[0608] Health management and emotion recognition functions

[0609] Users collect daily health data using wearable devices. This data is transmitted to a server in real time by the device.

[0610] The server analyzes health data using artificial intelligence algorithms and processes user facial expressions and voice data acquired from the device using an emotion engine to recognize the user's emotional state. This information is then reflected in notifications and health management feedback.

[0611] Entertainment features

[0612] The server generates optimal entertainment content based on the user's emotional state and past content viewing history. This process includes data analysis by an emotion engine.

[0613] The terminal provides the user with generated content and creates an environment where the user can view or listen to selected content. The user can operate the terminal using voice commands.

[0614] Communication function

[0615] Users can initiate video calls with family and friends in remote locations through their devices.

[0616] The device sends a request to the server to establish a video call connection and ensure stable communication.

[0617] During a video call, the emotion engine assesses the user's emotional state in real time and provides advice and suggestions as needed.

[0618] For example, if a user performing a morning health check is feeling a little depressed, the emotion engine will recognize that emotion, and the server will generate and provide relaxing music or video content tailored to the user's preferences. Furthermore, if a user feels anxious during a video call, the device may immediately suggest encouraging messages or relaxation techniques. By combining these emotion engines, a system that provides even greater support for the user's overall life can be realized.

[0619] The following describes the processing flow.

[0620] Step 1:

[0621] Users collect daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is transmitted to the device via Bluetooth or Wi-Fi connection.

[0622] Step 2:

[0623] The device sends the received health data to the server. Data transmission is protected by an encryption protocol.

[0624] Step 3:

[0625] The server analyzes health data using an AI algorithm. During this process, it compares the data to reference values ​​and evaluates whether any abnormal values ​​exist.

[0626] Step 4:

[0627] The server simultaneously uses video and audio data from the terminal to recognize the user's emotional state using an emotion engine. It analyzes emotions from the user's facial expressions and tone of voice.

[0628] Step 5:

[0629] Based on the analysis of health data and the results of emotion recognition, the server generates personalized feedback and notifications. This feedback includes advice tailored to the user's current health status and emotions.

[0630] Step 6:

[0631] The device notifies the user of notifications from the server. These notifications are displayed as voice messages using a voice assistant or as visual alerts on the screen.

[0632] Step 7:

[0633] Users can check their health status based on notifications and contact a medical institution if necessary. They can also accept recommendations for relaxation methods tailored to their emotions.

[0634] Step 8:

[0635] The server analyzes the user's emotional state and content viewing history to generate optimal entertainment content (music, videos, games).

[0636] Step 9:

[0637] The device provides the user with generated entertainment content. The user can control and enjoy the content using voice commands.

[0638] Step 10:

[0639] If a user wants to make a video call with family or friends using their device, they can initiate the call with a voice command.

[0640] Step 11:

[0641] The device sends a request to the server when initiating a call. The server manages the communication network and ensures the stability of the connection.

[0642] Step 12:

[0643] During video calls, the emotion engine tracks the user's emotional state in real time and suggests comfort messages and relaxation techniques as needed.

[0644] Step 13:

[0645] After the call ends, the device uploads call-related data to a server for later analysis and service improvement.

[0646] (Example 2)

[0647] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0648] In recent years, with the advancement of an aging society, managing the health and improving the quality of life of elderly users has become a crucial issue. In particular, there is a need to provide appropriate content and communication support while simultaneously considering the user's health information and emotional state. However, conventional systems have struggled to provide these functions in an integrated manner, and have been unable to provide sufficient support for users.

[0649] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0650] In this invention, the server includes means for acquiring user health information and recognizing emotional state, means for generating content based on the user's emotional state and past viewing history using a generative AI model, and means for establishing communication between the user and a remote person, evaluating emotions in real time, and providing advice. This enables advanced health management, entertainment, and communication support tailored to the user's individual condition.

[0651] "User" refers to an individual who uses this system, and specifically to elderly people.

[0652] "Health information" refers to data about the user's physical condition, including heart rate, body temperature, and steps taken.

[0653] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices.

[0654] An "artificial intelligence processing device" refers to a system that analyzes acquired data and recognizes emotional states.

[0655] A "communication device" is a device used to provide users with notifications and feedback based on analysis results.

[0656] A "content generation device" is a system for generating entertainment content based on a user's emotional state and past viewing history.

[0657] A "content delivery device" is a device that provides generated content to users via voice commands.

[0658] A "communication control device" is a device that establishes communication between a user and a person in a remote location and monitors the user's emotional state in real time during a call.

[0659] "Emotional state" refers to information that represents the user's psychological state, and is recognized through facial expressions, tone of voice, and other factors.

[0660] A "generative AI model" refers to an artificial intelligence program that generates optimal content based on user data.

[0661] This invention provides a system that offers health management, entertainment, and communication for elderly users, and provides support that takes their emotional state into consideration. Users wear wearable devices to acquire health information such as heart rate, steps taken, and body temperature on a daily basis. These devices can be general smartwatches or fitness trackers.

[0662] User health information is sent to a device via Bluetooth. This device, a smartphone or tablet, transfers this health data to a server located in the cloud via the internet. On the server, the health data is analyzed using artificial intelligence algorithms, employing AI frameworks such as TensorFlow. Additionally, user facial expressions and voice data are collected by the device and sent to the server. This emotional data is analyzed by an emotion engine such as the Affectiva SDK.

[0663] Based on the analysis results, the server evaluates the user's health and emotional state and generates appropriate feedback. For example, if the user is feeling stressed, the server recommends relaxing music or video content and provides it through the device. In this case, external content services such as Spotify and Netflix can be used.

[0664] In the entertainment features, the server uses a generative AI model to generate optimized content based on the user's past content viewing history and emotional state. Users can select and view their desired content using voice commands. A specific example would be the prompt, "Play some relaxing music."

[0665] The communication features allow users to enjoy conversations with family and friends in remote locations through video call applications. The device manages communication for stable video calls and uses an emotion engine to assess the user's emotional state in real time during calls. If the user feels anxious, support can be provided through relaxation techniques and encouraging messages.

[0666] In this way, this system integrates health, entertainment, and communication, making it possible to provide users with advanced, emotion-based services.

[0667] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0668] Step 1:

[0669] Users acquire health information using wearable devices. This health information includes heart rate, steps taken, and body temperature. This information is transmitted via Bluetooth to devices such as smartphones and tablets. The input is real-time data from the wearable device, and the output is the reception of this data by the device.

[0670] Step 2:

[0671] The device transmits received health information to a server in the cloud via the internet. During this process, the device collects the user's facial expressions and voice data, which are also transmitted to the server. The input consists of health and emotional data stored on the device, and the output is the transmission of data to the server.

[0672] Step 3:

[0673] The server analyzes health information using AI algorithms. Specifically, this involves a process that uses machine learning models to detect anomalies and evaluate patterns. The input is the health data received in step 2, and the analyzed health status is generated as output.

[0674] Step 4:

[0675] The server processes facial and voice data using an emotion engine to recognize the user's emotional state. The emotion engine uses deep learning to calculate an emotional state label from the data. The input is the emotional data from step 2, and the output is the recognized emotional state.

[0676] Step 5:

[0677] The server generates feedback for the user based on the analysis of their health and emotional state. For example, if the user is stressed, relaxing content will be recommended. This feedback is provided to the user through data transmission to the device. The input is the analysis results from steps 3 and 4, and the output is the recommendation notification.

[0678] Step 6:

[0679] The server uses a generative AI model to generate optimal entertainment content based on the user's past viewing history and emotional state. The generated content is sent to the device and provided to the user. The input is viewing history and emotional state, and the output is optimized content.

[0680] Step 7:

[0681] The terminal provides the generated content to the user using a voice command interface. The user can select content by voice and begin playback. The input is the content data received from the server, and the output is the user starting playback of the content.

[0682] Step 8:

[0683] The user initiates a video call with a remote person using a device. The device controls the communication and sends a call request to the server. The input is the video call request, and the output is the establishment of the call connection.

[0684] Step 9:

[0685] During a video call, the device uses real-time data to assess the user's emotional state and receives appropriate responses from the server. If the user shows signs of anxiety, the device notifies them of relaxation techniques or encouraging messages. The input is emotional state data, and the output is the notified response.

[0686] (Application Example 2)

[0687] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0688] In the lives of the elderly, there are challenges in receiving appropriate support due to insufficient management of their health status and understanding of their emotions. In particular, there is a lack of daily support and entertainment that is tailored to their emotional state. Therefore, a system is needed that can provide optimal services according to each individual's emotions and health condition.

[0689] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0690] In this invention, the server includes means for acquiring user health information, means for analyzing health information and emotional state, and means for providing notifications and content based on the analysis results and emotional state. This enables the provision of individually optimized support and entertainment tailored to the user's health and emotional state.

[0691] A "user" is an individual who utilizes the system's functions and receives support for health management, entertainment, and communication.

[0692] "Health information" refers to physiological and activity data acquired through sensor devices, and is information that indicates the user's physical condition.

[0693] "Emotional state" refers to information that represents the user's psychological condition, obtained as a result of the emotion engine's analysis based on data such as the user's facial expressions and voice.

[0694] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices, etc.

[0695] An "artificial intelligence processing device" is a computer device that analyzes acquired health information and emotional states and outputs the analysis results.

[0696] A "communication device" is a device used to transmit analyzed data to a user or an external party.

[0697] A "content generation device" is a part of a system that creates entertainment content according to the user's preferences and emotional state.

[0698] A "content provision device" is a device that provides generated content to users in the form of viewing or listening.

[0699] A "communication control device" is a device that establishes communication between a user and an external person and has the function of providing emotion-based advice as needed.

[0700] The system for realizing this invention comprehensively manages and supports the user's health and emotions, and has the following configuration.

[0701] The server acquires and collects health information provided by the user via wearable devices in real time. This includes basic physiological data such as heart rate and step count. The artificial intelligence processing unit within the server analyzes emotions using this data along with the user's facial expressions and voice data. The software used incorporates machine learning algorithms, giving it the ability to accurately identify the user's health and emotional state. Facial recognition technology is commonly used for processing facial expression data. Voice analysis technology is also incorporated for emotion recognition of voice.

[0702] The analysis results are transmitted to a communication device that connects to the user's smartphone or tablet and provides feedback to the user. Through this device, the user operates using voice commands. The content generation device creates optimal entertainment content based on the acquired emotional state and past viewing history. The generated content is provided to the user's device and set up for viewing.

[0703] Furthermore, the communication control device has the functionality to enable video calls between the user and family members or caregivers located remotely. During the call, the server's emotion engine operates, providing real-time advice tailored to the user's mental state.

[0704] As a concrete example, if a user performs a routine health check and feels mild anxiety, the system will suggest music with a relaxing effect. For instance, it might send a prompt to the generative AI model such as, "If the user is analyzed as feeling depressed, suggest relaxing nature music."

[0705] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0706] Step 1:

[0707] The server acquires user health information through wearable devices. This information includes physiological data such as heart rate and step count. The server collects this data in real time and stores it in an internal database to monitor the user's health status. The input is physiological data from the wearable device, and the output is health information stored in the internal database.

[0708] Step 2:

[0709] The server receives facial expression and voice data transmitted from the user's terminal. This data is input into the emotion engine and analyzed by machine learning algorithms. The resulting emotional state reflects the user's real-time psychological state. The input is facial expression and voice data, and the output is the analyzed emotional state.

[0710] Step 3:

[0711] The server uses a generative AI model to generate prompt sentences based on health information and emotional state. These prompt sentences are generated in a format such as, "If the user is analyzed as being in a depressed mood, suggest relaxing nature music." These prompt sentences are then used as instructions for the content generation device. The input is health information and emotional state, and the output is the generated prompt sentence.

[0712] Step 4:

[0713] The terminal provides the user with entertainment content received from a content generator based on prompt messages. The user can select and control content using voice commands. Input is the generated prompt messages, and output is the provision of selectable content to the user.

[0714] Step 5:

[0715] The user initiates a video call with family or caregivers in a remote location via their device. During the call, the server monitors the user's mental state in real time and provides emotionally-based advice as needed. The inputs are the video call request and the user's emotional state, while the output is the advice provided to the user.

[0716] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0717] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0718] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0719] [Fourth Embodiment]

[0720] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0721] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0722] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0723] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0724] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0725] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0726] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0727] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0728] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0729] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0730] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0731] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0732] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0733] This invention is a comprehensive system designed to support the lives of elderly users, providing a combination of three main functions: health management, entertainment, and communication. Specific embodiments of each function are described below.

[0734] Health management function

[0735] Users measure their daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is immediately transmitted to the device.

[0736] The device sends the measured data to the server. The server analyzes the data using advanced artificial intelligence algorithms and detects anomalies.

[0737] If an anomaly is detected, the server analyzes the results and sends a notification to the device. Upon receiving the notification, the device alerts the user through audio or visual alerts.

[0738] Entertainment features

[0739] The server analyzes the user's past content viewing history and preferences. Based on this, it generates the most suitable music, stories, or games.

[0740] The generated content is delivered to the user through the device. Users can select and enjoy the content using voice commands or touch controls.

[0741] Communication function

[0742] Users can initiate video calls with family and friends through the device's interface.

[0743] The device requests the necessary connection from the server for the video call. The server manages the connection to provide stable communication and guarantees call quality.

[0744] During a video call, the device processes video and audio in real time, providing users with smooth and high-quality communication.

[0745] For example, in a daily health check, the user uses a wearable device to take measurements, and the device automatically sends that data to a server. If the server detects an abnormal value, the device immediately issues an alert and, in some cases, also sends a notification to family members. Also, once a week, the user can have a video call with their grandchildren who live far away, during which the device provides high-quality audio and video, and the server maintains the stability of the communication.

[0746] Thus, the present invention is implemented as a system that comprehensively provides the necessary support in daily life in response to the diverse needs of the elderly.

[0747] The following describes the processing flow.

[0748] Step 1:

[0749] Users measure their health data (e.g., blood pressure and heart rate) using a wearable device. The measured data is automatically sent to the device.

[0750] Step 2:

[0751] The device transmits received health data to the server in real time. The data is transmitted using a secure communication protocol.

[0752] Step 3:

[0753] The server analyzes the received health data using artificial intelligence algorithms. It compares it with past data to detect any anomalies.

[0754] Step 4:

[0755] If an anomaly is detected, the server generates a warning message based on the result and sends it to the terminal. This message may include a recommendation to contact a medical institution.

[0756] Step 5:

[0757] The device communicates received warning messages to the user as audio and visual notifications. These notifications are delivered in a pre-configured manner.

[0758] Step 6:

[0759] Users can recognize notifications from their devices and contact medical institutions or family members as needed. They can also review past notifications.

[0760] Step 7:

[0761] The server periodically analyzes user preferences and past usage history to generate new entertainment content. This content is optimized to meet individual needs.

[0762] Step 8:

[0763] The device provides the user with newly generated content. The user can select and enjoy the content using voice commands.

[0764] Step 9:

[0765] Users initiate video calls with their family using the device's interface.

[0766] Step 10:

[0767] The terminal sends a request to the server to establish a video call connection. The server manages the call and performs the necessary controls to maintain communication stability.

[0768] Step 11:

[0769] During a video call, the device processes video and audio data in real time to provide the user with a high-quality experience.

[0770] Step 12:

[0771] After the call ends, the device uploads the call data to the server, where it is saved as reference information for future calls.

[0772] (Example 1)

[0773] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0774] For diverse users, including the elderly, to live with peace of mind, real-time monitoring of their health status and prompt notification of abnormalities are necessary. Furthermore, not only health management, but also personalized information provision to enrich daily life and smooth communication with external parties are essential. However, a system that can centrally manage all of these is not yet available, making it a challenge to improve users' quality of life.

[0775] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0776] In this invention, the server includes means for utilizing a wearable device for collecting health status data, means for utilizing a computing device for analyzing the collected health data and detecting abnormalities, and means for providing a communication device for issuing warnings to the user or others based on the analysis results. This enables real-time monitoring of the user's health status, prompt notification in the event of an abnormality, and provision of appropriate information.

[0777] A "wearable device" is a device that a user can wear and that has the function of continuously collecting health data.

[0778] A "computational device" is a device that processes and analyzes collected health data to detect abnormalities, and has the function of executing programs for data analysis.

[0779] A "communication device" is a device that sends notifications or warnings to users or others based on anomalies detected by a computing device.

[0780] An "information generation device" is a device that generates individually optimized information based on the user's preferences and has the function of creating new content, etc., by taking into account the user's past data.

[0781] An "information provision device" is a device that provides generated information to a user, enabling the user to view or manipulate that information.

[0782] A "communication management device" is a device that establishes and manages smooth communication between users and others, and has the function of ensuring call connection and quality.

[0783] This invention is a system that integrates health management, information provision, and communication functions to support the user's daily life. Specific embodiments of the system are described below.

[0784] Health management function

[0785] The user wears a wearable device, such as a wristwatch, to collect health data. This device continuously monitors the user's vital signs, such as heart rate and blood pressure, and transmits the data to a terminal via Bluetooth. The terminal then transmits this data to a server via the internet. The server analyzes this data in real time using analysis software with a generative AI model, and if an abnormality is detected, it notifies the user through the terminal.

[0786] Information provision function

[0787] The server analyzes the user's past information usage history and generates information based on the user's preferences. For example, it uses an AI model to create new movies and playlists based on the user's past movie and music viewing history. The generated information is sent to the device, and the user can view or interact with it through voice commands or touch controls.

[0788] Communication function

[0789] Users can communicate with others using the terminal's interface. When initiating a video call, the terminal requests a connection to the server. The server utilizes the functions of a communication management device to provide a stable communication environment. The server dynamically manages communication quality, and the terminal processes high-quality video and audio in real time, providing users with smooth communication.

[0790] For example, if a user's heart rate is detected to be abnormal in the morning, the server immediately sends an alert via the device and also sends information to the emergency contacts designated by the user. In the afternoon, the user can watch the latest movies through the device, and in the evening, they can enjoy video calls with their family. In this way, the service is provided to make the user's life richer and safer.

[0791] Examples of prompt messages include "Generate the latest recommended movies" and "Detect abnormal heart rate." This allows for the provision of flexible services tailored to user requests.

[0792] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0793] Step 1:

[0794] The user wears a wearable device that collects vital data such as heart rate and blood pressure in real time. This data is transmitted to a terminal via Bluetooth. The input is biometric data acquired from the wearable device, and the output is biometric data transmitted to the terminal. Specifically, the device periodically measures data and transfers it to the terminal.

[0795] Step 2:

[0796] The terminal transmits biometric data received from wearable devices to a server via the internet. The input is biometric data stored on the terminal, and the output is biometric data sent to the server. Specifically, an application running on the terminal transmits the data in the background, and SSL / TLS encrypted communication is used as the transmission protocol.

[0797] Step 3:

[0798] The server analyzes the received biometric data using a generating AI model to detect anomalies. The input is the biometric data sent to the server, and the output is the analysis results of the anomaly detection. Specifically, the AI ​​model in the server processes the data using a pattern recognition algorithm and classifies it into normal and abnormal values.

[0799] Step 4:

[0800] The server sends a notification to the terminal when an anomaly is detected. The input is the analysis result of the anomaly detection, and the output is the notification data sent to the terminal. Specifically, the server uses a push notification service to send an alert to the terminal, and when the terminal receives it, it notifies the user with a pop-up or warning sound.

[0801] Step 5:

[0802] The user receives this notification and follows the corresponding instructions. Input is the notification from the device, and output is the user's response or action. Specifically, this could involve following voice guidance to check data or consult with a doctor.

[0803] Step 6:

[0804] The server uses the user's past information usage history to generate new information (e.g., music and movie lists) based on the user's preferences, optimizing it with an AI model. The input is past data usage history, and the output is an optimized information list. Specifically, a recommendation algorithm based on historical data is in operation, generating individually personalized information.

[0805] Step 7:

[0806] The generated information is sent to the device and provided for the user to view or interact with. The input is an optimized information list sent from the server, and the output is the user's viewing or interaction with the content on their device. Specifically, access to the information is made possible through features such as information display and selection options within the app.

[0807] In this way, the system enables a process that smoothly supports users' daily health management, information utilization, and communication.

[0808] (Application Example 1)

[0809] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0810] In the lives of the elderly, there is a need to appropriately monitor their health and respond quickly when abnormalities occur. Furthermore, it is necessary to provide personalized entertainment and facilitate smooth communication with family members living far away. However, conventional systems have been unable to comprehensively provide all of these functions.

[0811] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0812] In this invention, the server includes a sensor device for monitoring the user's health status, an artificial intelligence processing device for analyzing the acquired health status data, and a communication device for notifying the user or a third party based on the analysis results. This makes it possible to monitor the user's health status in real time, quickly detect abnormalities, and notify relevant parties as needed.

[0813] "Health status" refers to information indicating the user's physical parameters, including biometric data acquired by sensor devices.

[0814] A "sensor device" is a device used to monitor a user's health status and has the function of collecting biometric data such as blood pressure and heart rate.

[0815] An "artificial intelligence processing device" is a device that analyzes acquired health status data and detects abnormalities, and it uses machine learning algorithms.

[0816] A "communication device" is a device that notifies the user or a third party based on the analysis results, and has network connectivity.

[0817] A "content generation device" is a system that generates content based on user preferences, providing personalized entertainment.

[0818] A "content delivery device" is a device that delivers generated content to users, transmitting information through sight or sound.

[0819] A "communication control device" is a device for managing two-way communication with users and for establishing real-time communication with external parties.

[0820] An "information processing device" is a device that processes acquired health status data in real time, enabling high-speed data analysis.

[0821] The system for realizing this invention has the function of monitoring the user's health status in real time and immediately notifying the user if an abnormality in their health status is detected. This system includes a sensor device for acquiring health information, an artificial intelligence processing device for analyzing the data, a communication device for sending notifications based on the analysis results, a content generation device for generating content according to the user's preferences, a content provision device for providing the generated content, a communication control device for managing two-way communication, and an information processing device for processing the acquired data in real time.

[0822] The server receives health data (e.g., blood pressure, heart rate) from sensor devices such as wearable devices and analyzes the data via an artificial intelligence processing unit that runs machine learning algorithms. If this analysis detects an abnormal health condition, the server sends an alert to the user's smartphone or care staff via a communication device. The specific hardware used includes wearable devices known as fitness trackers and smartphones, while the software uses Python's TensorFlow and Firebase Cloud Messaging.

[0823] The device learns the user's preferences from past usage history and provides personalized entertainment through a content generation device that creates music and stories. A communication control device allows the user to make high-quality video calls with family members in distant locations.

[0824] As a concrete example, when a user performs a morning health check, data acquired from a wearable device is sent to a server. If an abnormality is detected, a notification is immediately sent to the user's smartphone or the caregiver's terminal. Furthermore, when making a video call with grandchildren on the weekend, a communication control device enables high-quality video and audio communication.

[0825] Examples of input prompts for a generative AI model:

[0826] "Please design a proof-of-concept system that monitors the health status of elderly individuals in real time and automatically notifies them when an abnormality is detected."

[0827] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0828] Step 1:

[0829] The user wears a wearable device to measure health data (e.g., blood pressure, heart rate). The input is biometric data from the wearable device, and the output is information about the user's health status. This data is transmitted to the terminal via Bluetooth.

[0830] Step 2:

[0831] The device sends the received health data to a cloud server. In this process, the input is health status data, and the device acts as the sender of this data to the server. An internet connection is used for data transmission.

[0832] Step 3:

[0833] The server analyzes the received health status data. The input here is data sent from the terminal, and anomalies are detected using machine learning algorithms. The output is the analysis results.

[0834] Step 4:

[0835] Based on the analysis results, the server generates a notification if an abnormality is found in the health condition. The input is the analysis results, and the notification content is output. This notification is sent to the devices of care staff and family members as needed.

[0836] Step 5:

[0837] The device understands the user's preferences and generates entertainment content. The input is the user's past viewing history, and new, customized music and stories are output. A generative AI model is used for content generation.

[0838] Step 6:

[0839] The terminal establishes high-quality video calls with family members located remotely via a communication control device. The input is the user's call request, and the output is smooth audio and video. Streaming technology is used to send and receive data in real time.

[0840] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0841] This invention is an innovative system designed to support the lives of elderly users, incorporating health management, entertainment, and communication functions, aiming to realize a safe and fulfilling life. Furthermore, by incorporating an emotion engine, it is possible to provide advanced services that take into account the user's emotional state. Specific embodiments of each function are described below.

[0842] Health management and emotion recognition functions

[0843] Users collect daily health data using wearable devices. This data is transmitted to a server in real time by the device.

[0844] The server analyzes health data using artificial intelligence algorithms and processes user facial expressions and voice data acquired from the device using an emotion engine to recognize the user's emotional state. This information is then reflected in notifications and health management feedback.

[0845] Entertainment features

[0846] The server generates optimal entertainment content based on the user's emotional state and past content viewing history. This process includes data analysis by an emotion engine.

[0847] The terminal provides the user with generated content and creates an environment where the user can view or listen to selected content. The user can operate the terminal using voice commands.

[0848] Communication function

[0849] Users can initiate video calls with family and friends in remote locations through their devices.

[0850] The device sends a request to the server to establish a video call connection and ensure stable communication.

[0851] During a video call, the emotion engine assesses the user's emotional state in real time and provides advice and suggestions as needed.

[0852] For example, if a user performing a morning health check is feeling a little depressed, the emotion engine will recognize that emotion, and the server will generate and provide relaxing music or video content tailored to the user's preferences. Furthermore, if a user feels anxious during a video call, the device may immediately suggest encouraging messages or relaxation techniques. By combining these emotion engines, a system that provides even greater support for the user's overall life can be realized.

[0853] The following describes the processing flow.

[0854] Step 1:

[0855] Users collect daily health data (e.g., blood pressure, heart rate) using wearable devices. This data is transmitted to the device via Bluetooth or Wi-Fi connection.

[0856] Step 2:

[0857] The device sends the received health data to the server. Data transmission is protected by an encryption protocol.

[0858] Step 3:

[0859] The server analyzes health data using an AI algorithm. During this process, it compares the data to reference values ​​and evaluates whether any abnormal values ​​exist.

[0860] Step 4:

[0861] The server simultaneously uses video and audio data from the terminal to recognize the user's emotional state using an emotion engine. It analyzes emotions from the user's facial expressions and tone of voice.

[0862] Step 5:

[0863] Based on the analysis of health data and the results of emotion recognition, the server generates personalized feedback and notifications. This feedback includes advice tailored to the user's current health status and emotions.

[0864] Step 6:

[0865] The device notifies the user of notifications from the server. These notifications are displayed as voice messages using a voice assistant or as visual alerts on the screen.

[0866] Step 7:

[0867] Users can check their health status based on notifications and contact a medical institution if necessary. They can also accept recommendations for relaxation methods tailored to their emotions.

[0868] Step 8:

[0869] The server analyzes the user's emotional state and content viewing history to generate optimal entertainment content (music, videos, games).

[0870] Step 9:

[0871] The device provides the user with generated entertainment content. The user can control and enjoy the content using voice commands.

[0872] Step 10:

[0873] If a user wants to make a video call with family or friends using their device, they can initiate the call with a voice command.

[0874] Step 11:

[0875] The device sends a request to the server when initiating a call. The server manages the communication network and ensures the stability of the connection.

[0876] Step 12:

[0877] During video calls, the emotion engine tracks the user's emotional state in real time and suggests comfort messages and relaxation techniques as needed.

[0878] Step 13:

[0879] After the call ends, the device uploads call-related data to a server for later analysis and service improvement.

[0880] (Example 2)

[0881] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0882] In recent years, with the advancement of an aging society, managing the health and improving the quality of life of elderly users has become a crucial issue. In particular, there is a need to provide appropriate content and communication support while simultaneously considering the user's health information and emotional state. However, conventional systems have struggled to provide these functions in an integrated manner, and have been unable to provide sufficient support for users.

[0883] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0884] In this invention, the server includes means for acquiring user health information and recognizing emotional state, means for generating content based on the user's emotional state and past viewing history using a generative AI model, and means for establishing communication between the user and a remote person, evaluating emotions in real time, and providing advice. This enables advanced health management, entertainment, and communication support tailored to the user's individual condition.

[0885] "User" refers to an individual who uses this system, and specifically to elderly people.

[0886] "Health information" refers to data about the user's physical condition, including heart rate, body temperature, and steps taken.

[0887] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices.

[0888] An "artificial intelligence processing device" refers to a system that analyzes acquired data and recognizes emotional states.

[0889] A "communication device" is a device used to provide users with notifications and feedback based on analysis results.

[0890] A "content generation device" is a system for generating entertainment content based on a user's emotional state and past viewing history.

[0891] A "content delivery device" is a device that provides generated content to users via voice commands.

[0892] A "communication control device" is a device that establishes communication between a user and a person in a remote location and monitors the user's emotional state in real time during a call.

[0893] "Emotional state" refers to information that represents the user's psychological state, and is recognized through facial expressions, tone of voice, and other factors.

[0894] A "generative AI model" refers to an artificial intelligence program that generates optimal content based on user data.

[0895] This invention provides a system that offers health management, entertainment, and communication for elderly users, and provides support that takes their emotional state into consideration. Users wear wearable devices to acquire health information such as heart rate, steps taken, and body temperature on a daily basis. These devices can be general smartwatches or fitness trackers.

[0896] User health information is sent to a device via Bluetooth. This device, a smartphone or tablet, transfers this health data to a server located in the cloud via the internet. On the server, the health data is analyzed using artificial intelligence algorithms, employing AI frameworks such as TensorFlow. Additionally, user facial expressions and voice data are collected by the device and sent to the server. This emotional data is analyzed by an emotion engine such as the Affectiva SDK.

[0897] Based on the analysis results, the server evaluates the user's health and emotional state and generates appropriate feedback. For example, if the user is feeling stressed, the server recommends relaxing music or video content and provides it through the device. In this case, external content services such as Spotify and Netflix can be used.

[0898] In the entertainment features, the server uses a generative AI model to generate optimized content based on the user's past content viewing history and emotional state. Users can select and view their desired content using voice commands. A specific example would be the prompt, "Play some relaxing music."

[0899] The communication features allow users to enjoy conversations with family and friends in remote locations through video call applications. The device manages communication for stable video calls and uses an emotion engine to assess the user's emotional state in real time during calls. If the user feels anxious, support can be provided through relaxation techniques and encouraging messages.

[0900] In this way, this system integrates health, entertainment, and communication, making it possible to provide users with advanced, emotion-based services.

[0901] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0902] Step 1:

[0903] Users acquire health information using wearable devices. This health information includes heart rate, steps taken, and body temperature. This information is transmitted via Bluetooth to devices such as smartphones and tablets. The input is real-time data from the wearable device, and the output is the reception of this data by the device.

[0904] Step 2:

[0905] The device transmits received health information to a server in the cloud via the internet. During this process, the device collects the user's facial expressions and voice data, which are also transmitted to the server. The input consists of health and emotional data stored on the device, and the output is the transmission of data to the server.

[0906] Step 3:

[0907] The server analyzes health information using AI algorithms. Specifically, this involves a process that uses machine learning models to detect anomalies and evaluate patterns. The input is the health data received in step 2, and the analyzed health status is generated as output.

[0908] Step 4:

[0909] The server processes facial and voice data using an emotion engine to recognize the user's emotional state. The emotion engine uses deep learning to calculate an emotional state label from the data. The input is the emotional data from step 2, and the output is the recognized emotional state.

[0910] Step 5:

[0911] The server generates feedback for the user based on the analysis of their health and emotional state. For example, if the user is stressed, relaxing content will be recommended. This feedback is provided to the user through data transmission to the device. The input is the analysis results from steps 3 and 4, and the output is the recommendation notification.

[0912] Step 6:

[0913] The server uses a generative AI model to generate optimal entertainment content based on the user's past viewing history and emotional state. The generated content is sent to the device and provided to the user. The input is viewing history and emotional state, and the output is optimized content.

[0914] Step 7:

[0915] The terminal provides the generated content to the user using a voice command interface. The user can select content by voice and begin playback. The input is the content data received from the server, and the output is the user starting playback of the content.

[0916] Step 8:

[0917] The user initiates a video call with a remote person using a device. The device controls the communication and sends a call request to the server. The input is the video call request, and the output is the establishment of the call connection.

[0918] Step 9:

[0919] During a video call, the device uses real-time data to assess the user's emotional state and receives appropriate responses from the server. If the user shows signs of anxiety, the device notifies them of relaxation techniques or encouraging messages. The input is emotional state data, and the output is the notified response.

[0920] (Application Example 2)

[0921] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0922] In the lives of the elderly, there are challenges in receiving appropriate support due to insufficient management of their health status and understanding of their emotions. In particular, there is a lack of daily support and entertainment that is tailored to their emotional state. Therefore, a system is needed that can provide optimal services according to each individual's emotions and health condition.

[0923] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0924] In this invention, the server includes means for acquiring user health information, means for analyzing health information and emotional state, and means for providing notifications and content based on the analysis results and emotional state. This enables the provision of individually optimized support and entertainment tailored to the user's health and emotional state.

[0925] A "user" is an individual who utilizes the system's functions and receives support for health management, entertainment, and communication.

[0926] "Health information" refers to physiological and activity data acquired through sensor devices, and is information that indicates the user's physical condition.

[0927] "Emotional state" refers to information that represents the user's psychological condition, obtained as a result of the emotion engine's analysis based on data such as the user's facial expressions and voice.

[0928] A "sensor device" is a device used to acquire a user's health information, and includes wearable devices, etc.

[0929] An "artificial intelligence processing device" is a computer device that analyzes acquired health information and emotional states and outputs the analysis results.

[0930] A "communication device" is a device used to transmit analyzed data to a user or an external party.

[0931] A "content generation device" is a part of a system that creates entertainment content according to the user's preferences and emotional state.

[0932] A "content provision device" is a device that provides generated content to users in the form of viewing or listening.

[0933] A "communication control device" is a device that establishes communication between a user and an external person and has the function of providing emotion-based advice as needed.

[0934] The system for realizing this invention comprehensively manages and supports the user's health and emotions, and has the following configuration.

[0935] The server acquires and collects health information provided by the user via wearable devices in real time. This includes basic physiological data such as heart rate and step count. The artificial intelligence processing unit within the server analyzes emotions using this data along with the user's facial expressions and voice data. The software used incorporates machine learning algorithms, giving it the ability to accurately identify the user's health and emotional state. Facial recognition technology is commonly used for processing facial expression data. Voice analysis technology is also incorporated for emotion recognition of voice.

[0936] The analysis results are transmitted to a communication device that connects to the user's smartphone or tablet and provides feedback to the user. Through this device, the user operates using voice commands. The content generation device creates optimal entertainment content based on the acquired emotional state and past viewing history. The generated content is provided to the user's device and set up for viewing.

[0937] Furthermore, the communication control device has the functionality to enable video calls between the user and family members or caregivers located remotely. During the call, the server's emotion engine operates, providing real-time advice tailored to the user's mental state.

[0938] As a concrete example, if a user performs a routine health check and feels mild anxiety, the system will suggest music with a relaxing effect. For instance, it might send a prompt to the generative AI model such as, "If the user is analyzed as feeling depressed, suggest relaxing nature music."

[0939] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0940] Step 1:

[0941] The server acquires user health information through wearable devices. This information includes physiological data such as heart rate and step count. The server collects this data in real time and stores it in an internal database to monitor the user's health status. The input is physiological data from the wearable device, and the output is health information stored in the internal database.

[0942] Step 2:

[0943] The server receives facial expression and voice data transmitted from the user's terminal. This data is input into the emotion engine and analyzed by machine learning algorithms. The resulting emotional state reflects the user's real-time psychological state. The input is facial expression and voice data, and the output is the analyzed emotional state.

[0944] Step 3:

[0945] The server uses a generative AI model to generate prompt sentences based on health information and emotional state. These prompt sentences are generated in a format such as, "If the user is analyzed as being in a depressed mood, suggest relaxing nature music." These prompt sentences are then used as instructions for the content generation device. The input is health information and emotional state, and the output is the generated prompt sentence.

[0946] Step 4:

[0947] The terminal provides the user with entertainment content received from a content generator based on prompt messages. The user can select and control content using voice commands. Input is the generated prompt messages, and output is the provision of selectable content to the user.

[0948] Step 5:

[0949] The user initiates a video call with family or caregivers in a remote location via their device. During the call, the server monitors the user's mental state in real time and provides emotionally-based advice as needed. The inputs are the video call request and the user's emotional state, while the output is the advice provided to the user.

[0950] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0951] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0952] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0953] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0954] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0955] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0956] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0957] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0958] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0959] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0960] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0961] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0962] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0963] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0964] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0965] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0966] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0967] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0968] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0969] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0970] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0971] The following is further disclosed regarding the embodiments described above.

[0972] (Claim 1)

[0973] A sensor device for acquiring user health information,

[0974] An artificial intelligence processing unit for analyzing acquired health information,

[0975] A communication device that notifies the user based on the analysis results,

[0976] A content generation device that generates content according to the user's preferences,

[0977] A content provision device for providing the generated content,

[0978] A communication control device that establishes communication between a user and an external person,

[0979] A system that includes this.

[0980] (Claim 2)

[0981] The system according to claim 1, which detects anomalies in a user's health information and sends an alert to the user or an external person based on the anomaly.

[0982] (Claim 3)

[0983] The system according to claim 1, which optimizes the content to be provided next time based on the user's past content viewing history.

[0984] "Example 1"

[0985] (Claim 1)

[0986] A wearable device for collecting health status data,

[0987] A computing device for analyzing the collected health data,

[0988] A communication device that warns users based on the analysis results,

[0989] An information generation device that generates information tailored to the user's preferences,

[0990] An information providing device for providing generated information,

[0991] A communication management device that establishes communication between the user and others,

[0992] A system that includes this.

[0993] (Claim 2)

[0994] The system according to claim 1, which, when an abnormality is detected in health status data, sends a warning to the user or another person based on that abnormality.

[0995] (Claim 3)

[0996] The system according to claim 1, which optimizes the information provided to the user the next time based on the user's past information usage history.

[0997] "Application Example 1"

[0998] (Claim 1)

[0999] A sensor device for monitoring the user's health status,

[1000] An artificial intelligence processing unit for analyzing acquired health status data,

[1001] A communication device that notifies the user or a third party based on the analysis results,

[1002] A content generation device that generates content based on user preferences,

[1003] A content provision device for providing generated content,

[1004] A communication control device that manages bidirectional communication with the user,

[1005] An information processing device that processes acquired health status data in real time,

[1006] A system that includes this.

[1007] (Claim 2)

[1008] The system according to claim 1, which detects anomalies based on the user's health status data and sends an alert to the user or a third party based on the anomaly.

[1009] (Claim 3)

[1010] The system according to claim 1, which optimizes the content to be provided next time based on the user's past content consumption history.

[1011] "Example 2 of combining an emotion engine"

[1012] (Claim 1)

[1013] A sensor device for acquiring user health information,

[1014] An artificial intelligence processing unit for analyzing acquired health information and recognizing emotional states,

[1015] A communication device that provides feedback and notifications according to the user's emotional state,

[1016] A content generation device that generates content based on the user's emotional state and past viewing history,

[1017] A content provision device for providing generated content via voice commands,

[1018] A communication control device that establishes communication between a user and a person in a remote location and evaluates the emotional state in real time during communication,

[1019] A system that includes this.

[1020] (Claim 2)

[1021] The system according to claim 1, which detects abnormalities in a user's health information and emotional state, and sends an alert to the user or an external person based on the abnormality.

[1022] (Claim 3)

[1023] The system according to claim 1, which optimizes the content to be provided next time based on the user's past content viewing history and current emotional state.

[1024] "Application example 2 when combining with an emotional engine"

[1025] (Claim 1)

[1026] A sensor device for acquiring user health information,

[1027] An artificial intelligence processing unit for analyzing acquired health information,

[1028] A communication device that notifies the user based on the analysis results and the user's emotional state,

[1029] A content generation device that generates content according to the user's preferences and emotional state,

[1030] A content provision device for providing generated content to the user and for controlling the selected content,

[1031] A communication control device that establishes easy communication between a user and an external person and provides advice based on emotional state,

[1032] A system that includes this.

[1033] (Claim 2)

[1034] The system according to claim 1, which detects abnormalities and emotional states in a user's health information and sends an alert to the user or an external person based on those abnormalities and emotional states.

[1035] (Claim 3)

[1036] The system according to claim 1, which optimizes the content to be provided next time based on the user's past content viewing history and emotional state. [Explanation of Symbols]

[1037] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A sensor device for monitoring the user's health status, An artificial intelligence processing unit for analyzing acquired health status data, A communication device that notifies the user or a third party based on the analysis results, A content generation device that generates content based on user preferences, A content provision device for providing generated content, A communication control device that manages bidirectional communication with the user, An information processing device that processes acquired health status data in real time, A system that includes this.

2. The system according to claim 1, which detects anomalies based on the user's health status data and sends an alert to the user or a third party based on the anomaly.

3. The system according to claim 1, which optimizes the content to be provided next time based on the user's past content consumption history.

Images