System

Abstract

A system is provided.SOLUTION: A system comprising: means for monitoring a health condition of a user; means for generating health advice based on the health condition; and means for detecting and responding to an emergency situation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] As the aging society increases, the need for health management and nursing care support for the elderly increases. However, existing systems are unable to adequately manage health at home or respond to emergencies. Continuous health monitoring and appropriate responses for patients with chronic diseases are also not fully realized. In particular, there is a lack of systems suitable for elderly people living alone or patients receiving treatment at home, making it difficult to provide a safe and secure living environment. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for monitoring a user's health status, a means for generating health advice based on the health status, and a means for detecting and responding to emergencies. Specifically, the system includes a means for collecting text data, audio data, image data, and video data, a means for analyzing the data to evaluate the user's health status, and a means for providing appropriate health advice based on the evaluation results. The system also includes a means for collecting biometric data from the user using a sensor device, analyzing the biometric data to detect emergencies early, and a means for promptly notifying a medical institution when an emergency occurs. In this way, the present invention provides an environment where elderly people and patients with chronic diseases can live safely and securely at home, thereby improving the efficiency and quality of home medical care and nursing care.

[0006] "Users" refer to elderly people and patients with chronic diseases who use the system.

[0007] "Health monitoring means" refers to the part of the system that continuously monitors and collects data about the user's daily health.

[0008] "Means for generating health advice" refers to the part of the system that generates helpful guidance and suggestions for maintaining and improving health for the user based on the collected data.

[0009] "Means for detecting and responding to emergencies" refers to the part of the system that immediately detects when a user is in danger and takes appropriate action.

[0010] "Text data" refers to character information obtained from a user.

[0011] "Audio data" refers to audio information such as user speech and environmental sounds.

[0012] "Image data" refers to visual information obtained through a visual sensor such as a camera.

[0013] "Moving image data" refers to a series of video information in which consecutive image data are bound together in time.

[0014] "Sensor device" refers to a device used to measure and collect a user's biometric data.

[0015] "Biometric data" refers to physical information such as a user's heart rate and number of steps.

[0016] "Means for analysis" refers to the part of the system that analyzes the collected data and extracts meaningful information.

[0017] "Means for assessing" refers to the part of the system that assesses the health status of the user based on the analyzed data.

[0018] "Means of reporting" refers to the part of the system that promptly contacts medical institutions and registered contacts in the event of an emergency. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and those with chronic diseases. The system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[0041] 1. Initial Setup and Device Placement

[0042] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed so as to cover the user's living space.

[0043] 2. Data collection and transmission

[0044] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone capturing the user's speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. The collected data is periodically sent to a server.

[0045] 3. Data Reception and Preprocessing

[0046] Server: Receives data sent from the device and performs preprocessing. For example, resizing image data or removing noise from audio data. This preprocessing ensures that subsequent analysis can be performed efficiently.

[0047] 4. Data Analysis

[0048] Server: The preprocessed data is analyzed using an AI model. Specifically, image data is used to recognize the user's facial expressions and movements, voice data is used to analyze speech content and tone, and biometric data is used to detect abnormalities.

[0049] 5. Health assessment and advice

[0050] Server: Integrates the analysis results and evaluates the user's health condition. Based on this evaluation, it generates daily health advice. For example, if the user's activity level is low, it generates advice such as "exercise a little more" and sends it to the device.

[0051] Device: Receives advice from the server and notifies the user by voice or text. For example, the device may say, "It would be good to take a short walk today."

[0052] 6. Emergency Response

[0053] Server: If an emergency situation is detected during the analysis of the user's data, for example if the user falls, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[0054] The device also has a function to directly notify the user of this emergency response, for example, by issuing a voice message such as "A fall has been detected. We will call for help, so please don't worry."

[0055] Specific examples

[0056] Daily Monitoring

[0057] Device: The camera captures the living room and captures the user watching TV, the microphone captures ambient sound, and the sensor checks the user's heart rate.

[0058] Server: This data is sent to the server and analyzed by the AI ​​model. If the user is confirmed to be relaxed and there are no particular abnormalities, the monitoring results are stored in a database.

[0059] Fall Detection and Emergency Response

[0060] Device: The camera captures the user falling, and the fall data is sent to the server.

[0061] Server: The server analyzes the camera image to detect the user's fall and determines that it is an emergency. The server immediately notifies registered family members and the nearest medical institution, providing details along with the user's location.

[0062] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0063] As described above, the present invention is a system for enhancing the health and safety of elderly people and users with chronic diseases and for streamlining home medical care and nursing. This system consistently supports users' lives, from daily monitoring to emergency response.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and surrounding sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[0067] Step 2:

[0068] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is stored as segments every minute.

[0069] Step 3:

[0070] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. For example, it resizes image data to a size that is easy to analyze, and removes noise from audio data.

[0071] Step 4:

[0072] Terminal: Sends pre-processed data to the server, optimizing communication delays so that data is transferred in real time.

[0073] Step 5:

[0074] Server: Receives data sent from the device and stores it in a database. It checks the integrity of the received data and requests retransmission if necessary.

[0075] Step 6:

[0076] Server: Inputs the received data into the AI ​​model and starts the analysis process. For example, inputs image data into a facial recognition algorithm to detect the user's facial expressions and movements.

[0077] Step 7:

[0078] Server: Evaluates the user's health condition based on the analysis results. For example, if the heart rate is within the normal range and the facial expression is cheerful, it determines that the user is under little stress.

[0079] Step 8:

[0080] Server: Generates appropriate health advice based on the health assessment results. For example, "The weather is nice today, so I recommend taking a 15-minute walk."

[0081] Step 9:

[0082] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[0083] Step 10:

[0084] Terminal: Notifies the user of health advice received from the server. For example, a message such as "Let's take a walk today" is spoken through a speaker.

[0085] Step 11:

[0086] Server: If an emergency situation is detected during analysis, the server immediately initiates an emergency response protocol. For example, if a user falls, the server will detect the fall based on image analysis and determine that it is an emergency.

[0087] Step 12:

[0088] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[0089] Step 13:

[0090] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[0091] Step 14:

[0092] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Health management for the elderly and those with chronic diseases has become an important issue in modern society. In particular, monitoring of daily life and early detection of emergencies are required, but few systems can efficiently achieve these. In addition, improving the quality of collected data and analyzing it effectively are also challenges.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for monitoring a user's health status, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for preprocessing data, means for analyzing the monitoring data using an AI model, means for performing daily health management based on the analysis results, and means for responding to emergencies based on the analysis results. This enables monitoring of the daily lives of elderly people and people with chronic diseases, and enables rapid detection and response to emergencies. Furthermore, data preprocessing and AI analysis improve the quality of collected data, enabling more accurate health management and emergency responses.

[0098] "Users" refer to elderly people and individuals with chronic illnesses who use the system.

[0099] "Health monitoring" refers to the continuous collection and recording of a user's movements, facial expressions, speech, environmental sounds, and biometric data during their daily life using cameras, microphones, and various sensors.

[0100] "Generating health advice" refers to automatically creating appropriate advice for improving the user's health in their daily life based on collected data and the results of its analysis.

[0101] "Emergency detection" means detecting a user's fall or abnormal biometric data from the analyzed data and recognizing a situation that requires immediate action.

[0102] "Implementing emergency response" refers to the process of promptly notifying registered family members and medical institutions and requesting support in the event of a detected emergency.

[0103] "Data preprocessing" refers to the process of resizing, removing noise, standardizing the format, etc. of collected data in order to analyze it efficiently.

[0104] "Analyzing with an AI model" refers to using artificial intelligence to analyze various collected data and process it to understand and predict the user's health condition and behavioral patterns.

[0105] "Daily health management" refers to evaluating the user's health condition based on analyzed data and providing necessary advice and precautions.

[0106] "Emergency response" refers to a series of actions to provide prompt and appropriate notification and assistance when an abnormality or emergency situation is detected in a user.

[0107] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and users with chronic diseases. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions. Specific embodiments for implementing this system are described below.

[0108] Initial Setup and Device Deployment

[0109] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[0110] Data collection and transmission

[0111] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded. The collected data is periodically sent to a server.

[0112] Data reception and preprocessing

[0113] Server: Receives data sent from the device and performs preprocessing. Specifically, image data is resized to make it more efficient to handle, for example, reducing a high-resolution image of 1920x1080 to 640x360. Audio data is denoised to filter out background noise. Sensor data is sanitized and formatted to eliminate outliers.

[0114] Data analysis

[0115] Server: The preprocessed data is analyzed using an AI model. For image data, the AI ​​model recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[0116] Health assessment and advice

[0117] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[0118] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[0119] Emergency response

[0120] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[0121] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0122] Specific examples

[0123] Daily Monitoring

[0124] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and records the user's voice "enjoying the news program." The sensor checks the user's heart rate and records their relaxed state.

[0125] Server: This data is sent to the server and analyzed by the AI ​​model. It confirms that the user is relaxed, and if there are no particular abnormalities, it stores the monitoring results in a database. For example, it records "13:45, living room, normal."

[0126] Fall Detection and Emergency Response

[0127] Device: A camera captures a user falling in the living room and sends the fall data to the server.

[0128] Server: Image analysis confirms that the user has fallen and determines that it is an emergency. The server then notifies registered family members and the nearest medical institution, saying, "The user has fallen in the living room. Please provide emergency assistance."

[0129] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0130] As described above, this system can improve the health and safety of elderly people and users with chronic diseases, and can streamline home medical care and nursing. In addition, preprocessing of collected data and AI analysis enable accurate health management and rapid emergency response.

[0131] Prompt Sentence Examples

[0132] "Please explain an AI system that detects falls in elderly people and sends an emergency call."

[0133] "Please specify the capabilities of your AI system to monitor the health status of users with chronic diseases and provide daily health advice."

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] Initial Setup and Device Deployment

[0137] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[0138] Input: Required configuration information (position of camera, microphone, and sensor).

[0139] Output: Installation complete.

[0140] Step 2:

[0141] Data collection and transmission

[0142] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. In addition, sensors in wearable devices collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded.

[0143] Input: User's daily activities, facial expressions, speech, environmental sounds, and biometric data.

[0144] Output: Collected data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[0145] Step 3:

[0146] Data reception and preprocessing

[0147] Server: Receives data sent from the device and performs preprocessing. Specifically, it resizes image data to make it more efficient to handle. For example, it reduces a high-resolution image of 1920x1080 to 640x360. It performs noise reduction on audio data and filters out background noise. It also removes outliers from sensor data and standardizes its format.

[0148] Input: Transmitted data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[0149] Output: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[0150] Step 4:

[0151] Data analysis

[0152] Server: Analyzes the preprocessed data using an AI model. For image data, it recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[0153] Input: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[0154] Output: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[0155] Step 5:

[0156] Health assessment and advice

[0157] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[0158] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[0159] Input: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[0160] Output: Health advice (notification content).

[0161] Step 6:

[0162] Emergency response

[0163] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[0164] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0165] Input: Analysis results (emergency detection).

[0166] Output: Emergency response notification (notifying family and medical institutions, voice notification to user).

[0167] (Application example 1)

[0168] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0169] Logistics centers need a system to monitor the health status of employees in real time and improve work efficiency and safety. Ensuring the safety and health of elderly employees and employees with chronic illnesses is particularly challenging, as is responding quickly to emergencies.

[0170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0171] In this invention, the server includes means for monitoring the health status of a user, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for collecting biometric data from the wearable device, means for analyzing the collected biometric data and detecting abnormal values, means for providing work instructions and advice to the user in real time, and means for issuing emergency alerts, thereby enabling effective management of employee health conditions at a logistics center and ensuring a safe working environment.

[0172] A "means for monitoring a user's health condition" is a device that uses a wearable device or sensor device to monitor and record a user's daily health data in real time.

[0173] A "means for generating health advice" is software or algorithms that analyze the collected health data and provide appropriate advice based on the user's health status.

[0174] The "means for detecting and responding to emergencies" is a system that analyzes data collected from users, detects abnormalities and emergencies early, and automatically takes appropriate measures.

[0175] "Means for collecting biometric data from wearable devices" refers to technology that uses wearable devices to continuously obtain physiological data such as a user's heart rate, blood pressure, and activity level.

[0176] The "means for analyzing the collected biometric data and detecting abnormal values" refers to an algorithm that analyzes the collected biometric data and detects abnormal values ​​or danger signs in the user's health condition.

[0177] "Means for providing users with work instructions and advice in real time" refers to a system that provides users with appropriate work instructions and health advice in real time through smart glasses or a head-mounted display.

[0178] The "means for issuing emergency alerts" is a notification system that immediately issues a warning when an abnormality is detected in the user's health condition and prompts the user to take necessary emergency measures.

[0179] "Means for monitoring the work environment using cameras" refers to technology that uses cameras installed at the work site to monitor the state of the work environment and the actions of employees.

[0180] The "means of recognizing tasks and issuing support instructions" is an AI system that provides appropriate support instructions in real time based on the work situation recognized through cameras and sensors.

[0181] MODE FOR CARRYING OUT THE INVENTION

[0182] This invention is an AI-driven monitoring system for managing employee health and improving work efficiency at logistics centers. The system consists of a wearable device worn by the user, a camera that monitors the work environment, a terminal (such as smart glasses or a head-mounted display) that provides instructions and advice in real time, and a server that analyzes the data.

[0183] Initial Setup and Device Deployment

[0184] Users: Employees wear wearable devices and use terminals such as smart glasses or head-mounted displays, which continuously collect biometric data such as heart rate and activity level, and monitor the work environment with cameras.

[0185] Data collection and transmission

[0186] Terminal: The wearable device continuously collects the user's biometric data, and the terminal monitors the working environment with a camera. This data is transmitted to a server in real time.

[0187] Data reception and preprocessing

[0188] Server: Receives biometric data and work environment data (camera footage) sent from the device and performs preprocessing. Specifically, this includes resizing image data and removing noise from audio data. This preprocessing allows for efficient subsequent analysis.

[0189] Data analysis

[0190] Server: Analyzes the preprocessed data using a generative AI model, detecting abnormal values ​​in biometric data and recognizing work situations in camera footage.

[0191] Health assessment and advice

[0192] Server: Integrates the analysis results and evaluates the user's health condition and work situation. Based on this evaluation, it generates health advice and work instructions in real time. For example, if the user's heart rate is high or their activity level is low, it generates appropriate advice and sends it to the device.

[0193] Notifications on your device

[0194] Terminal: Displays advice and instructions from the server in real time and notifies the user. For example, through smart glasses, instructions such as "Take a short break" or "Please organize the next shelf" are provided.

[0195] Emergency response

[0196] Server: If an abnormality is detected in the biometric data, it detects an emergency and automatically takes appropriate action. It also generates an emergency alert and notifies the user and administrator. For example, if the heart rate suddenly becomes abnormally high, it sends an alert and notifies a medical institution.

[0197] Terminal: Notifies the user in real time with a voice or text message such as "An abnormality has been detected. Please move to a safe location."

[0198] Specific examples

[0199] Daily Monitoring

[0200] Device: A camera monitors the work area and captures the user's movements as they pick items. A wearable device monitors heart rate and activity levels.

[0201] Server: This data is sent to the server and analyzed by the generative AI model. If there are no particular abnormalities, the monitoring results are stored in a database.

[0202] Emergency alert transmission

[0203] Terminal: The wearable device detects that the user's heart rate is extremely high. The abnormal data is sent to the server.

[0204] Server: An abnormality is detected through data analysis and it is determined to be an emergency. The server immediately notifies registered medical institutions and administrators with details. The server also notifies the user by voice, saying, "An abnormality has been detected. Medical assistance has been called, so please do not worry."

[0205] Prompt Sentence Examples

[0206] "Assess whether employees can continue to work safely based on their current heart rate and activity level."

[0207] "Check in on the work and provide any real-time direction or advice you need."

[0208] As described above, the present invention is a comprehensive system for managing the health of employees and supporting their work in a logistics center, which can significantly improve employee safety and efficiency.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1: Initial setup and device deployment

[0211] Overview: A user wears a wearable device and uses smart glasses or a head-mounted display. A camera is also installed in the work environment.

[0212] Specific operation: The user puts on the designated wearable device and connects it to smart glasses or a head-mounted display. The camera is appropriately positioned to cover the entire work area.

[0213] Input: Wearable devices, smart glasses, head-mounted displays, cameras.

[0214] Output: Ready state.

[0215] Step 2: Data collection and transmission

[0216] Overview: The terminal collects biometric data from wearable devices and monitors the work environment with a camera. This data is sent to a server in real time.

[0217] Specific operation: The wearable device collects biometric data such as heart rate, blood pressure, and activity level. The camera captures images of the work environment and generates video data.

[0218] Input: Biometric data from wearable devices, video data from cameras.

[0219] Output: Biometric data and video data sent to the server.

[0220] Step 3: Receiving and Preprocessing Data

[0221] Overview: The server receives biometric data and video data sent from the terminal and performs preprocessing.

[0222] Specific operation: The server receives data sent from the wearable device and camera, and then performs preprocessing such as resizing the image data and removing noise from the audio data.

[0223] Input: Transmitted biometric data, video data.

[0224] Output: Preprocessed data.

[0225] Step 4: Data analysis

[0226] Overview: The server analyzes the preprocessed data using a generative AI model to evaluate the user's health condition and work status.

[0227] Specific operations: Executes an algorithm to detect abnormal values ​​based on preprocessed biometric data. Analyzes video data to recognize the working environment and movements.

[0228] Input: Preprocessed biometric data, preprocessed video data.

[0229] Output: Health status assessment results, work situation assessment results.

[0230] Step 5: Health assessment and advice

[0231] Overview: The server integrates the analysis results and generates advice and instructions based on the user's health condition and work situation, which are then provided to the user in real time.

[0232] Specific operation: Based on the health status evaluation results, necessary health advice and work instructions are generated. These advice and instructions are sent to the terminal and notified to the user.

[0233] Input: Health status assessment results, work situation assessment results.

[0234] Output: The advice or instructions generated.

[0235] Step 6: Notifications on your device

[0236] Overview: The device receives advice and instructions from the server and notifies the user in real time.

[0237] Specific operation: The device provides advice and instructions to the user through voice and text messages, and also displays visual information through the smart glasses.

[0238] Input: Advice or instructions sent by the server.

[0239] Output: Advice or instructions communicated to the user.

[0240] Step 7: Emergency response

[0241] Abstract: When an abnormality is detected in biometric data, the server detects an emergency and takes appropriate action and notifies the user.

[0242] Specific operation: If an abnormality is detected in the biometric data, an emergency alert is immediately generated to notify the user, and at the same time, registered medical institutions and administrators are notified.

[0243] Input: Biometric data in which anomalies were detected.

[0244] Output: Emergency alert, notification.

[0245] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0246] This invention combines a system that monitors a user's health status, generates health advice, detects and responds to emergencies, and an emotion engine that recognizes the user's emotions. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[0247] 1. Initial Setup and Device Placement

[0248] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed to cover the user's living space. It also has an emotion engine built in.

[0249] 2. Data collection and transmission

[0250] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone recording what the user says and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. An emotion engine then analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[0251] 3. Data Reception and Preprocessing

[0252] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing, such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[0253] 4. Data Analysis

[0254] Server: The preprocessed data is input into the AI ​​model and analysis begins. For example, image data is fed into a facial recognition algorithm to recognize the user's facial expressions and movements, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[0255] 5. Health and emotional assessment and advice

[0256] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it makes a comprehensive assessment, including whether the user is likely to be feeling stressed or happy. Based on the assessment results, it generates appropriate health advice. For example, if the user is feeling stressed, it suggests that they should relax.

[0257] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[0258] 6. Emergency Response

[0259] Server: If an emergency situation is detected during the process of analyzing the user's data, such as if the user falls or if the emotion engine indicates a state of panic, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[0260] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[0261] Specific examples

[0262] Example 1: Daily monitoring and health advice

[0263] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[0264] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[0265] Example 2: Fall detection and emergency response

[0266] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[0267] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[0268] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0269] As described above, the present invention is a system that comprehensively manages a user's health and emotions, and provides appropriate advice and emergency responses, thereby improving the quality and safety of the user's life. This system consistently supports the user's life, from daily monitoring to emergency response.

[0270] The processing flow will be explained below.

[0271] Step 1:

[0272] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and environmental sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[0273] Step 2:

[0274] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is saved in batches every 60 seconds.

[0275] Step 3:

[0276] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. Specifically, it resizes image data to a size that is easy to analyze and removes noise from audio data.

[0277] Step 4:

[0278] Terminal: Sends pre-processed data to the server. A transmission schedule is set to ensure real-time and efficient data transfer.

[0279] Step 5:

[0280] Server: Receives data sent from the terminal and stores it in a database. It checks the integrity of the received data and issues a resend request if there are any errors.

[0281] Step 6:

[0282] Server: Inputs the received data into the AI ​​model and emotion engine and starts the analysis process. For example, image data is input into a facial recognition algorithm to detect the user's facial expression, and voice data is analyzed to recognize the content of speech and emotions.

[0283] Step 7:

[0284] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it determines that the user looks happy and has a normal heart rate.

[0285] Step 8:

[0286] Server: Generates appropriate health advice based on the evaluation results. For example, if the user is feeling stressed, the server will suggest ways to relax.

[0287] Step 9:

[0288] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[0289] Step 10:

[0290] Device: Notifies the user of health advice received from the server. For example, it tells the user by voice, "Today, get some exercise and relax."

[0291] Step 11:

[0292] Server: If an emergency situation is detected during analysis, the server immediately initiates the emergency response protocol. For example, if the user falls or the emotion engine indicates that the user is panicking, it will be determined to be an emergency.

[0293] Step 12:

[0294] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[0295] Step 13:

[0296] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[0297] Step 14:

[0298] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[0299] Example 2

[0300] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] Currently, many systems monitor users' health status and have the ability to detect emergencies, but few systems exist that analyze a user's emotional state in real time and provide health advice based on this analysis. Because emotional fluctuations have a significant impact on health status, comprehensive management that includes emotional state is desirable. Furthermore, when detecting and responding to emergencies, taking the user's emotional state into consideration contributes to faster and more appropriate responses. However, existing systems lack the means to integrate these factors.

[0302] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for monitoring the user's health condition, means for generating health advice based on the health condition, means for detecting and responding to emergencies, means for recording the user's movements and speech content using a camera and microphone and collecting biometric data using a sensor device, and means for analyzing the user's emotions in real time using an emotion engine. This makes it possible to comprehensively manage not only the user's health condition but also their emotional state, provide appropriate advice, and respond quickly to emergencies.

[0303] "Means for monitoring the user's health condition" refers to a device or system that uses a camera, microphone, sensor device, etc. to observe and record the user's biometric data, movements, and speech in real time.

[0304] The "means for generating health advice" is a device or program for analyzing the user's health status data and providing appropriate health suggestions and instructions.

[0305] "Means for detecting and responding to emergencies" refers to a device or system that monitors user data in real time, quickly issues an alert if an abnormality or danger occurs, and takes the necessary action.

[0306] "Means for recording user actions and speech using a camera and microphone" refers to a device or system that collects video and audio data and monitors user actions and speech in real time.

[0307] "Means for collecting biometric data using a sensor device" refers to a device or system that measures and records biometric indicators such as heart rate, body temperature, and number of steps in real time.

[0308] "Means for analyzing user emotions in real time using an emotion engine" refers to a device or program that analyzes the user's emotional state from facial expressions, tone of voice, etc., and obtains the results in real time.

[0309] "Means for collecting text data, audio data, image data, and video data" refers to a device or system that collects various types of data related to users and stores it for analysis.

[0310] "Means for analyzing the data to assess the user's health and emotional state" refers to a device or program that uses the collected data to determine the user's current health and emotional state and generate an assessment result.

[0311] A "means for providing appropriate health advice" is a device or system that provides actionable health suggestions or instructions to the user based on the assessment results.

[0312] "Means for early detection of an emergency" refers to a device or program that analyzes data in real time and detects signs of an emergency.

[0313] "Means for quickly notifying medical institutions when an emergency occurs" refers to a device or system that automatically notifies designated medical institutions and relevant parties when an emergency is confirmed.

[0314] MODE FOR CARRYING OUT THE INVENTION

[0315] This invention is a system that comprehensively monitors a user's health and emotional state, provides appropriate health advice, and responds quickly to emergencies. The system mainly consists of the following elements:

[0316] Hardware and Software Configuration

[0317] 1. Terminal placement:

[0318] User: The user installs cameras, microphones, various sensors, and an AI-driven robot in their home. The devices are arranged to cover the user's living space. The emotion engine is also initially configured.

[0319] 2. Data Collection:

[0320] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature, and the emotion engine analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[0321] 3. Data reception and preprocessing:

[0322] Server: The server receives data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[0323] 4. Data Analysis:

[0324] Server: The preprocessed data is input into the AI ​​model for analysis. For example, image data is fed into a facial recognition algorithm, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[0325] 5. Health and emotional assessment and advice:

[0326] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates appropriate health advice.

[0327] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[0328] 6. Emergency Response:

[0329] Server: If an emergency situation is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[0330] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[0331] Specific examples

[0332] Example 1: Daily monitoring and health advice

[0333] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[0334] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[0335] Example 2: Fall detection and emergency response

[0336] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[0337] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[0338] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0339] This system allows users to comprehensively manage their health and emotional state, providing daily health advice and rapid response in emergencies.

[0340] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0341] Step 1: Initial setup and device deployment

[0342] User: Cameras, microphones, various sensors, and an AI-driven robot are placed in the user's home. The devices are installed to cover the entire living space of the user. The emotion engine is also initially configured and its operation is confirmed.

[0343] Input: The location and initial setting information for each device

[0344] Output: Device layout diagram after installation is complete and confirmation results of initial settings

[0345] Specific operation: Cameras and microphones are placed in the living room, bedroom, etc. to provide full coverage. A wearable device is worn by the user to periodically record biometric data such as heart rate and body temperature.

[0346] Step 2: Data collection and transmission

[0347] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature. The emotion engine analyzes the user's emotions in real time. This data is periodically sent to the server.

[0348] Input: Camera footage, audio data, biometric data, emotional data

[0349] Output: Collected data is sent to the server

[0350] Specific operation: The camera periodically captures the user's image and performs facial recognition and facial expression analysis. The microphone records the user's speech and detects specific keywords in real time. The sensor compiles and logs heart rate and step count. The emotion engine analyzes the user's voice and facial expressions and generates emotion tags.

[0351] Step 3: Receiving and Preprocessing Data

[0352] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[0353] Input: Raw data sent from the terminal

[0354] Output: Preprocessed data

[0355] What it does: The server receives the data packets and stores them in a database. Image data is resized to optimize storage efficiency. Audio data is filtered to extract clear speech. Emotion data is converted to a standard format and integrated with other data.

[0356] Step 4: Data analysis

[0357] Server: The preprocessed data is used by the AI ​​model for analysis. Image data is fed into a facial recognition algorithm, and audio data is analyzed to obtain speech content and tone. Emotion data recognized by the emotion engine is also integrated to understand the user's emotional state.

[0358] Input: Preprocessed data

[0359] Output: Analysis results (health status, emotional status)

[0360] How it works: Facial recognition algorithms analyze images to recognize the user's facial expressions. Voice analysis algorithms capture the text and tone of speech. Emotional state data is integrated over time to generate a daily emotional pattern.

[0361] Step 5: Health and Emotion Assessment and Advice

[0362] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates health advice. For example, it suggests ways to relax if the user is feeling stressed.

[0363] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[0364] Input: Analysis results

[0365] Output: Health advice

[0366] Specific operation: The health assessment algorithm analyzes various data and performs a comprehensive assessment. The generated health advice is sent to the device and notified to the user via voice message.

[0367] Step 6: Emergency response

[0368] Server: If an emergency is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[0369] Device: Directly notifies the user of emergency responses and provides necessary instructions. For example, a voice message will be issued saying, "A fall has been detected. We will call for help, so please don't worry."

[0370] Input: Emergency detection data

[0371] Output: Emergency call, user notification

[0372] How it works: The fall detection algorithm analyzes camera and sensor data to detect a fall. The emergency notification module automatically notifies designated contacts and explains the situation. The device then communicates the situation to the user via voice prompts and prompts them to take the necessary action.

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional health management systems monitor the health status of individual users and provide advice, but are not suitable for use in physical stores. Furthermore, there is a need to monitor the emotions of customers and store staff in real time and respond appropriately and immediately to emergencies. In such an environment, comprehensive monitoring is required, including not only health status but also emotional status.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring the user's health state, means for generating health advice based on the health state, means for detecting and responding to emergencies, means for a terminal installed in the store to monitor the health states and emotions of customers and store employees and provide appropriate advice, and means for immediately notifying a medical institution or relevant parties when an emergency is detected. This enables real-time monitoring of the health and emotional states of customers and store employees in a physical store, realizing comprehensive health management and rapid emergency response.

[0377] A "user" is an individual who uses the system, and includes customers and store clerks.

[0378] "Health Status" refers to the state of a user's physical and mental health.

[0379] "Monitoring" refers to the continuous observation and recording of a user's health and emotional state using cameras, microphones, and various sensors.

[0380] "Advice" is advice or suggestions generated based on your health and emotional state, encouraging appropriate behavior.

[0381] An "emergency" refers to a physical or mental crisis that requires immediate attention.

[0382] "Responding" means taking appropriate action in response to a detected emergency.

[0383] "Terminal" refers to a device that includes a camera, microphone, various sensors, and an AI-driven robot, and is a device that collects and analyzes user data.

[0384] "Emotion" refers to a user's state of mind or emotional response.

[0385] An "emotion engine" refers to software that analyzes a user's emotional state from data such as facial expressions and voice.

[0386] "Data collection" refers to obtaining text data, audio data, image data, and video data from users.

[0387] "Assessment" refers to analyzing collected data to determine the user's health and emotional state.

[0388] "Analytics" refers to the technical means of processing data to understand and recognize the health and emotional state of the user.

[0389] "Server" is a centralized computing system that receives, pre-processes, and analyzes data to assess the user's health and emotional state.

[0390] "Healthcare Provider" means a health care provider contacted to address an emergency.

[0391] "Notification" means informing the appropriate parties and agencies of a detected emergency.

[0392] An "AI model" is an artificial intelligence technology used to analyze collected data and assess a user's health and emotional state.

[0393] "Brick and mortar store" means a business establishment that exists in a physical location and offers goods and services to customers.

[0394] The system of this invention monitors the health and emotional states of customers and store staff in physical stores, provides appropriate advice, and responds quickly in the event of an emergency. The system includes a terminal installed in the store, a server that performs data analysis, and an emotion engine.

[0395] 1. Initial Setup and Device Placement

[0396] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. These terminals are placed to cover the area within the store to monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[0397] 2. Data collection and transmission

[0398] The device monitors the user's daily activities, with a camera capturing the user's movements and facial expressions and a microphone recording the user's speech and environmental sounds. Additionally, various sensors collect environmental and biometric data, such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes the user's emotions in real time and periodically sends the collected data to a server.

[0399] 3. Data Reception and Preprocessing

[0400] The server receives data sent from the device and stores it in a database. At that time, it checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[0401] 4. Data Analysis

[0402] The server inputs the preprocessed data into the AI ​​model and begins the analysis process. The analysis process integrates a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine to grasp the user's health and emotional state. Based on the results, a comprehensive health assessment is performed.

[0403] 5. Health and emotional assessment and advice

[0404] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[0405] 6. Emergency Response

[0406] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message to inform emergency responders, such as "Please wait until medical staff arrive."

[0407] Specific examples

[0408] Daily monitoring and health advice

[0409] While a user is watching TV in their living room, a camera captures their movements, a microphone captures their voice, and a sensor checks their heart rate. The emotion engine analyzes the user's state of relaxation. The server analyzes this data, finds no abnormalities, and stores it in a database.

[0410] Fall Detection and Emergency Response

[0411] If a user falls in the store, the camera captures the incident and the emotion engine analyzes emotions such as fear and pain. The server recognizes this as an emergency and immediately notifies medical institutions and store staff. The device then issues a voice message saying, "Please wait until medical staff arrives."

[0412] Prompt Sentence Examples

[0413] When a customer falls in a store, cameras and sensors detect the fall, and an emotion engine analyzes the fear and pain. This information is analyzed by a server and determined to be an emergency. Medical staff are immediately notified automatically, and a robot in the store issues a voice message saying, "Please wait until medical staff arrives."

[0414] This invention enables comprehensive monitoring of the health and emotional state of customers and store staff in brick-and-mortar stores and emergency response.

[0415] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0416] Step 1:

[0417] Initial Setup and Device Deployment

[0418] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. The terminals are positioned to cover the area within the store where they monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[0419] Specific behavior:

[0420] The locations for cameras, microphones, and sensors are determined and placed throughout the store. The emotion engine is installed on the robot and its operation is confirmed.

[0421] Input: Camera, microphone, various sensors, robot

[0422] Output: Installation complete

[0423] Step 2:

[0424] Data collection and transmission

[0425] The device monitors the user's daily activities, the camera captures the user's movements and facial expressions, and the microphone records the user's speech and environmental sounds. Various sensors collect environmental and biometric data such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes this data and periodically sends it to a server.

[0426] Specific behavior:

[0427] The system continuously captures video with a camera, records audio with a microphone, and collects biometric data with sensors, and transmits this data to a server at regular intervals.

[0428] Input: User movement data, speech data, environmental sounds, biometric data

[0429] Output: Collected dataset

[0430] Step 3:

[0431] Data reception and preprocessing

[0432] The server receives the data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[0433] Specific behavior:

[0434] The server verifies the consistency of the data, resizes the image data to a standard size, removes background noise from the audio data, and converts the emotion data into a specified format.

[0435] Input: Collected dataset

[0436] Output: Preprocessed dataset

[0437] Step 4:

[0438] Data analysis

[0439] The server runs an AI model based on the preprocessed data and begins the analysis process. By integrating a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine, the server can grasp the user's health and emotional state.

[0440] Specific behavior:

[0441] Facial recognition algorithms analyze facial expressions, tone is analyzed from audio data, and data is fed into an emotion engine to analyze emotional states.

[0442] Input: Preprocessed dataset

[0443] Output: Analyzed data and health / emotional state

[0444] Step 5:

[0445] Health and emotional assessment and advice

[0446] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[0447] Specific behavior:

[0448] The server generates advice based on the analysis results and sends it to the device, which then notifies the user of the advice via audio or video.

[0449] Input: Parsed data

[0450] Output: Health and emotional state assessment results, advice notification

[0451] Step 6:

[0452] Emergency response

[0453] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message saying, "Please wait until medical staff arrives."

[0454] Specific behavior:

[0455] When the server detects an emergency, it automatically contacts pre-registered medical institutions and family members, and the device notifies the user of the emergency response via voice.

[0456] Input: Parsed data

[0457] Output: Emergency notification, contact to medical institution

[0458] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0459] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0460] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0461] [Second embodiment]

[0462] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0463] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0464] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0465] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0466] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0467] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0468] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0469] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0470] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0471] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0472] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0473] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0474] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and those with chronic diseases. The system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[0475] 1. Initial Setup and Device Placement

[0476] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed so as to cover the user's living space.

[0477] 2. Data collection and transmission

[0478] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone capturing the user's speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. The collected data is periodically sent to a server.

[0479] 3. Data Reception and Preprocessing

[0480] Server: Receives data sent from the device and performs preprocessing. For example, resizing image data or removing noise from audio data. This preprocessing ensures that subsequent analysis can be performed efficiently.

[0481] 4. Data Analysis

[0482] Server: The preprocessed data is analyzed using an AI model. Specifically, image data is used to recognize the user's facial expressions and movements, voice data is used to analyze speech content and tone, and biometric data is used to detect abnormalities.

[0483] 5. Health assessment and advice

[0484] Server: Integrates the analysis results and evaluates the user's health condition. Based on this evaluation, it generates daily health advice. For example, if the user's activity level is low, it generates advice such as "exercise a little more" and sends it to the device.

[0485] Device: Receives advice from the server and notifies the user by voice or text. For example, the device may say, "It would be good to take a short walk today."

[0486] 6. Emergency Response

[0487] Server: If an emergency situation is detected during the analysis of the user's data, for example if the user falls, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[0488] The device also has a function to directly notify the user of this emergency response, for example, by issuing a voice message such as "A fall has been detected. We will call for help, so please don't worry."

[0489] Specific examples

[0490] Daily Monitoring

[0491] Device: The camera captures the living room and captures the user watching TV, the microphone captures ambient sound, and the sensor checks the user's heart rate.

[0492] Server: This data is sent to the server and analyzed by the AI ​​model. If the user is confirmed to be relaxed and there are no particular abnormalities, the monitoring results are stored in a database.

[0493] Fall Detection and Emergency Response

[0494] Device: The camera captures the user falling, and the fall data is sent to the server.

[0495] Server: The server analyzes the camera image to detect the user's fall and determines that it is an emergency. The server immediately notifies registered family members and the nearest medical institution, providing details along with the user's location.

[0496] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0497] As described above, the present invention is a system for enhancing the health and safety of elderly people and users with chronic diseases and for streamlining home medical care and nursing. This system consistently supports users' lives, from daily monitoring to emergency response.

[0498] The processing flow will be explained below.

[0499] Step 1:

[0500] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and surrounding sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[0501] Step 2:

[0502] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is stored as segments every minute.

[0503] Step 3:

[0504] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. For example, it resizes image data to a size that is easy to analyze, and removes noise from audio data.

[0505] Step 4:

[0506] Terminal: Sends pre-processed data to the server, optimizing communication delays so that data is transferred in real time.

[0507] Step 5:

[0508] Server: Receives data sent from the device and stores it in a database. It checks the integrity of the received data and requests retransmission if necessary.

[0509] Step 6:

[0510] Server: Inputs the received data into the AI ​​model and starts the analysis process. For example, inputs image data into a facial recognition algorithm to detect the user's facial expressions and movements.

[0511] Step 7:

[0512] Server: Evaluates the user's health condition based on the analysis results. For example, if the heart rate is within the normal range and the facial expression is cheerful, it determines that the user is under little stress.

[0513] Step 8:

[0514] Server: Generates appropriate health advice based on the health assessment results. For example, "The weather is nice today, so I recommend taking a 15-minute walk."

[0515] Step 9:

[0516] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[0517] Step 10:

[0518] Terminal: Notifies the user of health advice received from the server. For example, a message such as "Let's take a walk today" is spoken through a speaker.

[0519] Step 11:

[0520] Server: If an emergency situation is detected during analysis, the server immediately initiates an emergency response protocol. For example, if a user falls, the server will detect the fall based on image analysis and determine that it is an emergency.

[0521] Step 12:

[0522] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[0523] Step 13:

[0524] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[0525] Step 14:

[0526] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[0527] Example 1

[0528] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0529] Health management for the elderly and those with chronic diseases has become an important issue in modern society. In particular, monitoring of daily life and early detection of emergencies are required, but few systems can efficiently achieve these. In addition, improving the quality of collected data and analyzing it effectively are also challenges.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0531] In this invention, the server includes means for monitoring a user's health status, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for preprocessing data, means for analyzing the monitoring data using an AI model, means for performing daily health management based on the analysis results, and means for responding to emergencies based on the analysis results. This enables monitoring of the daily lives of elderly people and people with chronic diseases, and enables rapid detection and response to emergencies. Furthermore, data preprocessing and AI analysis improve the quality of collected data, enabling more accurate health management and emergency responses.

[0532] "Users" refer to elderly people and individuals with chronic illnesses who use the system.

[0533] "Health monitoring" refers to the continuous collection and recording of a user's movements, facial expressions, speech, environmental sounds, and biometric data during their daily life using cameras, microphones, and various sensors.

[0534] "Generating health advice" refers to automatically creating appropriate advice for improving the user's health in their daily life based on collected data and the results of its analysis.

[0535] "Emergency detection" means detecting a user's fall or abnormal biometric data from the analyzed data and recognizing a situation that requires immediate action.

[0536] "Implementing emergency response" refers to the process of promptly notifying registered family members and medical institutions and requesting support in the event of a detected emergency.

[0537] "Data preprocessing" refers to the process of resizing, removing noise, standardizing the format, etc. of collected data in order to analyze it efficiently.

[0538] "Analyzing with an AI model" refers to using artificial intelligence to analyze various collected data and process it to understand and predict the user's health condition and behavioral patterns.

[0539] "Daily health management" refers to evaluating the user's health condition based on analyzed data and providing necessary advice and precautions.

[0540] "Emergency response" refers to a series of actions to provide prompt and appropriate notification and assistance when an abnormality or emergency situation is detected in a user.

[0541] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and users with chronic diseases. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions. Specific embodiments for implementing this system are described below.

[0542] Initial Setup and Device Deployment

[0543] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[0544] Data collection and transmission

[0545] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded. The collected data is periodically sent to a server.

[0546] Data reception and preprocessing

[0547] Server: Receives data sent from the device and performs preprocessing. Specifically, image data is resized to make it more efficient to handle, for example, reducing a high-resolution image of 1920x1080 to 640x360. Audio data is denoised to filter out background noise. Sensor data is sanitized and formatted to eliminate outliers.

[0548] Data analysis

[0549] Server: The preprocessed data is analyzed using an AI model. For image data, the AI ​​model recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[0550] Health assessment and advice

[0551] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[0552] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[0553] Emergency response

[0554] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[0555] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0556] Specific examples

[0557] Daily Monitoring

[0558] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and records the user's voice "enjoying the news program." The sensor checks the user's heart rate and records their relaxed state.

[0559] Server: This data is sent to the server and analyzed by the AI ​​model. It confirms that the user is relaxed, and if there are no particular abnormalities, it stores the monitoring results in a database. For example, it records "13:45, living room, normal."

[0560] Fall Detection and Emergency Response

[0561] Device: A camera captures a user falling in the living room and sends the fall data to the server.

[0562] Server: Image analysis confirms that the user has fallen and determines that it is an emergency. The server then notifies registered family members and the nearest medical institution, saying, "The user has fallen in the living room. Please provide emergency assistance."

[0563] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0564] As described above, this system can improve the health and safety of elderly people and users with chronic diseases, and can streamline home medical care and nursing. In addition, preprocessing of collected data and AI analysis enable accurate health management and rapid emergency response.

[0565] Prompt Sentence Examples

[0566] "Please explain an AI system that detects falls in elderly people and sends an emergency call."

[0567] "Please specify the capabilities of your AI system to monitor the health status of users with chronic diseases and provide daily health advice."

[0568] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0569] Step 1:

[0570] Initial Setup and Device Deployment

[0571] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[0572] Input: Required configuration information (position of camera, microphone, and sensor).

[0573] Output: Installation complete.

[0574] Step 2:

[0575] Data collection and transmission

[0576] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. In addition, sensors in wearable devices collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded.

[0577] Input: User's daily activities, facial expressions, speech, environmental sounds, and biometric data.

[0578] Output: Collected data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[0579] Step 3:

[0580] Data reception and preprocessing

[0581] Server: Receives data sent from the device and performs preprocessing. Specifically, it resizes image data to make it more efficient to handle. For example, it reduces a high-resolution image of 1920x1080 to 640x360. It performs noise reduction on audio data and filters out background noise. It also removes outliers from sensor data and standardizes its format.

[0582] Input: Transmitted data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[0583] Output: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[0584] Step 4:

[0585] Data analysis

[0586] Server: Analyzes the preprocessed data using an AI model. For image data, it recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[0587] Input: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[0588] Output: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[0589] Step 5:

[0590] Health assessment and advice

[0591] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[0592] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[0593] Input: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[0594] Output: Health advice (notification content).

[0595] Step 6:

[0596] Emergency response

[0597] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[0598] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0599] Input: Analysis results (emergency detection).

[0600] Output: Emergency response notification (notifying family and medical institutions, voice notification to user).

[0601] (Application example 1)

[0602] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0603] Logistics centers need a system to monitor the health status of employees in real time and improve work efficiency and safety. Ensuring the safety and health of elderly employees and employees with chronic illnesses is particularly challenging, as is responding quickly to emergencies.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0605] In this invention, the server includes means for monitoring the health status of a user, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for collecting biometric data from the wearable device, means for analyzing the collected biometric data and detecting abnormal values, means for providing work instructions and advice to the user in real time, and means for issuing emergency alerts, thereby enabling effective management of employee health conditions at a logistics center and ensuring a safe working environment.

[0606] A "means for monitoring a user's health condition" is a device that uses a wearable device or sensor device to monitor and record a user's daily health data in real time.

[0607] A "means for generating health advice" is software or algorithms that analyze the collected health data and provide appropriate advice based on the user's health status.

[0608] The "means for detecting and responding to emergencies" is a system that analyzes data collected from users, detects abnormalities and emergencies early, and automatically takes appropriate measures.

[0609] "Means for collecting biometric data from wearable devices" refers to technology that uses wearable devices to continuously obtain physiological data such as a user's heart rate, blood pressure, and activity level.

[0610] The "means for analyzing the collected biometric data and detecting abnormal values" refers to an algorithm that analyzes the collected biometric data and detects abnormal values ​​or danger signs in the user's health condition.

[0611] "Means for providing users with work instructions and advice in real time" refers to a system that provides users with appropriate work instructions and health advice in real time through smart glasses or a head-mounted display.

[0612] The "means for issuing emergency alerts" is a notification system that immediately issues a warning when an abnormality is detected in the user's health condition and prompts the user to take necessary emergency measures.

[0613] "Means for monitoring the work environment using cameras" refers to technology that uses cameras installed at the work site to monitor the state of the work environment and the actions of employees.

[0614] The "means of recognizing tasks and issuing support instructions" is an AI system that provides appropriate support instructions in real time based on the work situation recognized through cameras and sensors.

[0615] MODE FOR CARRYING OUT THE INVENTION

[0616] This invention is an AI-driven monitoring system for managing employee health and improving work efficiency at logistics centers. The system consists of a wearable device worn by the user, a camera that monitors the work environment, a terminal (such as smart glasses or a head-mounted display) that provides instructions and advice in real time, and a server that analyzes the data.

[0617] Initial Setup and Device Deployment

[0618] Users: Employees wear wearable devices and use terminals such as smart glasses or head-mounted displays, which continuously collect biometric data such as heart rate and activity level, and monitor the work environment with cameras.

[0619] Data collection and transmission

[0620] Terminal: The wearable device continuously collects the user's biometric data, and the terminal monitors the working environment with a camera. This data is transmitted to a server in real time.

[0621] Data reception and preprocessing

[0622] Server: Receives biometric data and work environment data (camera footage) sent from the device and performs preprocessing. Specifically, this includes resizing image data and removing noise from audio data. This preprocessing allows for efficient subsequent analysis.

[0623] Data analysis

[0624] Server: Analyzes the preprocessed data using a generative AI model, detecting abnormal values ​​in biometric data and recognizing work situations in camera footage.

[0625] Health assessment and advice

[0626] Server: Integrates the analysis results and evaluates the user's health condition and work situation. Based on this evaluation, it generates health advice and work instructions in real time. For example, if the user's heart rate is high or their activity level is low, it generates appropriate advice and sends it to the device.

[0627] Notifications on your device

[0628] Terminal: Displays advice and instructions from the server in real time and notifies the user. For example, through smart glasses, instructions such as "Take a short break" or "Please organize the next shelf" are provided.

[0629] Emergency response

[0630] Server: If an abnormality is detected in the biometric data, it detects an emergency and automatically takes appropriate action. It also generates an emergency alert and notifies the user and administrator. For example, if the heart rate suddenly becomes abnormally high, it sends an alert and notifies a medical institution.

[0631] Terminal: Notifies the user in real time with a voice or text message such as "An abnormality has been detected. Please move to a safe location."

[0632] Specific examples

[0633] Daily Monitoring

[0634] Device: A camera monitors the work area and captures the user's movements as they pick items. A wearable device monitors heart rate and activity levels.

[0635] Server: This data is sent to the server and analyzed by the generative AI model. If there are no particular abnormalities, the monitoring results are stored in a database.

[0636] Emergency alert transmission

[0637] Terminal: The wearable device detects that the user's heart rate is extremely high. The abnormal data is sent to the server.

[0638] Server: An abnormality is detected through data analysis and it is determined to be an emergency. The server immediately notifies registered medical institutions and administrators with details. The server also notifies the user by voice, saying, "An abnormality has been detected. Medical assistance has been called, so please do not worry."

[0639] Prompt Sentence Examples

[0640] "Assess whether employees can continue to work safely based on their current heart rate and activity level."

[0641] "Check in on the work and provide any real-time direction or advice you need."

[0642] As described above, the present invention is a comprehensive system for managing the health of employees and supporting their work in a logistics center, which can significantly improve employee safety and efficiency.

[0643] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0644] Step 1: Initial setup and device deployment

[0645] Overview: A user wears a wearable device and uses smart glasses or a head-mounted display. A camera is also installed in the work environment.

[0646] Specific operation: The user puts on the designated wearable device and connects it to smart glasses or a head-mounted display. The camera is appropriately positioned to cover the entire work area.

[0647] Input: Wearable devices, smart glasses, head-mounted displays, cameras.

[0648] Output: Ready state.

[0649] Step 2: Data collection and transmission

[0650] Overview: The terminal collects biometric data from wearable devices and monitors the work environment with a camera. This data is sent to a server in real time.

[0651] Specific operation: The wearable device collects biometric data such as heart rate, blood pressure, and activity level. The camera captures images of the work environment and generates video data.

[0652] Input: Biometric data from wearable devices, video data from cameras.

[0653] Output: Biometric data and video data sent to the server.

[0654] Step 3: Receiving and Preprocessing Data

[0655] Overview: The server receives biometric data and video data sent from the terminal and performs preprocessing.

[0656] Specific operation: The server receives data sent from the wearable device and camera, and then performs preprocessing such as resizing the image data and removing noise from the audio data.

[0657] Input: Transmitted biometric data, video data.

[0658] Output: Preprocessed data.

[0659] Step 4: Data analysis

[0660] Overview: The server analyzes the preprocessed data using a generative AI model to evaluate the user's health condition and work status.

[0661] Specific operations: Executes an algorithm to detect abnormal values ​​based on preprocessed biometric data. Analyzes video data to recognize the working environment and movements.

[0662] Input: Preprocessed biometric data, preprocessed video data.

[0663] Output: Health status assessment results, work situation assessment results.

[0664] Step 5: Health assessment and advice

[0665] Overview: The server integrates the analysis results and generates advice and instructions based on the user's health condition and work situation, which are then provided to the user in real time.

[0666] Specific operation: Based on the health status evaluation results, necessary health advice and work instructions are generated. These advice and instructions are sent to the terminal and notified to the user.

[0667] Input: Health status assessment results, work situation assessment results.

[0668] Output: The advice or instructions generated.

[0669] Step 6: Notifications on your device

[0670] Overview: The device receives advice and instructions from the server and notifies the user in real time.

[0671] Specific operation: The device provides advice and instructions to the user through voice and text messages, and also displays visual information through the smart glasses.

[0672] Input: Advice or instructions sent by the server.

[0673] Output: Advice or instructions communicated to the user.

[0674] Step 7: Emergency response

[0675] Abstract: When an abnormality is detected in biometric data, the server detects an emergency and takes appropriate action and notifies the user.

[0676] Specific operation: If an abnormality is detected in the biometric data, an emergency alert is immediately generated to notify the user, and at the same time, registered medical institutions and administrators are notified.

[0677] Input: Biometric data in which anomalies were detected.

[0678] Output: Emergency alert, notification.

[0679] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0680] This invention combines a system that monitors a user's health status, generates health advice, detects and responds to emergencies, and an emotion engine that recognizes the user's emotions. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[0681] 1. Initial Setup and Device Placement

[0682] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed to cover the user's living space. It also has an emotion engine built in.

[0683] 2. Data collection and transmission

[0684] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone recording what the user says and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. An emotion engine then analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[0685] 3. Data Reception and Preprocessing

[0686] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing, such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[0687] 4. Data Analysis

[0688] Server: The preprocessed data is input into the AI ​​model and analysis begins. For example, image data is fed into a facial recognition algorithm to recognize the user's facial expressions and movements, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[0689] 5. Health and emotional assessment and advice

[0690] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it makes a comprehensive assessment, including whether the user is likely to be feeling stressed or happy. Based on the assessment results, it generates appropriate health advice. For example, if the user is feeling stressed, it suggests that they should relax.

[0691] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[0692] 6. Emergency Response

[0693] Server: If an emergency situation is detected during the process of analyzing the user's data, such as if the user falls or if the emotion engine indicates a state of panic, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[0694] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[0695] Specific examples

[0696] Example 1: Daily monitoring and health advice

[0697] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[0698] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[0699] Example 2: Fall detection and emergency response

[0700] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[0701] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[0702] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0703] As described above, the present invention is a system that comprehensively manages a user's health and emotions, and provides appropriate advice and emergency responses, thereby improving the quality and safety of the user's life. This system consistently supports the user's life, from daily monitoring to emergency response.

[0704] The processing flow will be explained below.

[0705] Step 1:

[0706] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and environmental sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[0707] Step 2:

[0708] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is saved in batches every 60 seconds.

[0709] Step 3:

[0710] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. Specifically, it resizes image data to a size that is easy to analyze and removes noise from audio data.

[0711] Step 4:

[0712] Terminal: Sends pre-processed data to the server. A transmission schedule is set to ensure real-time and efficient data transfer.

[0713] Step 5:

[0714] Server: Receives data sent from the terminal and stores it in a database. It checks the integrity of the received data and issues a resend request if there are any errors.

[0715] Step 6:

[0716] Server: Inputs the received data into the AI ​​model and emotion engine and starts the analysis process. For example, image data is input into a facial recognition algorithm to detect the user's facial expression, and voice data is analyzed to recognize the content of speech and emotions.

[0717] Step 7:

[0718] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it determines that the user looks happy and has a normal heart rate.

[0719] Step 8:

[0720] Server: Generates appropriate health advice based on the evaluation results. For example, if the user is feeling stressed, the server will suggest ways to relax.

[0721] Step 9:

[0722] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[0723] Step 10:

[0724] Device: Notifies the user of health advice received from the server. For example, it tells the user by voice, "Today, get some exercise and relax."

[0725] Step 11:

[0726] Server: If an emergency situation is detected during analysis, the server immediately initiates the emergency response protocol. For example, if the user falls or the emotion engine indicates that the user is panicking, it will be determined to be an emergency.

[0727] Step 12:

[0728] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[0729] Step 13:

[0730] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[0731] Step 14:

[0732] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[0733] Example 2

[0734] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0735] Currently, many systems monitor users' health status and have the ability to detect emergencies, but few systems exist that analyze a user's emotional state in real time and provide health advice based on this analysis. Because emotional fluctuations have a significant impact on health status, comprehensive management that includes emotional state is desirable. Furthermore, when detecting and responding to emergencies, taking the user's emotional state into consideration contributes to faster and more appropriate responses. However, existing systems lack the means to integrate these factors.

[0736] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for monitoring the user's health condition, means for generating health advice based on the health condition, means for detecting and responding to emergencies, means for recording the user's movements and speech content using a camera and microphone and collecting biometric data using a sensor device, and means for analyzing the user's emotions in real time using an emotion engine. This makes it possible to comprehensively manage not only the user's health condition but also their emotional state, provide appropriate advice, and respond quickly to emergencies.

[0737] "Means for monitoring the user's health condition" refers to a device or system that uses a camera, microphone, sensor device, etc. to observe and record the user's biometric data, movements, and speech in real time.

[0738] The "means for generating health advice" is a device or program for analyzing the user's health status data and providing appropriate health suggestions and instructions.

[0739] "Means for detecting and responding to emergencies" refers to a device or system that monitors user data in real time, quickly issues an alert if an abnormality or danger occurs, and takes the necessary action.

[0740] "Means for recording user actions and speech using a camera and microphone" refers to a device or system that collects video and audio data and monitors user actions and speech in real time.

[0741] "Means for collecting biometric data using a sensor device" refers to a device or system that measures and records biometric indicators such as heart rate, body temperature, and number of steps in real time.

[0742] "Means for analyzing user emotions in real time using an emotion engine" refers to a device or program that analyzes the user's emotional state from facial expressions, tone of voice, etc., and obtains the results in real time.

[0743] "Means for collecting text data, audio data, image data, and video data" refers to a device or system that collects various types of data related to users and stores it for analysis.

[0744] "Means for analyzing the data to assess the user's health and emotional state" refers to a device or program that uses the collected data to determine the user's current health and emotional state and generate an assessment result.

[0745] A "means for providing appropriate health advice" is a device or system that provides actionable health suggestions or instructions to the user based on the assessment results.

[0746] "Means for early detection of an emergency" refers to a device or program that analyzes data in real time and detects signs of an emergency.

[0747] "Means for quickly notifying medical institutions when an emergency occurs" refers to a device or system that automatically notifies designated medical institutions and relevant parties when an emergency is confirmed.

[0748] MODE FOR CARRYING OUT THE INVENTION

[0749] This invention is a system that comprehensively monitors a user's health and emotional state, provides appropriate health advice, and responds quickly to emergencies. The system mainly consists of the following elements:

[0750] Hardware and Software Configuration

[0751] 1. Terminal placement:

[0752] User: The user installs cameras, microphones, various sensors, and an AI-driven robot in their home. The devices are arranged to cover the user's living space. The emotion engine is also initially configured.

[0753] 2. Data Collection:

[0754] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature, and the emotion engine analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[0755] 3. Data reception and preprocessing:

[0756] Server: The server receives data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[0757] 4. Data Analysis:

[0758] Server: The preprocessed data is input into the AI ​​model for analysis. For example, image data is fed into a facial recognition algorithm, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[0759] 5. Health and emotional assessment and advice:

[0760] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates appropriate health advice.

[0761] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[0762] 6. Emergency Response:

[0763] Server: If an emergency situation is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[0764] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[0765] Specific examples

[0766] Example 1: Daily monitoring and health advice

[0767] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[0768] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[0769] Example 2: Fall detection and emergency response

[0770] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[0771] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[0772] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0773] This system allows users to comprehensively manage their health and emotional state, providing daily health advice and rapid response in emergencies.

[0774] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0775] Step 1: Initial setup and device deployment

[0776] User: Cameras, microphones, various sensors, and an AI-driven robot are placed in the user's home. The devices are installed to cover the entire living space of the user. The emotion engine is also initially configured and its operation is confirmed.

[0777] Input: The location and initial setting information for each device

[0778] Output: Device layout diagram after installation is complete and confirmation results of initial settings

[0779] Specific operation: Cameras and microphones are placed in the living room, bedroom, etc. to provide full coverage. A wearable device is worn by the user to periodically record biometric data such as heart rate and body temperature.

[0780] Step 2: Data collection and transmission

[0781] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature. The emotion engine analyzes the user's emotions in real time. This data is periodically sent to the server.

[0782] Input: Camera footage, audio data, biometric data, emotional data

[0783] Output: Collected data is sent to the server

[0784] Specific operation: The camera periodically captures the user's image and performs facial recognition and facial expression analysis. The microphone records the user's speech and detects specific keywords in real time. The sensor compiles and logs heart rate and step count. The emotion engine analyzes the user's voice and facial expressions and generates emotion tags.

[0785] Step 3: Receiving and Preprocessing Data

[0786] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[0787] Input: Raw data sent from the terminal

[0788] Output: Preprocessed data

[0789] What it does: The server receives the data packets and stores them in a database. Image data is resized to optimize storage efficiency. Audio data is filtered to extract clear speech. Emotion data is converted to a standard format and integrated with other data.

[0790] Step 4: Data analysis

[0791] Server: The preprocessed data is used by the AI ​​model for analysis. Image data is fed into a facial recognition algorithm, and audio data is analyzed to obtain speech content and tone. Emotion data recognized by the emotion engine is also integrated to understand the user's emotional state.

[0792] Input: Preprocessed data

[0793] Output: Analysis results (health status, emotional status)

[0794] How it works: Facial recognition algorithms analyze images to recognize the user's facial expressions. Voice analysis algorithms capture the text and tone of speech. Emotional state data is integrated over time to generate a daily emotional pattern.

[0795] Step 5: Health and Emotion Assessment and Advice

[0796] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates health advice. For example, it suggests ways to relax if the user is feeling stressed.

[0797] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[0798] Input: Analysis results

[0799] Output: Health advice

[0800] Specific operation: The health assessment algorithm analyzes various data and performs a comprehensive assessment. The generated health advice is sent to the device and notified to the user via voice message.

[0801] Step 6: Emergency response

[0802] Server: If an emergency is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[0803] Device: Directly notifies the user of emergency responses and provides necessary instructions. For example, a voice message will be issued saying, "A fall has been detected. We will call for help, so please don't worry."

[0804] Input: Emergency detection data

[0805] Output: Emergency call, user notification

[0806] How it works: The fall detection algorithm analyzes camera and sensor data to detect a fall. The emergency notification module automatically notifies designated contacts and explains the situation. The device then communicates the situation to the user via voice prompts and prompts them to take the necessary action.

[0807] (Application example 2)

[0808] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0809] Conventional health management systems monitor the health status of individual users and provide advice, but are not suitable for use in physical stores. Furthermore, there is a need to monitor the emotions of customers and store staff in real time and respond appropriately and immediately to emergencies. In such an environment, comprehensive monitoring is required, including not only health status but also emotional status.

[0810] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring the user's health state, means for generating health advice based on the health state, means for detecting and responding to emergencies, means for a terminal installed in the store to monitor the health states and emotions of customers and store employees and provide appropriate advice, and means for immediately notifying a medical institution or relevant parties when an emergency is detected. This enables real-time monitoring of the health and emotional states of customers and store employees in a physical store, realizing comprehensive health management and rapid emergency response.

[0811] A "user" is an individual who uses the system, and includes customers and store clerks.

[0812] "Health Status" refers to the state of a user's physical and mental health.

[0813] "Monitoring" refers to the continuous observation and recording of a user's health and emotional state using cameras, microphones, and various sensors.

[0814] "Advice" is advice or suggestions generated based on your health and emotional state, encouraging appropriate behavior.

[0815] An "emergency" refers to a physical or mental crisis that requires immediate attention.

[0816] "Responding" means taking appropriate action in response to a detected emergency.

[0817] "Terminal" refers to a device that includes a camera, microphone, various sensors, and an AI-driven robot, and is a device that collects and analyzes user data.

[0818] "Emotion" refers to a user's state of mind or emotional response.

[0819] An "emotion engine" refers to software that analyzes a user's emotional state from data such as facial expressions and voice.

[0820] "Data collection" refers to obtaining text data, audio data, image data, and video data from users.

[0821] "Assessment" refers to analyzing collected data to determine the user's health and emotional state.

[0822] "Analytics" refers to the technical means of processing data to understand and recognize the health and emotional state of the user.

[0823] "Server" is a centralized computing system that receives, pre-processes, and analyzes data to assess the user's health and emotional state.

[0824] "Healthcare Provider" means a health care provider contacted to address an emergency.

[0825] "Notification" means informing the appropriate parties and agencies of a detected emergency.

[0826] An "AI model" is an artificial intelligence technology used to analyze collected data and assess a user's health and emotional state.

[0827] "Brick and mortar store" means a business establishment that exists in a physical location and offers goods and services to customers.

[0828] The system of this invention monitors the health and emotional states of customers and store staff in physical stores, provides appropriate advice, and responds quickly in the event of an emergency. The system includes a terminal installed in the store, a server that performs data analysis, and an emotion engine.

[0829] 1. Initial Setup and Device Placement

[0830] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. These terminals are placed to cover the area within the store to monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[0831] 2. Data collection and transmission

[0832] The device monitors the user's daily activities, with a camera capturing the user's movements and facial expressions and a microphone recording the user's speech and environmental sounds. Additionally, various sensors collect environmental and biometric data, such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes the user's emotions in real time and periodically sends the collected data to a server.

[0833] 3. Data Reception and Preprocessing

[0834] The server receives data sent from the device and stores it in a database. At that time, it checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[0835] 4. Data Analysis

[0836] The server inputs the preprocessed data into the AI ​​model and begins the analysis process. The analysis process integrates a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine to grasp the user's health and emotional state. Based on the results, a comprehensive health assessment is performed.

[0837] 5. Health and emotional assessment and advice

[0838] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[0839] 6. Emergency Response

[0840] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message to inform emergency responders, such as "Please wait until medical staff arrive."

[0841] Specific examples

[0842] Daily monitoring and health advice

[0843] While a user is watching TV in their living room, a camera captures their movements, a microphone captures their voice, and a sensor checks their heart rate. The emotion engine analyzes the user's state of relaxation. The server analyzes this data, finds no abnormalities, and stores it in a database.

[0844] Fall Detection and Emergency Response

[0845] If a user falls in the store, the camera captures the incident and the emotion engine analyzes emotions such as fear and pain. The server recognizes this as an emergency and immediately notifies medical institutions and store staff. The device then issues a voice message saying, "Please wait until medical staff arrives."

[0846] Prompt Sentence Examples

[0847] When a customer falls in a store, cameras and sensors detect the fall, and an emotion engine analyzes the fear and pain. This information is analyzed by a server and determined to be an emergency. Medical staff are immediately notified automatically, and a robot in the store issues a voice message saying, "Please wait until medical staff arrives."

[0848] This invention enables comprehensive monitoring of the health and emotional state of customers and store staff in brick-and-mortar stores and emergency response.

[0849] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0850] Step 1:

[0851] Initial Setup and Device Deployment

[0852] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. The terminals are positioned to cover the area within the store where they monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[0853] Specific behavior:

[0854] The locations for cameras, microphones, and sensors are determined and placed throughout the store. The emotion engine is installed on the robot and its operation is confirmed.

[0855] Input: Camera, microphone, various sensors, robot

[0856] Output: Installation complete

[0857] Step 2:

[0858] Data collection and transmission

[0859] The device monitors the user's daily activities, the camera captures the user's movements and facial expressions, and the microphone records the user's speech and environmental sounds. Various sensors collect environmental and biometric data such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes this data and periodically sends it to a server.

[0860] Specific behavior:

[0861] The system continuously captures video with a camera, records audio with a microphone, and collects biometric data with sensors, and transmits this data to a server at regular intervals.

[0862] Input: User movement data, speech data, environmental sounds, biometric data

[0863] Output: Collected dataset

[0864] Step 3:

[0865] Data reception and preprocessing

[0866] The server receives the data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[0867] Specific behavior:

[0868] The server verifies the consistency of the data, resizes the image data to a standard size, removes background noise from the audio data, and converts the emotion data into a specified format.

[0869] Input: Collected dataset

[0870] Output: Preprocessed dataset

[0871] Step 4:

[0872] Data analysis

[0873] The server runs an AI model based on the preprocessed data and begins the analysis process. By integrating a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine, the server can grasp the user's health and emotional state.

[0874] Specific behavior:

[0875] Facial recognition algorithms analyze facial expressions, tone is analyzed from audio data, and data is fed into an emotion engine to analyze emotional states.

[0876] Input: Preprocessed dataset

[0877] Output: Analyzed data and health / emotional state

[0878] Step 5:

[0879] Health and emotional assessment and advice

[0880] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[0881] Specific behavior:

[0882] The server generates advice based on the analysis results and sends it to the device, which then notifies the user of the advice via audio or video.

[0883] Input: Parsed data

[0884] Output: Health and emotional state assessment results, advice notification

[0885] Step 6:

[0886] Emergency response

[0887] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message saying, "Please wait until medical staff arrives."

[0888] Specific behavior:

[0889] When the server detects an emergency, it automatically contacts pre-registered medical institutions and family members, and the device notifies the user of the emergency response via voice.

[0890] Input: Parsed data

[0891] Output: Emergency notification, contact to medical institution

[0892] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0893] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0894] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0895] [Third embodiment]

[0896] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0897] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0898] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0899] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0900] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0901] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0902] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0903] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0904] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0905] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0906] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0907] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0908] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and those with chronic diseases. The system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[0909] 1. Initial Setup and Device Placement

[0910] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed so as to cover the user's living space.

[0911] 2. Data collection and transmission

[0912] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone capturing the user's speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. The collected data is periodically sent to a server.

[0913] 3. Data Reception and Preprocessing

[0914] Server: Receives data sent from the device and performs preprocessing. For example, resizing image data or removing noise from audio data. This preprocessing ensures that subsequent analysis can be performed efficiently.

[0915] 4. Data Analysis

[0916] Server: The preprocessed data is analyzed using an AI model. Specifically, image data is used to recognize the user's facial expressions and movements, voice data is used to analyze speech content and tone, and biometric data is used to detect abnormalities.

[0917] 5. Health assessment and advice

[0918] Server: Integrates the analysis results and evaluates the user's health condition. Based on this evaluation, it generates daily health advice. For example, if the user's activity level is low, it generates advice such as "exercise a little more" and sends it to the device.

[0919] Device: Receives advice from the server and notifies the user by voice or text. For example, the device may say, "It would be good to take a short walk today."

[0920] 6. Emergency Response

[0921] Server: If an emergency situation is detected during the analysis of the user's data, for example if the user falls, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[0922] The device also has a function to directly notify the user of this emergency response, for example, by issuing a voice message such as "A fall has been detected. We will call for help, so please don't worry."

[0923] Specific examples

[0924] Daily Monitoring

[0925] Device: The camera captures the living room and captures the user watching TV, the microphone captures ambient sound, and the sensor checks the user's heart rate.

[0926] Server: This data is sent to the server and analyzed by the AI ​​model. If the user is confirmed to be relaxed and there are no particular abnormalities, the monitoring results are stored in a database.

[0927] Fall Detection and Emergency Response

[0928] Device: The camera captures the user falling, and the fall data is sent to the server.

[0929] Server: The server analyzes the camera image to detect the user's fall and determines that it is an emergency. The server immediately notifies registered family members and the nearest medical institution, providing details along with the user's location.

[0930] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0931] As described above, the present invention is a system for enhancing the health and safety of elderly people and users with chronic diseases and for streamlining home medical care and nursing. This system consistently supports users' lives, from daily monitoring to emergency response.

[0932] The processing flow will be explained below.

[0933] Step 1:

[0934] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and surrounding sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[0935] Step 2:

[0936] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is stored as segments every minute.

[0937] Step 3:

[0938] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. For example, it resizes image data to a size that is easy to analyze, and removes noise from audio data.

[0939] Step 4:

[0940] Terminal: Sends pre-processed data to the server, optimizing communication delays so that data is transferred in real time.

[0941] Step 5:

[0942] Server: Receives data sent from the device and stores it in a database. It checks the integrity of the received data and requests retransmission if necessary.

[0943] Step 6:

[0944] Server: Inputs the received data into the AI ​​model and starts the analysis process. For example, inputs image data into a facial recognition algorithm to detect the user's facial expressions and movements.

[0945] Step 7:

[0946] Server: Evaluates the user's health condition based on the analysis results. For example, if the heart rate is within the normal range and the facial expression is cheerful, it determines that the user is under little stress.

[0947] Step 8:

[0948] Server: Generates appropriate health advice based on the health assessment results. For example, "The weather is nice today, so I recommend taking a 15-minute walk."

[0949] Step 9:

[0950] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[0951] Step 10:

[0952] Terminal: Notifies the user of health advice received from the server. For example, a message such as "Let's take a walk today" is spoken through a speaker.

[0953] Step 11:

[0954] Server: If an emergency situation is detected during analysis, the server immediately initiates an emergency response protocol. For example, if a user falls, the server will detect the fall based on image analysis and determine that it is an emergency.

[0955] Step 12:

[0956] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[0957] Step 13:

[0958] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[0959] Step 14:

[0960] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[0961] Example 1

[0962] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0963] Health management for the elderly and those with chronic diseases has become an important issue in modern society. In particular, monitoring of daily life and early detection of emergencies are required, but few systems can efficiently achieve these. In addition, improving the quality of collected data and analyzing it effectively are also challenges.

[0964] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0965] In this invention, the server includes means for monitoring a user's health status, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for preprocessing data, means for analyzing the monitoring data using an AI model, means for performing daily health management based on the analysis results, and means for responding to emergencies based on the analysis results. This enables monitoring of the daily lives of elderly people and people with chronic diseases, and enables rapid detection and response to emergencies. Furthermore, data preprocessing and AI analysis improve the quality of collected data, enabling more accurate health management and emergency responses.

[0966] "Users" refer to elderly people and individuals with chronic illnesses who use the system.

[0967] "Health monitoring" refers to the continuous collection and recording of a user's movements, facial expressions, speech, environmental sounds, and biometric data during their daily life using cameras, microphones, and various sensors.

[0968] "Generating health advice" refers to automatically creating appropriate advice for improving the user's health in their daily life based on collected data and the results of its analysis.

[0969] "Emergency detection" means detecting a user's fall or abnormal biometric data from the analyzed data and recognizing a situation that requires immediate action.

[0970] "Implementing emergency response" refers to the process of promptly notifying registered family members and medical institutions and requesting support in the event of a detected emergency.

[0971] "Data preprocessing" refers to the process of resizing, removing noise, standardizing the format, etc. of collected data in order to analyze it efficiently.

[0972] "Analyzing with an AI model" refers to using artificial intelligence to analyze various collected data and process it to understand and predict the user's health condition and behavioral patterns.

[0973] "Daily health management" refers to evaluating the user's health condition based on analyzed data and providing necessary advice and precautions.

[0974] "Emergency response" refers to a series of actions to provide prompt and appropriate notification and assistance when an abnormality or emergency situation is detected in a user.

[0975] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and users with chronic diseases. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions. Specific embodiments for implementing this system are described below.

[0976] Initial Setup and Device Deployment

[0977] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[0978] Data collection and transmission

[0979] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded. The collected data is periodically sent to a server.

[0980] Data reception and preprocessing

[0981] Server: Receives data sent from the device and performs preprocessing. Specifically, image data is resized to make it more efficient to handle, for example, reducing a high-resolution image of 1920x1080 to 640x360. Audio data is denoised to filter out background noise. Sensor data is sanitized and formatted to eliminate outliers.

[0982] Data analysis

[0983] Server: The preprocessed data is analyzed using an AI model. For image data, the AI ​​model recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[0984] Health assessment and advice

[0985] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[0986] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[0987] Emergency response

[0988] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[0989] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0990] Specific examples

[0991] Daily Monitoring

[0992] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and records the user's voice "enjoying the news program." The sensor checks the user's heart rate and records their relaxed state.

[0993] Server: This data is sent to the server and analyzed by the AI ​​model. It confirms that the user is relaxed, and if there are no particular abnormalities, it stores the monitoring results in a database. For example, it records "13:45, living room, normal."

[0994] Fall Detection and Emergency Response

[0995] Device: A camera captures a user falling in the living room and sends the fall data to the server.

[0996] Server: Image analysis confirms that the user has fallen and determines that it is an emergency. The server then notifies registered family members and the nearest medical institution, saying, "The user has fallen in the living room. Please provide emergency assistance."

[0997] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[0998] As described above, this system can improve the health and safety of elderly people and users with chronic diseases, and can streamline home medical care and nursing. In addition, preprocessing of collected data and AI analysis enable accurate health management and rapid emergency response.

[0999] Prompt Sentence Examples

[1000] "Please explain an AI system that detects falls in elderly people and sends an emergency call."

[1001] "Please specify the capabilities of your AI system to monitor the health status of users with chronic diseases and provide daily health advice."

[1002] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1003] Step 1:

[1004] Initial Setup and Device Deployment

[1005] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[1006] Input: Required configuration information (position of camera, microphone, and sensor).

[1007] Output: Installation complete.

[1008] Step 2:

[1009] Data collection and transmission

[1010] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. In addition, sensors in wearable devices collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded.

[1011] Input: User's daily activities, facial expressions, speech, environmental sounds, and biometric data.

[1012] Output: Collected data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[1013] Step 3:

[1014] Data reception and preprocessing

[1015] Server: Receives data sent from the device and performs preprocessing. Specifically, it resizes image data to make it more efficient to handle. For example, it reduces a high-resolution image of 1920x1080 to 640x360. It performs noise reduction on audio data and filters out background noise. It also removes outliers from sensor data and standardizes its format.

[1016] Input: Transmitted data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[1017] Output: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[1018] Step 4:

[1019] Data analysis

[1020] Server: Analyzes the preprocessed data using an AI model. For image data, it recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[1021] Input: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[1022] Output: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[1023] Step 5:

[1024] Health assessment and advice

[1025] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[1026] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[1027] Input: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[1028] Output: Health advice (notification content).

[1029] Step 6:

[1030] Emergency response

[1031] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[1032] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1033] Input: Analysis results (emergency detection).

[1034] Output: Emergency response notification (notifying family and medical institutions, voice notification to user).

[1035] (Application example 1)

[1036] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1037] Logistics centers need a system to monitor the health status of employees in real time and improve work efficiency and safety. Ensuring the safety and health of elderly employees and employees with chronic illnesses is particularly challenging, as is responding quickly to emergencies.

[1038] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1039] In this invention, the server includes means for monitoring the health status of a user, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for collecting biometric data from the wearable device, means for analyzing the collected biometric data and detecting abnormal values, means for providing work instructions and advice to the user in real time, and means for issuing emergency alerts, thereby enabling effective management of employee health conditions at a logistics center and ensuring a safe working environment.

[1040] A "means for monitoring a user's health condition" is a device that uses a wearable device or sensor device to monitor and record a user's daily health data in real time.

[1041] A "means for generating health advice" is software or algorithms that analyze the collected health data and provide appropriate advice based on the user's health status.

[1042] The "means for detecting and responding to emergencies" is a system that analyzes data collected from users, detects abnormalities and emergencies early, and automatically takes appropriate measures.

[1043] "Means for collecting biometric data from wearable devices" refers to technology that uses wearable devices to continuously obtain physiological data such as a user's heart rate, blood pressure, and activity level.

[1044] The "means for analyzing the collected biometric data and detecting abnormal values" refers to an algorithm that analyzes the collected biometric data and detects abnormal values ​​or danger signs in the user's health condition.

[1045] "Means for providing users with work instructions and advice in real time" refers to a system that provides users with appropriate work instructions and health advice in real time through smart glasses or a head-mounted display.

[1046] The "means for issuing emergency alerts" is a notification system that immediately issues a warning when an abnormality is detected in the user's health condition and prompts the user to take necessary emergency measures.

[1047] "Means for monitoring the work environment using cameras" refers to technology that uses cameras installed at the work site to monitor the state of the work environment and the actions of employees.

[1048] The "means of recognizing tasks and issuing support instructions" is an AI system that provides appropriate support instructions in real time based on the work situation recognized through cameras and sensors.

[1049] MODE FOR CARRYING OUT THE INVENTION

[1050] This invention is an AI-driven monitoring system for managing employee health and improving work efficiency at logistics centers. The system consists of a wearable device worn by the user, a camera that monitors the work environment, a terminal (such as smart glasses or a head-mounted display) that provides instructions and advice in real time, and a server that analyzes the data.

[1051] Initial Setup and Device Deployment

[1052] Users: Employees wear wearable devices and use terminals such as smart glasses or head-mounted displays, which continuously collect biometric data such as heart rate and activity level, and monitor the work environment with cameras.

[1053] Data collection and transmission

[1054] Terminal: The wearable device continuously collects the user's biometric data, and the terminal monitors the working environment with a camera. This data is transmitted to a server in real time.

[1055] Data reception and preprocessing

[1056] Server: Receives biometric data and work environment data (camera footage) sent from the device and performs preprocessing. Specifically, this includes resizing image data and removing noise from audio data. This preprocessing allows for efficient subsequent analysis.

[1057] Data analysis

[1058] Server: Analyzes the preprocessed data using a generative AI model, detecting abnormal values ​​in biometric data and recognizing work situations in camera footage.

[1059] Health assessment and advice

[1060] Server: Integrates the analysis results and evaluates the user's health condition and work situation. Based on this evaluation, it generates health advice and work instructions in real time. For example, if the user's heart rate is high or their activity level is low, it generates appropriate advice and sends it to the device.

[1061] Notifications on your device

[1062] Terminal: Displays advice and instructions from the server in real time and notifies the user. For example, through smart glasses, instructions such as "Take a short break" or "Please organize the next shelf" are provided.

[1063] Emergency response

[1064] Server: If an abnormality is detected in the biometric data, it detects an emergency and automatically takes appropriate action. It also generates an emergency alert and notifies the user and administrator. For example, if the heart rate suddenly becomes abnormally high, it sends an alert and notifies a medical institution.

[1065] Terminal: Notifies the user in real time with a voice or text message such as "An abnormality has been detected. Please move to a safe location."

[1066] Specific examples

[1067] Daily Monitoring

[1068] Device: A camera monitors the work area and captures the user's movements as they pick items. A wearable device monitors heart rate and activity levels.

[1069] Server: This data is sent to the server and analyzed by the generative AI model. If there are no particular abnormalities, the monitoring results are stored in a database.

[1070] Emergency alert transmission

[1071] Terminal: The wearable device detects that the user's heart rate is extremely high. The abnormal data is sent to the server.

[1072] Server: An abnormality is detected through data analysis and it is determined to be an emergency. The server immediately notifies registered medical institutions and administrators with details. The server also notifies the user by voice, saying, "An abnormality has been detected. Medical assistance has been called, so please do not worry."

[1073] Prompt Sentence Examples

[1074] "Assess whether employees can continue to work safely based on their current heart rate and activity level."

[1075] "Check in on the work and provide any real-time direction or advice you need."

[1076] As described above, the present invention is a comprehensive system for managing the health of employees and supporting their work in a logistics center, which can significantly improve employee safety and efficiency.

[1077] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1078] Step 1: Initial setup and device deployment

[1079] Overview: A user wears a wearable device and uses smart glasses or a head-mounted display. A camera is also installed in the work environment.

[1080] Specific operation: The user puts on the designated wearable device and connects it to smart glasses or a head-mounted display. The camera is appropriately positioned to cover the entire work area.

[1081] Input: Wearable devices, smart glasses, head-mounted displays, cameras.

[1082] Output: Ready state.

[1083] Step 2: Data collection and transmission

[1084] Overview: The terminal collects biometric data from wearable devices and monitors the work environment with a camera. This data is sent to a server in real time.

[1085] Specific operation: The wearable device collects biometric data such as heart rate, blood pressure, and activity level. The camera captures images of the work environment and generates video data.

[1086] Input: Biometric data from wearable devices, video data from cameras.

[1087] Output: Biometric data and video data sent to the server.

[1088] Step 3: Receiving and Preprocessing Data

[1089] Overview: The server receives biometric data and video data sent from the terminal and performs preprocessing.

[1090] Specific operation: The server receives data sent from the wearable device and camera, and then performs preprocessing such as resizing the image data and removing noise from the audio data.

[1091] Input: Transmitted biometric data, video data.

[1092] Output: Preprocessed data.

[1093] Step 4: Data analysis

[1094] Overview: The server analyzes the preprocessed data using a generative AI model to evaluate the user's health condition and work status.

[1095] Specific operations: Executes an algorithm to detect abnormal values ​​based on preprocessed biometric data. Analyzes video data to recognize the working environment and movements.

[1096] Input: Preprocessed biometric data, preprocessed video data.

[1097] Output: Health status assessment results, work situation assessment results.

[1098] Step 5: Health assessment and advice

[1099] Overview: The server integrates the analysis results and generates advice and instructions based on the user's health condition and work situation, which are then provided to the user in real time.

[1100] Specific operation: Based on the health status evaluation results, necessary health advice and work instructions are generated. These advice and instructions are sent to the terminal and notified to the user.

[1101] Input: Health status assessment results, work situation assessment results.

[1102] Output: The advice or instructions generated.

[1103] Step 6: Notifications on your device

[1104] Overview: The device receives advice and instructions from the server and notifies the user in real time.

[1105] Specific operation: The device provides advice and instructions to the user through voice and text messages, and also displays visual information through the smart glasses.

[1106] Input: Advice or instructions sent by the server.

[1107] Output: Advice or instructions communicated to the user.

[1108] Step 7: Emergency response

[1109] Abstract: When an abnormality is detected in biometric data, the server detects an emergency and takes appropriate action and notifies the user.

[1110] Specific operation: If an abnormality is detected in the biometric data, an emergency alert is immediately generated to notify the user, and at the same time, registered medical institutions and administrators are notified.

[1111] Input: Biometric data in which anomalies were detected.

[1112] Output: Emergency alert, notification.

[1113] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1114] This invention combines a system that monitors a user's health status, generates health advice, detects and responds to emergencies, and an emotion engine that recognizes the user's emotions. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[1115] 1. Initial Setup and Device Placement

[1116] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed to cover the user's living space. It also has an emotion engine built in.

[1117] 2. Data collection and transmission

[1118] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone recording what the user says and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. An emotion engine then analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[1119] 3. Data Reception and Preprocessing

[1120] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing, such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[1121] 4. Data Analysis

[1122] Server: The preprocessed data is input into the AI ​​model and analysis begins. For example, image data is fed into a facial recognition algorithm to recognize the user's facial expressions and movements, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[1123] 5. Health and emotional assessment and advice

[1124] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it makes a comprehensive assessment, including whether the user is likely to be feeling stressed or happy. Based on the assessment results, it generates appropriate health advice. For example, if the user is feeling stressed, it suggests that they should relax.

[1125] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[1126] 6. Emergency Response

[1127] Server: If an emergency situation is detected during the process of analyzing the user's data, such as if the user falls or if the emotion engine indicates a state of panic, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[1128] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[1129] Specific examples

[1130] Example 1: Daily monitoring and health advice

[1131] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[1132] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[1133] Example 2: Fall detection and emergency response

[1134] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[1135] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[1136] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1137] As described above, the present invention is a system that comprehensively manages a user's health and emotions, and provides appropriate advice and emergency responses, thereby improving the quality and safety of the user's life. This system consistently supports the user's life, from daily monitoring to emergency response.

[1138] The processing flow will be explained below.

[1139] Step 1:

[1140] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and environmental sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[1141] Step 2:

[1142] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is saved in batches every 60 seconds.

[1143] Step 3:

[1144] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. Specifically, it resizes image data to a size that is easy to analyze and removes noise from audio data.

[1145] Step 4:

[1146] Terminal: Sends pre-processed data to the server. A transmission schedule is set to ensure real-time and efficient data transfer.

[1147] Step 5:

[1148] Server: Receives data sent from the terminal and stores it in a database. It checks the integrity of the received data and issues a resend request if there are any errors.

[1149] Step 6:

[1150] Server: Inputs the received data into the AI ​​model and emotion engine and starts the analysis process. For example, image data is input into a facial recognition algorithm to detect the user's facial expression, and voice data is analyzed to recognize the content of speech and emotions.

[1151] Step 7:

[1152] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it determines that the user looks happy and has a normal heart rate.

[1153] Step 8:

[1154] Server: Generates appropriate health advice based on the evaluation results. For example, if the user is feeling stressed, the server will suggest ways to relax.

[1155] Step 9:

[1156] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[1157] Step 10:

[1158] Device: Notifies the user of health advice received from the server. For example, it tells the user by voice, "Today, get some exercise and relax."

[1159] Step 11:

[1160] Server: If an emergency situation is detected during analysis, the server immediately initiates the emergency response protocol. For example, if the user falls or the emotion engine indicates that the user is panicking, it will be determined to be an emergency.

[1161] Step 12:

[1162] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[1163] Step 13:

[1164] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[1165] Step 14:

[1166] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[1167] Example 2

[1168] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1169] Currently, many systems monitor users' health status and have the ability to detect emergencies, but few systems exist that analyze a user's emotional state in real time and provide health advice based on this analysis. Because emotional fluctuations have a significant impact on health status, comprehensive management that includes emotional state is desirable. Furthermore, when detecting and responding to emergencies, taking the user's emotional state into consideration contributes to faster and more appropriate responses. However, existing systems lack the means to integrate these factors.

[1170] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for monitoring the user's health condition, means for generating health advice based on the health condition, means for detecting and responding to emergencies, means for recording the user's movements and speech content using a camera and microphone and collecting biometric data using a sensor device, and means for analyzing the user's emotions in real time using an emotion engine. This makes it possible to comprehensively manage not only the user's health condition but also their emotional state, provide appropriate advice, and respond quickly to emergencies.

[1171] "Means for monitoring the user's health condition" refers to a device or system that uses a camera, microphone, sensor device, etc. to observe and record the user's biometric data, movements, and speech in real time.

[1172] The "means for generating health advice" is a device or program for analyzing the user's health status data and providing appropriate health suggestions and instructions.

[1173] "Means for detecting and responding to emergencies" refers to a device or system that monitors user data in real time, quickly issues an alert if an abnormality or danger occurs, and takes the necessary action.

[1174] "Means for recording user actions and speech using a camera and microphone" refers to a device or system that collects video and audio data and monitors user actions and speech in real time.

[1175] "Means for collecting biometric data using a sensor device" refers to a device or system that measures and records biometric indicators such as heart rate, body temperature, and number of steps in real time.

[1176] "Means for analyzing user emotions in real time using an emotion engine" refers to a device or program that analyzes the user's emotional state from facial expressions, tone of voice, etc., and obtains the results in real time.

[1177] "Means for collecting text data, audio data, image data, and video data" refers to a device or system that collects various types of data related to users and stores it for analysis.

[1178] "Means for analyzing the data to assess the user's health and emotional state" refers to a device or program that uses the collected data to determine the user's current health and emotional state and generate an assessment result.

[1179] A "means for providing appropriate health advice" is a device or system that provides actionable health suggestions or instructions to the user based on the assessment results.

[1180] "Means for early detection of an emergency" refers to a device or program that analyzes data in real time and detects signs of an emergency.

[1181] "Means for quickly notifying medical institutions when an emergency occurs" refers to a device or system that automatically notifies designated medical institutions and relevant parties when an emergency is confirmed.

[1182] MODE FOR CARRYING OUT THE INVENTION

[1183] This invention is a system that comprehensively monitors a user's health and emotional state, provides appropriate health advice, and responds quickly to emergencies. The system mainly consists of the following elements:

[1184] Hardware and Software Configuration

[1185] 1. Terminal placement:

[1186] User: The user installs cameras, microphones, various sensors, and an AI-driven robot in their home. The devices are arranged to cover the user's living space. The emotion engine is also initially configured.

[1187] 2. Data Collection:

[1188] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature, and the emotion engine analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[1189] 3. Data reception and preprocessing:

[1190] Server: The server receives data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[1191] 4. Data Analysis:

[1192] Server: The preprocessed data is input into the AI ​​model for analysis. For example, image data is fed into a facial recognition algorithm, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[1193] 5. Health and emotional assessment and advice:

[1194] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates appropriate health advice.

[1195] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[1196] 6. Emergency Response:

[1197] Server: If an emergency situation is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[1198] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[1199] Specific examples

[1200] Example 1: Daily monitoring and health advice

[1201] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[1202] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[1203] Example 2: Fall detection and emergency response

[1204] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[1205] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[1206] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1207] This system allows users to comprehensively manage their health and emotional state, providing daily health advice and rapid response in emergencies.

[1208] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1209] Step 1: Initial setup and device deployment

[1210] User: Cameras, microphones, various sensors, and an AI-driven robot are placed in the user's home. The devices are installed to cover the entire living space of the user. The emotion engine is also initially configured and its operation is confirmed.

[1211] Input: The location and initial setting information for each device

[1212] Output: Device layout diagram after installation is complete and confirmation results of initial settings

[1213] Specific operation: Cameras and microphones are placed in the living room, bedroom, etc. to provide full coverage. A wearable device is worn by the user to periodically record biometric data such as heart rate and body temperature.

[1214] Step 2: Data collection and transmission

[1215] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature. The emotion engine analyzes the user's emotions in real time. This data is periodically sent to the server.

[1216] Input: Camera footage, audio data, biometric data, emotional data

[1217] Output: Collected data is sent to the server

[1218] Specific operation: The camera periodically captures the user's image and performs facial recognition and facial expression analysis. The microphone records the user's speech and detects specific keywords in real time. The sensor compiles and logs heart rate and step count. The emotion engine analyzes the user's voice and facial expressions and generates emotion tags.

[1219] Step 3: Receiving and Preprocessing Data

[1220] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[1221] Input: Raw data sent from the terminal

[1222] Output: Preprocessed data

[1223] What it does: The server receives the data packets and stores them in a database. Image data is resized to optimize storage efficiency. Audio data is filtered to extract clear speech. Emotion data is converted to a standard format and integrated with other data.

[1224] Step 4: Data analysis

[1225] Server: The preprocessed data is used by the AI ​​model for analysis. Image data is fed into a facial recognition algorithm, and audio data is analyzed to obtain speech content and tone. Emotion data recognized by the emotion engine is also integrated to understand the user's emotional state.

[1226] Input: Preprocessed data

[1227] Output: Analysis results (health status, emotional status)

[1228] How it works: Facial recognition algorithms analyze images to recognize the user's facial expressions. Voice analysis algorithms capture the text and tone of speech. Emotional state data is integrated over time to generate a daily emotional pattern.

[1229] Step 5: Health and Emotion Assessment and Advice

[1230] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates health advice. For example, it suggests ways to relax if the user is feeling stressed.

[1231] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[1232] Input: Analysis results

[1233] Output: Health advice

[1234] Specific operation: The health assessment algorithm analyzes various data and performs a comprehensive assessment. The generated health advice is sent to the device and notified to the user via voice message.

[1235] Step 6: Emergency response

[1236] Server: If an emergency is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[1237] Device: Directly notifies the user of emergency responses and provides necessary instructions. For example, a voice message will be issued saying, "A fall has been detected. We will call for help, so please don't worry."

[1238] Input: Emergency detection data

[1239] Output: Emergency call, user notification

[1240] How it works: The fall detection algorithm analyzes camera and sensor data to detect a fall. The emergency notification module automatically notifies designated contacts and explains the situation. The device then communicates the situation to the user via voice prompts and prompts them to take the necessary action.

[1241] (Application example 2)

[1242] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1243] Conventional health management systems monitor the health status of individual users and provide advice, but are not suitable for use in physical stores. Furthermore, there is a need to monitor the emotions of customers and store staff in real time and respond appropriately and immediately to emergencies. In such an environment, comprehensive monitoring is required, including not only health status but also emotional status.

[1244] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring the user's health state, means for generating health advice based on the health state, means for detecting and responding to emergencies, means for a terminal installed in the store to monitor the health states and emotions of customers and store employees and provide appropriate advice, and means for immediately notifying a medical institution or relevant parties when an emergency is detected. This enables real-time monitoring of the health and emotional states of customers and store employees in a physical store, realizing comprehensive health management and rapid emergency response.

[1245] A "user" is an individual who uses the system, and includes customers and store clerks.

[1246] "Health Status" refers to the state of a user's physical and mental health.

[1247] "Monitoring" refers to the continuous observation and recording of a user's health and emotional state using cameras, microphones, and various sensors.

[1248] "Advice" is advice or suggestions generated based on your health and emotional state, encouraging appropriate behavior.

[1249] An "emergency" refers to a physical or mental crisis that requires immediate attention.

[1250] "Responding" means taking appropriate action in response to a detected emergency.

[1251] "Terminal" refers to a device that includes a camera, microphone, various sensors, and an AI-driven robot, and is a device that collects and analyzes user data.

[1252] "Emotion" refers to a user's state of mind or emotional response.

[1253] An "emotion engine" refers to software that analyzes a user's emotional state from data such as facial expressions and voice.

[1254] "Data collection" refers to obtaining text data, audio data, image data, and video data from users.

[1255] "Assessment" refers to analyzing collected data to determine the user's health and emotional state.

[1256] "Analytics" refers to the technical means of processing data to understand and recognize the health and emotional state of the user.

[1257] "Server" is a centralized computing system that receives, pre-processes, and analyzes data to assess the user's health and emotional state.

[1258] "Healthcare Provider" means a health care provider contacted to address an emergency.

[1259] "Notification" means informing the appropriate parties and agencies of a detected emergency.

[1260] An "AI model" is an artificial intelligence technology used to analyze collected data and assess a user's health and emotional state.

[1261] "Brick and mortar store" means a business establishment that exists in a physical location and offers goods and services to customers.

[1262] The system of this invention monitors the health and emotional states of customers and store staff in physical stores, provides appropriate advice, and responds quickly in the event of an emergency. The system includes a terminal installed in the store, a server that performs data analysis, and an emotion engine.

[1263] 1. Initial Setup and Device Placement

[1264] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. These terminals are placed to cover the area within the store to monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[1265] 2. Data collection and transmission

[1266] The device monitors the user's daily activities, with a camera capturing the user's movements and facial expressions and a microphone recording the user's speech and environmental sounds. Additionally, various sensors collect environmental and biometric data, such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes the user's emotions in real time and periodically sends the collected data to a server.

[1267] 3. Data Reception and Preprocessing

[1268] The server receives data sent from the device and stores it in a database. At that time, it checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[1269] 4. Data Analysis

[1270] The server inputs the preprocessed data into the AI ​​model and begins the analysis process. The analysis process integrates a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine to grasp the user's health and emotional state. Based on the results, a comprehensive health assessment is performed.

[1271] 5. Health and emotional assessment and advice

[1272] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[1273] 6. Emergency Response

[1274] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message to inform emergency responders, such as "Please wait until medical staff arrive."

[1275] Specific examples

[1276] Daily monitoring and health advice

[1277] While a user is watching TV in their living room, a camera captures their movements, a microphone captures their voice, and a sensor checks their heart rate. The emotion engine analyzes the user's state of relaxation. The server analyzes this data, finds no abnormalities, and stores it in a database.

[1278] Fall Detection and Emergency Response

[1279] If a user falls in the store, the camera captures the incident and the emotion engine analyzes emotions such as fear and pain. The server recognizes this as an emergency and immediately notifies medical institutions and store staff. The device then issues a voice message saying, "Please wait until medical staff arrives."

[1280] Prompt Sentence Examples

[1281] When a customer falls in a store, cameras and sensors detect the fall, and an emotion engine analyzes the fear and pain. This information is analyzed by a server and determined to be an emergency. Medical staff are immediately notified automatically, and a robot in the store issues a voice message saying, "Please wait until medical staff arrives."

[1282] This invention enables comprehensive monitoring of the health and emotional state of customers and store staff in brick-and-mortar stores and emergency response.

[1283] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1284] Step 1:

[1285] Initial Setup and Device Deployment

[1286] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. The terminals are positioned to cover the area within the store where they monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[1287] Specific behavior:

[1288] The locations for cameras, microphones, and sensors are determined and placed throughout the store. The emotion engine is installed on the robot and its operation is confirmed.

[1289] Input: Camera, microphone, various sensors, robot

[1290] Output: Installation complete

[1291] Step 2:

[1292] Data collection and transmission

[1293] The device monitors the user's daily activities, the camera captures the user's movements and facial expressions, and the microphone records the user's speech and environmental sounds. Various sensors collect environmental and biometric data such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes this data and periodically sends it to a server.

[1294] Specific behavior:

[1295] The system continuously captures video with a camera, records audio with a microphone, and collects biometric data with sensors, and transmits this data to a server at regular intervals.

[1296] Input: User movement data, speech data, environmental sounds, biometric data

[1297] Output: Collected dataset

[1298] Step 3:

[1299] Data reception and preprocessing

[1300] The server receives the data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[1301] Specific behavior:

[1302] The server verifies the consistency of the data, resizes the image data to a standard size, removes background noise from the audio data, and converts the emotion data into a specified format.

[1303] Input: Collected dataset

[1304] Output: Preprocessed dataset

[1305] Step 4:

[1306] Data analysis

[1307] The server runs an AI model based on the preprocessed data and begins the analysis process. By integrating a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine, the server can grasp the user's health and emotional state.

[1308] Specific behavior:

[1309] Facial recognition algorithms analyze facial expressions, tone is analyzed from audio data, and data is fed into an emotion engine to analyze emotional states.

[1310] Input: Preprocessed dataset

[1311] Output: Analyzed data and health / emotional state

[1312] Step 5:

[1313] Health and emotional assessment and advice

[1314] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[1315] Specific behavior:

[1316] The server generates advice based on the analysis results and sends it to the device, which then notifies the user of the advice via audio or video.

[1317] Input: Parsed data

[1318] Output: Health and emotional state assessment results, advice notification

[1319] Step 6:

[1320] Emergency response

[1321] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message saying, "Please wait until medical staff arrives."

[1322] Specific behavior:

[1323] When the server detects an emergency, it automatically contacts pre-registered medical institutions and family members, and the device notifies the user of the emergency response via voice.

[1324] Input: Parsed data

[1325] Output: Emergency notification, contact to medical institution

[1326] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1327] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1328] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1329] [Fourth embodiment]

[1330] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1331] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1332] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1333] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1334] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1335] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1336] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1337] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1338] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1339] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1340] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1341] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1342] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1343] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and those with chronic diseases. The system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[1344] 1. Initial Setup and Device Placement

[1345] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed so as to cover the user's living space.

[1346] 2. Data collection and transmission

[1347] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone capturing the user's speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. The collected data is periodically sent to a server.

[1348] 3. Data Reception and Preprocessing

[1349] Server: Receives data sent from the device and performs preprocessing. For example, resizing image data or removing noise from audio data. This preprocessing ensures that subsequent analysis can be performed efficiently.

[1350] 4. Data Analysis

[1351] Server: The preprocessed data is analyzed using an AI model. Specifically, image data is used to recognize the user's facial expressions and movements, voice data is used to analyze speech content and tone, and biometric data is used to detect abnormalities.

[1352] 5. Health assessment and advice

[1353] Server: Integrates the analysis results and evaluates the user's health condition. Based on this evaluation, it generates daily health advice. For example, if the user's activity level is low, it generates advice such as "exercise a little more" and sends it to the device.

[1354] Device: Receives advice from the server and notifies the user by voice or text. For example, the device may say, "It would be good to take a short walk today."

[1355] 6. Emergency Response

[1356] Server: If an emergency situation is detected during the analysis of the user's data, for example if the user falls, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[1357] The device also has a function to directly notify the user of this emergency response, for example, by issuing a voice message such as "A fall has been detected. We will call for help, so please don't worry."

[1358] Specific examples

[1359] Daily Monitoring

[1360] Device: The camera captures the living room and captures the user watching TV, the microphone captures ambient sound, and the sensor checks the user's heart rate.

[1361] Server: This data is sent to the server and analyzed by the AI ​​model. If the user is confirmed to be relaxed and there are no particular abnormalities, the monitoring results are stored in a database.

[1362] Fall Detection and Emergency Response

[1363] Device: The camera captures the user falling, and the fall data is sent to the server.

[1364] Server: The server analyzes the camera image to detect the user's fall and determines that it is an emergency. The server immediately notifies registered family members and the nearest medical institution, providing details along with the user's location.

[1365] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1366] As described above, the present invention is a system for enhancing the health and safety of elderly people and users with chronic diseases and for streamlining home medical care and nursing. This system consistently supports users' lives, from daily monitoring to emergency response.

[1367] The processing flow will be explained below.

[1368] Step 1:

[1369] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and surrounding sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[1370] Step 2:

[1371] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is stored as segments every minute.

[1372] Step 3:

[1373] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. For example, it resizes image data to a size that is easy to analyze, and removes noise from audio data.

[1374] Step 4:

[1375] Terminal: Sends pre-processed data to the server, optimizing communication delays so that data is transferred in real time.

[1376] Step 5:

[1377] Server: Receives data sent from the device and stores it in a database. It checks the integrity of the received data and requests retransmission if necessary.

[1378] Step 6:

[1379] Server: Inputs the received data into the AI ​​model and starts the analysis process. For example, inputs image data into a facial recognition algorithm to detect the user's facial expressions and movements.

[1380] Step 7:

[1381] Server: Evaluates the user's health condition based on the analysis results. For example, if the heart rate is within the normal range and the facial expression is cheerful, it determines that the user is under little stress.

[1382] Step 8:

[1383] Server: Generates appropriate health advice based on the health assessment results. For example, "The weather is nice today, so I recommend taking a 15-minute walk."

[1384] Step 9:

[1385] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[1386] Step 10:

[1387] Terminal: Notifies the user of health advice received from the server. For example, a message such as "Let's take a walk today" is spoken through a speaker.

[1388] Step 11:

[1389] Server: If an emergency situation is detected during analysis, the server immediately initiates an emergency response protocol. For example, if a user falls, the server will detect the fall based on image analysis and determine that it is an emergency.

[1390] Step 12:

[1391] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[1392] Step 13:

[1393] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[1394] Step 14:

[1395] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[1396] Example 1

[1397] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1398] Health management for the elderly and those with chronic diseases has become an important issue in modern society. In particular, monitoring of daily life and early detection of emergencies are required, but few systems can efficiently achieve these. In addition, improving the quality of collected data and analyzing it effectively are also challenges.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1400] In this invention, the server includes means for monitoring a user's health status, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for preprocessing data, means for analyzing the monitoring data using an AI model, means for performing daily health management based on the analysis results, and means for responding to emergencies based on the analysis results. This enables monitoring of the daily lives of elderly people and people with chronic diseases, and enables rapid detection and response to emergencies. Furthermore, data preprocessing and AI analysis improve the quality of collected data, enabling more accurate health management and emergency responses.

[1401] "Users" refer to elderly people and individuals with chronic illnesses who use the system.

[1402] "Health monitoring" refers to the continuous collection and recording of a user's movements, facial expressions, speech, environmental sounds, and biometric data during their daily life using cameras, microphones, and various sensors.

[1403] "Generating health advice" refers to automatically creating appropriate advice for improving the user's health in their daily life based on collected data and the results of its analysis.

[1404] "Emergency detection" means detecting a user's fall or abnormal biometric data from the analyzed data and recognizing a situation that requires immediate action.

[1405] "Implementing emergency response" refers to the process of promptly notifying registered family members and medical institutions and requesting support in the event of a detected emergency.

[1406] "Data preprocessing" refers to the process of resizing, removing noise, standardizing the format, etc. of collected data in order to analyze it efficiently.

[1407] "Analyzing with an AI model" refers to using artificial intelligence to analyze various collected data and process it to understand and predict the user's health condition and behavioral patterns.

[1408] "Daily health management" refers to evaluating the user's health condition based on analyzed data and providing necessary advice and precautions.

[1409] "Emergency response" refers to a series of actions to provide prompt and appropriate notification and assistance when an abnormality or emergency situation is detected in a user.

[1410] This invention is an AI-driven robot system for the purpose of health management and nursing care support for elderly people and users with chronic diseases. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions. Specific embodiments for implementing this system are described below.

[1411] Initial Setup and Device Deployment

[1412] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[1413] Data collection and transmission

[1414] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded. The collected data is periodically sent to a server.

[1415] Data reception and preprocessing

[1416] Server: Receives data sent from the device and performs preprocessing. Specifically, image data is resized to make it more efficient to handle, for example, reducing a high-resolution image of 1920x1080 to 640x360. Audio data is denoised to filter out background noise. Sensor data is sanitized and formatted to eliminate outliers.

[1417] Data analysis

[1418] Server: The preprocessed data is analyzed using an AI model. For image data, the AI ​​model recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[1419] Health assessment and advice

[1420] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[1421] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[1422] Emergency response

[1423] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[1424] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1425] Specific examples

[1426] Daily Monitoring

[1427] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and records the user's voice "enjoying the news program." The sensor checks the user's heart rate and records their relaxed state.

[1428] Server: This data is sent to the server and analyzed by the AI ​​model. It confirms that the user is relaxed, and if there are no particular abnormalities, it stores the monitoring results in a database. For example, it records "13:45, living room, normal."

[1429] Fall Detection and Emergency Response

[1430] Device: A camera captures a user falling in the living room and sends the fall data to the server.

[1431] Server: Image analysis confirms that the user has fallen and determines that it is an emergency. The server then notifies registered family members and the nearest medical institution, saying, "The user has fallen in the living room. Please provide emergency assistance."

[1432] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1433] As described above, this system can improve the health and safety of elderly people and users with chronic diseases, and can streamline home medical care and nursing. In addition, preprocessing of collected data and AI analysis enable accurate health management and rapid emergency response.

[1434] Prompt Sentence Examples

[1435] "Please explain an AI system that detects falls in elderly people and sends an emergency call."

[1436] "Please specify the capabilities of your AI system to monitor the health status of users with chronic diseases and provide daily health advice."

[1437] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1438] Step 1:

[1439] Initial Setup and Device Deployment

[1440] User: First, the user installs cameras, microphones, various sensors, and an AI-driven robot (hereinafter referred to as "terminals") in appropriate locations in their home. These terminals are positioned so that they cover the user's living space. For example, a camera can be placed in the corner of the living room and a microphone can be placed next to the bed.

[1441] Input: Required configuration information (position of camera, microphone, and sensor).

[1442] Output: Installation complete.

[1443] Step 2:

[1444] Data collection and transmission

[1445] Device: To monitor the user's daily life, a camera captures the user's movements and facial expressions, and a microphone captures speech and environmental sounds. In addition, sensors in wearable devices collect biometric data such as the user's heart rate and number of steps. For example, the number of steps and heart rate during a morning walk can be recorded.

[1446] Input: User's daily activities, facial expressions, speech, environmental sounds, and biometric data.

[1447] Output: Collected data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[1448] Step 3:

[1449] Data reception and preprocessing

[1450] Server: Receives data sent from the device and performs preprocessing. Specifically, it resizes image data to make it more efficient to handle. For example, it reduces a high-resolution image of 1920x1080 to 640x360. It performs noise reduction on audio data and filters out background noise. It also removes outliers from sensor data and standardizes its format.

[1451] Input: Transmitted data (video of movements and facial expressions, audio of speech and environmental sounds, biometric data).

[1452] Output: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[1453] Step 4:

[1454] Data analysis

[1455] Server: Analyzes the preprocessed data using an AI model. For image data, it recognizes the user's facial expressions and movements. For example, it analyzes whether the user is smiling while sitting in the living room or walking. For voice data, it uses voice recognition technology to analyze the content and tone of speech, recognizing when a user says, "The weather is nice today." For sensor data, it detects anomalies, detecting an abnormality when the heart rate exceeds the normal range.

[1456] Input: Preprocessed data (resized image data, denoised audio data, unified format sensor data).

[1457] Output: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[1458] Step 5:

[1459] Health assessment and advice

[1460] Server: Integrates the results of various data analyses to evaluate the user's health condition. For example, it detects that the user has had a series of days of low activity and determines that the user is not getting enough exercise. Based on this evaluation result, it generates individual health advice. For example, it generates advice such as "You're not getting enough exercise, so it would be good to take a short walk," and sends it to the device.

[1461] Device: Receives advice from the server and notifies the user by voice or text. For example, a voice message saying, "It would be good to take a short walk today."

[1462] Input: Analysis results (recognition results of facial expressions and movements, analysis results of speech content and tone, presence or absence of abnormalities).

[1463] Output: Health advice (notification content).

[1464] Step 6:

[1465] Emergency response

[1466] Server: If an emergency situation is detected during data analysis, for example if the user has fallen, the server will initiate an emergency response. The server will notify registered family members and the nearest medical institution and send a message such as "The user has fallen in the living room. Please send emergency assistance."

[1467] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1468] Input: Analysis results (emergency detection).

[1469] Output: Emergency response notification (notifying family and medical institutions, voice notification to user).

[1470] (Application example 1)

[1471] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1472] Logistics centers need a system to monitor the health status of employees in real time and improve work efficiency and safety. Ensuring the safety and health of elderly employees and employees with chronic illnesses is particularly challenging, as is responding quickly to emergencies.

[1473] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1474] In this invention, the server includes means for monitoring the health status of a user, means for generating health advice based on the health status, means for detecting and responding to emergencies, means for collecting biometric data from the wearable device, means for analyzing the collected biometric data and detecting abnormal values, means for providing work instructions and advice to the user in real time, and means for issuing emergency alerts, thereby enabling effective management of employee health conditions at a logistics center and ensuring a safe working environment.

[1475] A "means for monitoring a user's health condition" is a device that uses a wearable device or sensor device to monitor and record a user's daily health data in real time.

[1476] A "means for generating health advice" is software or algorithms that analyze the collected health data and provide appropriate advice based on the user's health status.

[1477] The "means for detecting and responding to emergencies" is a system that analyzes data collected from users, detects abnormalities and emergencies early, and automatically takes appropriate measures.

[1478] "Means for collecting biometric data from wearable devices" refers to technology that uses wearable devices to continuously obtain physiological data such as a user's heart rate, blood pressure, and activity level.

[1479] The "means for analyzing the collected biometric data and detecting abnormal values" refers to an algorithm that analyzes the collected biometric data and detects abnormal values ​​or danger signs in the user's health condition.

[1480] "Means for providing users with work instructions and advice in real time" refers to a system that provides users with appropriate work instructions and health advice in real time through smart glasses or a head-mounted display.

[1481] The "means for issuing emergency alerts" is a notification system that immediately issues a warning when an abnormality is detected in the user's health condition and prompts the user to take necessary emergency measures.

[1482] "Means for monitoring the work environment using cameras" refers to technology that uses cameras installed at the work site to monitor the state of the work environment and the actions of employees.

[1483] The "means of recognizing tasks and issuing support instructions" is an AI system that provides appropriate support instructions in real time based on the work situation recognized through cameras and sensors.

[1484] MODE FOR CARRYING OUT THE INVENTION

[1485] This invention is an AI-driven monitoring system for managing employee health and improving work efficiency at logistics centers. The system consists of a wearable device worn by the user, a camera that monitors the work environment, a terminal (such as smart glasses or a head-mounted display) that provides instructions and advice in real time, and a server that analyzes the data.

[1486] Initial Setup and Device Deployment

[1487] Users: Employees wear wearable devices and use terminals such as smart glasses or head-mounted displays, which continuously collect biometric data such as heart rate and activity level, and monitor the work environment with cameras.

[1488] Data collection and transmission

[1489] Terminal: The wearable device continuously collects the user's biometric data, and the terminal monitors the working environment with a camera. This data is transmitted to a server in real time.

[1490] Data reception and preprocessing

[1491] Server: Receives biometric data and work environment data (camera footage) sent from the device and performs preprocessing. Specifically, this includes resizing image data and removing noise from audio data. This preprocessing allows for efficient subsequent analysis.

[1492] Data analysis

[1493] Server: Analyzes the preprocessed data using a generative AI model, detecting abnormal values ​​in biometric data and recognizing work situations in camera footage.

[1494] Health assessment and advice

[1495] Server: Integrates the analysis results and evaluates the user's health condition and work situation. Based on this evaluation, it generates health advice and work instructions in real time. For example, if the user's heart rate is high or their activity level is low, it generates appropriate advice and sends it to the device.

[1496] Notifications on your device

[1497] Terminal: Displays advice and instructions from the server in real time and notifies the user. For example, through smart glasses, instructions such as "Take a short break" or "Please organize the next shelf" are provided.

[1498] Emergency response

[1499] Server: If an abnormality is detected in the biometric data, it detects an emergency and automatically takes appropriate action. It also generates an emergency alert and notifies the user and administrator. For example, if the heart rate suddenly becomes abnormally high, it sends an alert and notifies a medical institution.

[1500] Terminal: Notifies the user in real time with a voice or text message such as "An abnormality has been detected. Please move to a safe location."

[1501] Specific examples

[1502] Daily Monitoring

[1503] Device: A camera monitors the work area and captures the user's movements as they pick items. A wearable device monitors heart rate and activity levels.

[1504] Server: This data is sent to the server and analyzed by the generative AI model. If there are no particular abnormalities, the monitoring results are stored in a database.

[1505] Emergency alert transmission

[1506] Terminal: The wearable device detects that the user's heart rate is extremely high. The abnormal data is sent to the server.

[1507] Server: An abnormality is detected through data analysis and it is determined to be an emergency. The server immediately notifies registered medical institutions and administrators with details. The server also notifies the user by voice, saying, "An abnormality has been detected. Medical assistance has been called, so please do not worry."

[1508] Prompt Sentence Examples

[1509] "Assess whether employees can continue to work safely based on their current heart rate and activity level."

[1510] "Check in on the work and provide any real-time direction or advice you need."

[1511] As described above, the present invention is a comprehensive system for managing the health of employees and supporting their work in a logistics center, which can significantly improve employee safety and efficiency.

[1512] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1513] Step 1: Initial setup and device deployment

[1514] Overview: A user wears a wearable device and uses smart glasses or a head-mounted display. A camera is also installed in the work environment.

[1515] Specific operation: The user puts on the designated wearable device and connects it to smart glasses or a head-mounted display. The camera is appropriately positioned to cover the entire work area.

[1516] Input: Wearable devices, smart glasses, head-mounted displays, cameras.

[1517] Output: Ready state.

[1518] Step 2: Data collection and transmission

[1519] Overview: The terminal collects biometric data from wearable devices and monitors the work environment with a camera. This data is sent to a server in real time.

[1520] Specific operation: The wearable device collects biometric data such as heart rate, blood pressure, and activity level. The camera captures images of the work environment and generates video data.

[1521] Input: Biometric data from wearable devices, video data from cameras.

[1522] Output: Biometric data and video data sent to the server.

[1523] Step 3: Receiving and Preprocessing Data

[1524] Overview: The server receives biometric data and video data sent from the terminal and performs preprocessing.

[1525] Specific operation: The server receives data sent from the wearable device and camera, and then performs preprocessing such as resizing the image data and removing noise from the audio data.

[1526] Input: Transmitted biometric data, video data.

[1527] Output: Preprocessed data.

[1528] Step 4: Data analysis

[1529] Overview: The server analyzes the preprocessed data using a generative AI model to evaluate the user's health condition and work status.

[1530] Specific operations: Executes an algorithm to detect abnormal values ​​based on preprocessed biometric data. Analyzes video data to recognize the working environment and movements.

[1531] Input: Preprocessed biometric data, preprocessed video data.

[1532] Output: Health status assessment results, work situation assessment results.

[1533] Step 5: Health assessment and advice

[1534] Overview: The server integrates the analysis results and generates advice and instructions based on the user's health condition and work situation, which are then provided to the user in real time.

[1535] Specific operation: Based on the health status evaluation results, necessary health advice and work instructions are generated. These advice and instructions are sent to the terminal and notified to the user.

[1536] Input: Health status assessment results, work situation assessment results.

[1537] Output: The advice or instructions generated.

[1538] Step 6: Notifications on your device

[1539] Overview: The device receives advice and instructions from the server and notifies the user in real time.

[1540] Specific operation: The device provides advice and instructions to the user through voice and text messages, and also displays visual information through the smart glasses.

[1541] Input: Advice or instructions sent by the server.

[1542] Output: Advice or instructions communicated to the user.

[1543] Step 7: Emergency response

[1544] Abstract: When an abnormality is detected in biometric data, the server detects an emergency and takes appropriate action and notifies the user.

[1545] Specific operation: If an abnormality is detected in the biometric data, an emergency alert is immediately generated to notify the user, and at the same time, registered medical institutions and administrators are notified.

[1546] Input: Biometric data in which anomalies were detected.

[1547] Output: Emergency alert, notification.

[1548] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1549] This invention combines a system that monitors a user's health status, generates health advice, detects and responds to emergencies, and an emotion engine that recognizes the user's emotions. This system consists of a terminal installed in the user's living environment and a server that analyzes data and takes various actions.

[1550] 1. Initial Setup and Device Placement

[1551] User: A camera, microphone, various sensors, and an AI-driven robot (hereinafter referred to as the terminal) are installed in the user's home. This terminal is placed to cover the user's living space. It also has an emotion engine built in.

[1552] 2. Data collection and transmission

[1553] Device: Monitors the user's daily life, with a camera capturing the user's movements and facial expressions, and a microphone recording what the user says and environmental sounds. Sensors in wearable devices also collect biometric data such as the user's heart rate and number of steps taken. An emotion engine then analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[1554] 3. Data Reception and Preprocessing

[1555] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing, such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[1556] 4. Data Analysis

[1557] Server: The preprocessed data is input into the AI ​​model and analysis begins. For example, image data is fed into a facial recognition algorithm to recognize the user's facial expressions and movements, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[1558] 5. Health and emotional assessment and advice

[1559] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it makes a comprehensive assessment, including whether the user is likely to be feeling stressed or happy. Based on the assessment results, it generates appropriate health advice. For example, if the user is feeling stressed, it suggests that they should relax.

[1560] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[1561] 6. Emergency Response

[1562] Server: If an emergency situation is detected during the process of analyzing the user's data, such as if the user falls or if the emotion engine indicates a state of panic, the server will immediately initiate emergency response and automatically notify registered medical institutions and family members.

[1563] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[1564] Specific examples

[1565] Example 1: Daily monitoring and health advice

[1566] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[1567] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[1568] Example 2: Fall detection and emergency response

[1569] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[1570] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[1571] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1572] As described above, the present invention is a system that comprehensively manages a user's health and emotions, and provides appropriate advice and emergency responses, thereby improving the quality and safety of the user's life. This system consistently supports the user's life, from daily monitoring to emergency response.

[1573] The processing flow will be explained below.

[1574] Step 1:

[1575] Device: Cameras, microphones, and sensors installed in the user's living environment are activated and begin collecting data. For example, the camera captures video of the living room, and the microphone records the user's speech and environmental sounds. Also, wearable devices collect biometric data such as heart rate and number of steps.

[1576] Step 2:

[1577] Terminal: Temporarily stores collected text data, audio data, image data, and video data. For example, data is saved in batches every 60 seconds.

[1578] Step 3:

[1579] Terminal: Preprocesses the temporarily stored data and converts it into a format that can be sent. Specifically, it resizes image data to a size that is easy to analyze and removes noise from audio data.

[1580] Step 4:

[1581] Terminal: Sends pre-processed data to the server. A transmission schedule is set to ensure real-time and efficient data transfer.

[1582] Step 5:

[1583] Server: Receives data sent from the terminal and stores it in a database. It checks the integrity of the received data and issues a resend request if there are any errors.

[1584] Step 6:

[1585] Server: Inputs the received data into the AI ​​model and emotion engine and starts the analysis process. For example, image data is input into a facial recognition algorithm to detect the user's facial expression, and voice data is analyzed to recognize the content of speech and emotions.

[1586] Step 7:

[1587] Server: Integrates the analysis results and evaluates the user's health and emotional state. For example, it determines that the user looks happy and has a normal heart rate.

[1588] Step 8:

[1589] Server: Generates appropriate health advice based on the evaluation results. For example, if the user is feeling stressed, the server will suggest ways to relax.

[1590] Step 9:

[1591] Server: Sends the generated health advice to the device, verifies the integrity of the sent content, and ensures that the device receives it.

[1592] Step 10:

[1593] Device: Notifies the user of health advice received from the server. For example, it tells the user by voice, "Today, get some exercise and relax."

[1594] Step 11:

[1595] Server: If an emergency situation is detected during analysis, the server immediately initiates the emergency response protocol. For example, if the user falls or the emotion engine indicates that the user is panicking, it will be determined to be an emergency.

[1596] Step 12:

[1597] Server: Sends emergency notifications in real time to medical institutions and registered family members. For example, it sends a message saying, "The user has fallen. Urgent action is required."

[1598] Step 13:

[1599] Device: At the same time, an emergency notification is sent to the user via voice, such as "A fall has been detected. Help has been called, so please do not worry."

[1600] Step 14:

[1601] Server: After the emergency response, check the response results and whether follow-up is required. If necessary, continue monitoring until the user's condition stabilizes.

[1602] Example 2

[1603] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1604] Currently, many systems monitor users' health status and have the ability to detect emergencies, but few systems exist that analyze a user's emotional state in real time and provide health advice based on this analysis. Because emotional fluctuations have a significant impact on health status, comprehensive management that includes emotional state is desirable. Furthermore, when detecting and responding to emergencies, taking the user's emotional state into consideration contributes to faster and more appropriate responses. However, existing systems lack the means to integrate these factors.

[1605] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for monitoring the user's health condition, means for generating health advice based on the health condition, means for detecting and responding to emergencies, means for recording the user's movements and speech content using a camera and microphone and collecting biometric data using a sensor device, and means for analyzing the user's emotions in real time using an emotion engine. This makes it possible to comprehensively manage not only the user's health condition but also their emotional state, provide appropriate advice, and respond quickly to emergencies.

[1606] "Means for monitoring the user's health condition" refers to a device or system that uses a camera, microphone, sensor device, etc. to observe and record the user's biometric data, movements, and speech in real time.

[1607] The "means for generating health advice" is a device or program for analyzing the user's health status data and providing appropriate health suggestions and instructions.

[1608] "Means for detecting and responding to emergencies" refers to a device or system that monitors user data in real time, quickly issues an alert if an abnormality or danger occurs, and takes the necessary action.

[1609] "Means for recording user actions and speech using a camera and microphone" refers to a device or system that collects video and audio data and monitors user actions and speech in real time.

[1610] "Means for collecting biometric data using a sensor device" refers to a device or system that measures and records biometric indicators such as heart rate, body temperature, and number of steps in real time.

[1611] "Means for analyzing user emotions in real time using an emotion engine" refers to a device or program that analyzes the user's emotional state from facial expressions, tone of voice, etc., and obtains the results in real time.

[1612] "Means for collecting text data, audio data, image data, and video data" refers to a device or system that collects various types of data related to users and stores it for analysis.

[1613] "Means for analyzing the data to assess the user's health and emotional state" refers to a device or program that uses the collected data to determine the user's current health and emotional state and generate an assessment result.

[1614] A "means for providing appropriate health advice" is a device or system that provides actionable health suggestions or instructions to the user based on the assessment results.

[1615] "Means for early detection of an emergency" refers to a device or program that analyzes data in real time and detects signs of an emergency.

[1616] "Means for quickly notifying medical institutions when an emergency occurs" refers to a device or system that automatically notifies designated medical institutions and relevant parties when an emergency is confirmed.

[1617] MODE FOR CARRYING OUT THE INVENTION

[1618] This invention is a system that comprehensively monitors a user's health and emotional state, provides appropriate health advice, and responds quickly to emergencies. The system mainly consists of the following elements:

[1619] Hardware and Software Configuration

[1620] 1. Terminal placement:

[1621] User: The user installs cameras, microphones, various sensors, and an AI-driven robot in their home. The devices are arranged to cover the user's living space. The emotion engine is also initially configured.

[1622] 2. Data Collection:

[1623] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature, and the emotion engine analyzes the user's emotions in real time. The collected data is periodically sent to a server.

[1624] 3. Data reception and preprocessing:

[1625] Server: The server receives data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[1626] 4. Data Analysis:

[1627] Server: The preprocessed data is input into the AI ​​model for analysis. For example, image data is fed into a facial recognition algorithm, and voice data is analyzed to obtain the content and tone of speech. Emotional data recognized by the emotion engine is also integrated to understand the user's emotional state.

[1628] 5. Health and emotional assessment and advice:

[1629] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates appropriate health advice.

[1630] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[1631] 6. Emergency Response:

[1632] Server: If an emergency situation is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[1633] The device also has a function to directly notify the user of emergency responses, such as issuing a voice message saying, "A fall has been detected. We will call for help, so please don't worry."

[1634] Specific examples

[1635] Example 1: Daily monitoring and health advice

[1636] Device: The camera captures the living room and captures the user watching TV. The microphone captures the surrounding sounds and the sensor checks the user's heart rate. The emotion engine analyzes the user's state of relaxation.

[1637] Server: This data is sent to the server, where it is analyzed using the AI ​​model and emotion engine. It confirms that the user is relaxed and that there are no particular abnormalities, and stores the monitoring results in a database.

[1638] Example 2: Fall detection and emergency response

[1639] Device: The camera captures the user's fall. The fall data is sent to the server. The emotion engine also captures emotions such as fear and pain.

[1640] Server: Based on image analysis of the camera and emotional data, the server determines that the user has fallen and the urgency of the situation, and immediately notifies registered family members and the nearest medical institution, providing details along with location information.

[1641] Device: Notify the user via voice message: "A fall has been detected. Medical assistance has been called, so please rest assured."

[1642] This system allows users to comprehensively manage their health and emotional state, providing daily health advice and rapid response in emergencies.

[1643] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1644] Step 1: Initial setup and device deployment

[1645] User: Cameras, microphones, various sensors, and an AI-driven robot are placed in the user's home. The devices are installed to cover the entire living space of the user. The emotion engine is also initially configured and its operation is confirmed.

[1646] Input: The location and initial setting information for each device

[1647] Output: Device layout diagram after installation is complete and confirmation results of initial settings

[1648] Specific operation: Cameras and microphones are placed in the living room, bedroom, etc. to provide full coverage. A wearable device is worn by the user to periodically record biometric data such as heart rate and body temperature.

[1649] Step 2: Data collection and transmission

[1650] Device: The camera captures the user's movements and facial expressions, and the microphone records speech and environmental sounds. Sensors collect biometric data such as heart rate, steps taken, and body temperature. The emotion engine analyzes the user's emotions in real time. This data is periodically sent to the server.

[1651] Input: Camera footage, audio data, biometric data, emotional data

[1652] Output: Collected data is sent to the server

[1653] Specific operation: The camera periodically captures the user's image and performs facial recognition and facial expression analysis. The microphone records the user's speech and detects specific keywords in real time. The sensor compiles and logs heart rate and step count. The emotion engine analyzes the user's voice and facial expressions and generates emotion tags.

[1654] Step 3: Receiving and Preprocessing Data

[1655] Server: Receives data sent from the device and stores it in a database. After verifying the integrity of the received data, it performs preprocessing such as resizing image data, removing noise from audio data, and converting the format of emotion data.

[1656] Input: Raw data sent from the terminal

[1657] Output: Preprocessed data

[1658] What it does: The server receives the data packets and stores them in a database. Image data is resized to optimize storage efficiency. Audio data is filtered to extract clear speech. Emotion data is converted to a standard format and integrated with other data.

[1659] Step 4: Data analysis

[1660] Server: The preprocessed data is used by the AI ​​model for analysis. Image data is fed into a facial recognition algorithm, and audio data is analyzed to obtain speech content and tone. Emotion data recognized by the emotion engine is also integrated to understand the user's emotional state.

[1661] Input: Preprocessed data

[1662] Output: Analysis results (health status, emotional status)

[1663] How it works: Facial recognition algorithms analyze images to recognize the user's facial expressions. Voice analysis algorithms capture the text and tone of speech. Emotional state data is integrated over time to generate a daily emotional pattern.

[1664] Step 5: Health and Emotion Assessment and Advice

[1665] Server: Integrates the analysis results and evaluates the user's health and emotional state. Based on the evaluation results, it generates health advice. For example, it suggests ways to relax if the user is feeling stressed.

[1666] Terminal: Receives advice sent from the server and notifies the user. For example, it notifies the user by voice, "Today, do some exercise and relax."

[1667] Input: Analysis results

[1668] Output: Health advice

[1669] Specific operation: The health assessment algorithm analyzes various data and performs a comprehensive assessment. The generated health advice is sent to the device and notified to the user via voice message.

[1670] Step 6: Emergency response

[1671] Server: If an emergency is detected during the analysis of the user's data, the server automatically notifies registered medical institutions and family members.

[1672] Device: Directly notifies the user of emergency responses and provides necessary instructions. For example, a voice message will be issued saying, "A fall has been detected. We will call for help, so please don't worry."

[1673] Input: Emergency detection data

[1674] Output: Emergency call, user notification

[1675] How it works: The fall detection algorithm analyzes camera and sensor data to detect a fall. The emergency notification module automatically notifies designated contacts and explains the situation. The device then communicates the situation to the user via voice prompts and prompts them to take the necessary action.

[1676] (Application example 2)

[1677] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1678] Conventional health management systems monitor the health status of individual users and provide advice, but are not suitable for use in physical stores. Furthermore, there is a need to monitor the emotions of customers and store staff in real time and respond appropriately and immediately to emergencies. In such an environment, comprehensive monitoring is required, including not only health status but also emotional status.

[1679] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring the user's health state, means for generating health advice based on the health state, means for detecting and responding to emergencies, means for a terminal installed in the store to monitor the health states and emotions of customers and store employees and provide appropriate advice, and means for immediately notifying a medical institution or relevant parties when an emergency is detected. This enables real-time monitoring of the health and emotional states of customers and store employees in a physical store, realizing comprehensive health management and rapid emergency response.

[1680] A "user" is an individual who uses the system, and includes customers and store clerks.

[1681] "Health Status" refers to the state of a user's physical and mental health.

[1682] "Monitoring" refers to the continuous observation and recording of a user's health and emotional state using cameras, microphones, and various sensors.

[1683] "Advice" is advice or suggestions generated based on your health and emotional state, encouraging appropriate behavior.

[1684] An "emergency" refers to a physical or mental crisis that requires immediate attention.

[1685] "Responding" means taking appropriate action in response to a detected emergency.

[1686] "Terminal" refers to a device that includes a camera, microphone, various sensors, and an AI-driven robot, and is a device that collects and analyzes user data.

[1687] "Emotion" refers to a user's state of mind or emotional response.

[1688] An "emotion engine" refers to software that analyzes a user's emotional state from data such as facial expressions and voice.

[1689] "Data collection" refers to obtaining text data, audio data, image data, and video data from users.

[1690] "Assessment" refers to analyzing collected data to determine the user's health and emotional state.

[1691] "Analytics" refers to the technical means of processing data to understand and recognize the health and emotional state of the user.

[1692] "Server" is a centralized computing system that receives, pre-processes, and analyzes data to assess the user's health and emotional state.

[1693] "Healthcare Provider" means a health care provider contacted to address an emergency.

[1694] "Notification" means informing the appropriate parties and agencies of a detected emergency.

[1695] An "AI model" is an artificial intelligence technology used to analyze collected data and assess a user's health and emotional state.

[1696] "Brick and mortar store" means a business establishment that exists in a physical location and offers goods and services to customers.

[1697] The system of this invention monitors the health and emotional states of customers and store staff in physical stores, provides appropriate advice, and responds quickly in the event of an emergency. The system includes a terminal installed in the store, a server that performs data analysis, and an emotion engine.

[1698] 1. Initial Setup and Device Placement

[1699] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. These terminals are placed to cover the area within the store to monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[1700] 2. Data collection and transmission

[1701] The device monitors the user's daily activities, with a camera capturing the user's movements and facial expressions and a microphone recording the user's speech and environmental sounds. Additionally, various sensors collect environmental and biometric data, such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes the user's emotions in real time and periodically sends the collected data to a server.

[1702] 3. Data Reception and Preprocessing

[1703] The server receives data sent from the device and stores it in a database. At that time, it checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[1704] 4. Data Analysis

[1705] The server inputs the preprocessed data into the AI ​​model and begins the analysis process. The analysis process integrates a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine to grasp the user's health and emotional state. Based on the results, a comprehensive health assessment is performed.

[1706] 5. Health and emotional assessment and advice

[1707] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[1708] 6. Emergency Response

[1709] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message to inform emergency responders, such as "Please wait until medical staff arrive."

[1710] Specific examples

[1711] Daily monitoring and health advice

[1712] While a user is watching TV in their living room, a camera captures their movements, a microphone captures their voice, and a sensor checks their heart rate. The emotion engine analyzes the user's state of relaxation. The server analyzes this data, finds no abnormalities, and stores it in a database.

[1713] Fall Detection and Emergency Response

[1714] If a user falls in the store, the camera captures the incident and the emotion engine analyzes emotions such as fear and pain. The server recognizes this as an emergency and immediately notifies medical institutions and store staff. The device then issues a voice message saying, "Please wait until medical staff arrives."

[1715] Prompt Sentence Examples

[1716] When a customer falls in a store, cameras and sensors detect the fall, and an emotion engine analyzes the fear and pain. This information is analyzed by a server and determined to be an emergency. Medical staff are immediately notified automatically, and a robot in the store issues a voice message saying, "Please wait until medical staff arrives."

[1717] This invention enables comprehensive monitoring of the health and emotional state of customers and store staff in brick-and-mortar stores and emergency response.

[1718] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1719] Step 1:

[1720] Initial Setup and Device Deployment

[1721] Users (customers and store clerks) install cameras, microphones, various sensors, and AI-driven robots (hereafter referred to as terminals) in physical stores. The terminals are positioned to cover the area within the store where they monitor the health and emotional state of the customers. They also incorporate an emotion engine.

[1722] Specific behavior:

[1723] The locations for cameras, microphones, and sensors are determined and placed throughout the store. The emotion engine is installed on the robot and its operation is confirmed.

[1724] Input: Camera, microphone, various sensors, robot

[1725] Output: Installation complete

[1726] Step 2:

[1727] Data collection and transmission

[1728] The device monitors the user's daily activities, the camera captures the user's movements and facial expressions, and the microphone records the user's speech and environmental sounds. Various sensors collect environmental and biometric data such as heart rate, temperature, humidity, and light intensity. The emotion engine analyzes this data and periodically sends it to a server.

[1729] Specific behavior:

[1730] The system continuously captures video with a camera, records audio with a microphone, and collects biometric data with sensors, and transmits this data to a server at regular intervals.

[1731] Input: User movement data, speech data, environmental sounds, biometric data

[1732] Output: Collected dataset

[1733] Step 3:

[1734] Data reception and preprocessing

[1735] The server receives the data sent from the device and stores it in a database. It checks the integrity of the received data and performs preprocessing. Specifically, this includes resizing image data, removing noise from audio data, and converting the format of emotion data.

[1736] Specific behavior:

[1737] The server verifies the consistency of the data, resizes the image data to a standard size, removes background noise from the audio data, and converts the emotion data into a specified format.

[1738] Input: Collected dataset

[1739] Output: Preprocessed dataset

[1740] Step 4:

[1741] Data analysis

[1742] The server runs an AI model based on the preprocessed data and begins the analysis process. By integrating a facial recognition algorithm using image data, an analysis algorithm for voice data, and an emotion engine, the server can grasp the user's health and emotional state.

[1743] Specific behavior:

[1744] Facial recognition algorithms analyze facial expressions, tone is analyzed from audio data, and data is fed into an emotion engine to analyze emotional states.

[1745] Input: Preprocessed dataset

[1746] Output: Analyzed data and health / emotional state

[1747] Step 5:

[1748] Health and emotional assessment and advice

[1749] The server evaluates the user's health and emotional state based on the integrated analysis results. For example, if the user is feeling stressed, it generates advice on the need to relax. Notifications to the user are provided via the device using audio and video.

[1750] Specific behavior:

[1751] The server generates advice based on the analysis results and sends it to the device, which then notifies the user of the advice via audio or video.

[1752] Input: Parsed data

[1753] Output: Health and emotional state assessment results, advice notification

[1754] Step 6:

[1755] Emergency response

[1756] If the server detects an emergency while analyzing the user's data, it will immediately notify registered medical institutions and relevant parties. For example, if the user falls, it will send an email or phone call informing them of the user's location and the situation. The device will also issue a voice message saying, "Please wait until medical staff arrives."

[1757] Specific behavior:

[1758] When the server detects an emergency, it automatically contacts pre-registered medical institutions and family members, and the device notifies the user of the emergency response via voice.

[1759] Input: Parsed data

[1760] Output: Emergency notification, contact to medical institution

[1761] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1762] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1763] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1764] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1765] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1766] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1767] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1768] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1769] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1770] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1771] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1772] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1773] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1774] Alternativel...

Claims

1. a means for monitoring the health status of a user; means for generating health advice based on the health status; a means of detecting and responding to emergencies; A system including:

2. Means for collecting text data, audio data, image data, and video data; means for analyzing the data to assess the user's health condition; means for providing appropriate health advice based on the evaluation results; The system of claim 1 , comprising:

3. means for collecting biometric data from a user via a sensor device; means for analyzing the biometric data to detect an emergency situation at an early stage; A means for promptly notifying a medical institution when the emergency occurs; The system of claim 1 , comprising:

Images