System

Abstract

A system is provided.SOLUTION: A system comprising: a device for measuring biological information of a user; communication means for receiving measurement data from the device; analysis means for analyzing the received measurement data; notification means for providing health advice to the user based on a result of the analysis; and a messaging interface for transmitting the health advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people face health challenges due to issues such as aging, lifestyle-related diseases, and stress. Extending healthy lifespan, especially among the elderly, is an urgent issue, and concrete measures are needed to maintain long, independent lives. However, conventional health monitoring systems lack sufficient functionality for real-time health data collection and analysis and for providing individually tailored health advice. As a result, users' health management may be left inadequate, potentially leading to a deterioration in their health. The present invention aims to solve these issues and support users' health management. [Means for solving the problem]

[0005] The present invention provides a system including a device for measuring a user's biological information, a communication means for receiving measurement data from the device, an analysis means for analyzing the received measurement data, a notification means for providing the user with health advice based on the analysis results, and a messaging interface for transmitting the health advice. Specifically, the analysis means includes a function for analyzing the user's dietary data and calculating calories and nutritional balance. The notification means also includes a function for sending preventive alerts in real time based on the user's health status. The analysis means also includes a function for analyzing the user's sleep data and generating advice for improving sleep quality, and a function for analyzing the user's stress level and providing counseling for stress management. This realizes a system that allows users to perform comprehensive health management and extend their healthy lifespan.

[0006] "Biometric information" is data obtained from the user's body, and includes heart rate, sleep time, activity level, number of steps, and the like.

[0007] A "device" is a hardware device for measuring a user's biometric information, and includes wearable devices such as smart watches.

[0008] "Communication means" refers to a means for receiving data from a device that measures a user's biometric information and transmitting it to a cloud server or other device, and includes wireless communication such as Bluetooth and Wi-Fi.

[0009] "Analysis means" refers to software algorithms or hardware for processing and analyzing received biometric information, including AI and machine learning models.

[0010] "Notification means" refers to a means for conveying information to the user based on the analysis results, and includes push notifications and text messages.

[0011] A "messaging interface" is a platform for sending and receiving messages between users and the system, and includes messaging applications such as LINE and SMS.

[0012] "Health advice" is information that is useful to the user and is generated by the analysis means, and includes specific suggestions and improvements regarding diet, exercise, sleep, stress management, and the like.

[0013] A "preventive alert" is a notification sent when an abnormality in the user's health condition is detected, and includes a warning message to encourage early action.

[0014] "Dietary data" is information about meals consumed by the user, and includes information recorded as photographs of meals and notes.

[0015] "Sleep data" is information about the user's sleep status, including the amount of sleep time, the ratio of deep sleep to light sleep, and the like.

[0016] "Stress level" is an index that indicates the user's stress state and is evaluated based on self-reporting and sensor data.

[0017] "Counseling" refers to the provision of interactive advice to support users' mental health, including through chatbots and expert consultations. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0040] System Configuration

[0041] The system includes the following main components to collect and analyze a user's biometric information and provide health advice:

[0042] 1. User device (smartwatch)

[0043] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0044] 3. Cloud Server

[0045] 4. User device (smartphone app)

[0046] 5. Messaging interface (e.g., LINE)

[0047] Program processing

[0048] Data collection

[0049] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[0050] Data Transfer

[0051] The data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user takes a photo of their meal and enters the meal data into the app, it is also sent to the cloud server.

[0052] Data analysis

[0053] The cloud server analyzes all the data it receives. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Dietary data is then analyzed using image recognition technology to calculate calories and nutritional balance.

[0054] Generating health advice

[0055] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if the user's heart rate or stress level is abnormal, it will suggest measures to improve the situation or relaxation methods.

[0056] Real-time notifications

[0057] The cloud server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[0058] Supporting preventative care and access to healthcare

[0059] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user has consistently been experiencing insufficient sleep, it will send a notification saying, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[0060] Specific examples

[0061] 1. Data Collection and Transfer

[0062] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0063] Terminal (smartphone app): The user takes a photo of the meal and the data is sent to the cloud server via the smartphone app.

[0064] 2. Data analysis and health advice generation

[0065] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0066] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[0067] 3. Real-time notifications and preventative care

[0068] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[0069] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0070] 4. Supporting access to medical care

[0071] Device (smartphone app): The user opens the app, selects from the provided list of medical institutions, and makes an appointment.

[0072] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] Device (Smartwatch): The user wears a smartwatch to monitor their heart rate, steps, sleep, and activity. The smartwatch continuously measures these data and temporarily stores them in the device's internal memory.

[0076] Step 2:

[0077] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi at pre-set intervals, such as every 10 or 30 minutes.

[0078] Step 3:

[0079] Terminal (smartphone app): The smartphone imports the received data into the app and uploads it to the cloud server. The user is notified when the upload is complete.

[0080] Step 4:

[0081] User: When eating a meal, the user takes a photo of the meal using a smartphone app and adds a brief note (including ingredients and the name of the dish). After adding the information, the data is sent to the cloud server.

[0082] Step 5:

[0083] Server: The cloud server stores the received biometric information and dietary data and saves it in an analysis database. The database also records hourly data.

[0084] Step 6:

[0085] Server: Analyzes the stored data in real time. For heart rate data, it calculates the average heart rate and maximum and minimum heart rates over the past 24 hours, and for sleep data, it analyzes the quality of sleep (the ratio of deep sleep to light sleep).

[0086] Step 7:

[0087] Server: Analyzes photos of meal data using image recognition technology to identify ingredients. Retrieves the calories and nutrients of ingredients from a database and calculates total calorie and macronutrient intake.

[0088] Step 8:

[0089] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if excessive calorie intake is detected, it will suggest ingredients and recipes to reduce calorie intake.

[0090] Step 9:

[0091] Server: The generated health advice is sent to the user in real time via a messaging interface such as LINE. For example, a message such as "Since you haven't exercised much today, try doing 15 minutes of stretching" is delivered.

[0092] Step 10:

[0093] Server: Analyzes user data over a long period of time and generates preventative alerts if signs of a worsening health condition are detected. For example, if a user has not had enough sleep for several consecutive days, a notification will be sent stating that "consultation with a medical institution is recommended."

[0094] Step 11:

[0095] Device (smartphone app): The user receives a preventive alert, opens the app, searches for nearby medical institutions within the app, selects an appropriate medical institution, and makes an appointment.

[0096] Through these steps, users can understand their health status in real time and receive appropriate health advice, enabling them to manage their health and take preventative medicine.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Health management has become increasingly important in recent years, and health management systems that utilize biometric information have been attracting attention. However, existing systems are limited in the amount of biometric information they can acquire, making it difficult to achieve comprehensive health management for users. Furthermore, dietary data acquisition and analysis are insufficient, making it difficult to properly evaluate a user's nutritional balance. Furthermore, real-time health advice and preventative alerts are often not provided, hindering users' prompt health management.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes a device for measuring a user's biometric information, memory means for temporarily storing the data, communication means, analysis means, image recognition means, notification means, and a messaging interface, which enable comprehensive collection and management of biometric information, detailed analysis of dietary data, and provision of real-time health advice and preventative alerts.

[0102] "Devices that measure biological information" are devices that measure biological data such as a user's heart rate, sleep time, and activity level, and generally refer to smartwatches and fitness trackers with heart rate monitoring functions.

[0103] "Memory means" refers to a storage device for temporarily storing measured biometric information, and corresponds to the internal memory of a smartwatch or smartphone.

[0104] "Communication means" refers to a device for transmitting measured data to other devices or servers, and includes wireless communication technologies such as Bluetooth and Wi-Fi.

[0105] "Analysis means" refers to means for analyzing received measurement data, and includes artificial intelligence algorithms and data analysis software.

[0106] "Image recognition means" refers to means for analyzing dietary data, and includes image analysis technology for analyzing photos of meals to calculate calories and nutrients.

[0107] "Notification means" refers to a means for providing health advice to users based on the analysis results, and includes smartphone notifications and message sending functions.

[0108] "Messaging interface" refers to a communication means for sending health advice to users, and includes messaging applications such as LINE and email.

[0109] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments for carrying out the present invention will be described in detail below.

[0110] System configuration

[0111] The system includes the following main components for collecting and analyzing a user's biometric information and providing health advice:

[0112] 1. User device (smartwatch)

[0113] This is a device that measures biometric information such as heart rate, sleep time, and activity level.

[0114] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0115] It is a means for transmitting measurement data to other devices or servers.

[0116] 3. Cloud Server

[0117] This server analyzes the received data and provides health advice to users. The analysis uses artificial intelligence algorithms and image analysis technology.

[0118] 4. User device (smartphone app)

[0119] It is a means for uploading data transferred from a smartwatch to a cloud server.

[0120] 5. Messaging interface (e.g., LINE)

[0121] It is a means for sending health advice to users.

[0122] Program processing

[0123] Data collection

[0124] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[0125] Data Transfer

[0126] The data stored on the device (smartwatch) is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi. The device (smartphone app) then uploads the data to a cloud server. When the user takes photos of their meals and enters them into the app, the meal data is also sent to the cloud server.

[0127] Data analysis

[0128] The server analyzes all the data it receives. Heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies. Dietary data is analyzed using image recognition technology to calculate calories and nutritional balance.

[0129] Generating health advice

[0130] The server generates optimal health advice for the user based on the analysis results. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if heart rate or stress levels are abnormal, it will suggest measures to improve or relaxation methods.

[0131] Real-time notifications

[0132] The server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[0133] Supporting preventative care and access to healthcare

[0134] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has consistently been experiencing insufficient sleep, the server will send a notification saying, "We recommend that you consult a medical institution." The user can easily make an appointment with a medical institution using the device (smartphone app).

[0135] Specific examples

[0136] 1. Data Collection and Transfer

[0137] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0138] Users take photos of their meals and the data is sent to a cloud server via a smartphone app.

[0139] 2. Data analysis and health advice generation

[0140] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0141] Server: Analyzes dietary data, identifies nutrient deficiencies, and suggests corresponding ingredient lists and recipes.

[0142] 3. Real-time notifications and preventative care

[0143] Server: Sends a message to the user via LINE, such as "We recommend walking for 20 minutes today."

[0144] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0145] 4. Supporting access to medical care

[0146] The user opens the smartphone app, selects from the provided list of medical institutions, and makes an appointment.

[0147] Examples of prompt statements

[0148] Prompt text when user enters meal data:

[0149] Take a photo of what you had for dinner last night and send it to us along with the details of the meal. For example, write "Salad, steak, and rice."

[0150] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[0151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0152] Step 1:

[0153] Input: The user puts on the smartwatch.

[0154] How it works: When a user wears a smartwatch with heart rate monitoring throughout the day, it automatically starts collecting heart rate, vital signs, and sleep data. The data is temporarily stored in the smartwatch's internal memory.

[0155] Output: Temporarily stored heart rate, vital activity and sleep data.

[0156] Step 2:

[0157] Input: Biometric data stored on the smartwatch.

[0158] Specific operation: The device (smartwatch) periodically transfers data to the user's smartphone via Bluetooth or Wi-Fi. During the transfer process, the user checks the smartphone's connection and pairs it with the smartwatch if necessary.

[0159] Output: Biometric data transmitted to a smartphone.

[0160] Step 3:

[0161] Input: Biometric data transferred to smartphone.

[0162] Specific operation: The device (smartphone app) uploads all biometric data, including data entered into the app by the user after taking photos of their meals, to a cloud server. The user can initiate the upload by pressing the "Data Sync" button in the app.

[0163] Output: Biometric data and dietary data uploaded to a cloud server.

[0164] Step 4:

[0165] Input: Biometric data and dietary data uploaded to a cloud server.

[0166] How it works: The server uses AI algorithms to analyze the received data. It analyzes the heart rate data to detect the average heart rate over 24 hours and fluctuations in heart rate during sleep. It also uses image recognition technology to calculate calories and nutritional balance from the dietary data.

[0167] Output: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[0168] Step 5:

[0169] Input: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[0170] How it works: Based on the analysis results, the server generates optimal health advice for the user. For example, if your heart rate is too high, it will suggest relaxation techniques, or if you are lacking certain nutrients, it will suggest the necessary ingredients and recipes.

[0171] Output: The generated health advice.

[0172] Step 6:

[0173] Input: Generated health advice.

[0174] Specific operation: The server notifies the user of health advice in real time, sending messages such as "We recommend walking for 20 minutes today" via a messaging interface such as LINE.

[0175] Output: Health advice notification sent to user.

[0176] Step 7:

[0177] Input: Analysis results of long-term biometric data.

[0178] Specific operation: The server analyzes data over a long period of time, and if it detects an abnormality, such as persistent lack of sleep, it generates and sends a preventative alert such as "We recommend that you consult a medical institution."

[0179] Output: Proactive alerts sent to users.

[0180] Step 8:

[0181] Input: Medical consultation advice sent to the user.

[0182] Specific operation: The user opens the smartphone app and makes an appointment by selecting from the provided list of medical institutions. The user selects the "Medical Institution Appointment" tab in the app and specifies the desired date and time.

[0183] Output: Medical appointment.

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] This invention relates to a system for monitoring worker health and ensuring a safe working environment. In particular, it aims to prevent health problems caused by overwork and stress by monitoring workers' biological information in real time and providing appropriate health advice and preventive measures based on the results. Another objective is to improve worker productivity and safety by improving the working environment.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes a means for measuring a user's biometric information, a means for receiving measurement data from the device, a means for analyzing the received measurement data, a means for providing health advice to the user based on the analysis results, a means for transmitting the health advice, a means for monitoring the worker's health status based on the user's biometric information, a means for improving the work environment based on the monitoring results, and a means for displaying the advice in real time. This allows for continuous monitoring of the worker's health status and immediate notification of any abnormalities. Furthermore, by providing appropriate health advice and preventive alerts, health problems caused by overwork and stress can be prevented and a safe work environment can be maintained.

[0189] "User" means an individual who utilizes the System to provide biometric information and receive health monitoring and advice.

[0190] "Biometric information" refers to data about your physical condition, such as your heart rate, activity level, and body temperature.

[0191] A "measuring device" is a device for measuring a user's biometric information, and includes, for example, a smartwatch or fitness tracker.

[0192] "Communication means" refers to a means for transmitting data acquired from a measuring device to the system, and may use Bluetooth, Wi-Fi, etc.

[0193] The "analysis means" is a means for evaluating and analyzing the health condition based on the received biometric information data.

[0194] "Notification means" refers to a means for notifying the user of health advice and warnings based on the analysis results.

[0195] A "messaging interface" is a means of communication for sending notifications and advice to users, and includes, for example, LINE and email.

[0196] "Monitoring means" refers to a means for continuously checking the health status of a user based on their biometric information and detecting any abnormalities or changes.

[0197] "Work environment improvement measures" are measures to provide users with specific advice and measures to improve their work environment based on the monitoring results.

[0198] A "display device" is a device that displays advice and warnings to users in real time, and includes digital signs in factories and smartphones.

[0199] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0200] System Configuration

[0201] The system includes the following main components to collect and analyze users' biometric information and provide advice to improve the working environment:

[0202] 1. Measuring device (smartwatch): worn by the user.

[0203] 2. Communication means (Bluetooth, Wi-Fi): Receives and transfers measurement data.

[0204] 3. Local server: Installed within the factory.

[0205] 4. Cloud server: Analyzes the data.

[0206] 5. Smartphone app: Notifies users.

[0207] 6. Display device (digital signage): displays advice in real time.

[0208] 7. Robots: Monitor the health of workers in factories.

[0209] Program processing

[0210] Data collection

[0211] Once the user wears the smartwatch, biometric data collection begins: the smartwatch continuously measures the user's heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory.

[0212] Data Transfer

[0213] The collected data is transferred to a local server via Bluetooth or Wi-Fi, and then uploaded from the local server to a cloud server.

[0214] Data analysis

[0215] The cloud server analyzes all received data and uses AI algorithms (using TensorFlow) to detect patterns and abnormalities in biometric information.

[0216] Generating Advice

[0217] Based on the analysis results, the cloud server generates optimal health advice for the user, for example, recommending rest if stress levels are high.

[0218] Real-time notifications

[0219] The cloud server then sends the generated health advice to users in real time via a smartphone app or digital signage.

[0220] Supporting preventative care and access to healthcare

[0221] The cloud server analyzes long-term data and sends preventative alerts as needed, including automatic notification to medical institutions if necessary.

[0222] Hardware and software used

[0223] Hardware:

[0224] Smartwatch

[0225] Local Server

[0226] Digital signage in factories

[0227] robot

[0228] software:

[0229] Android SDK (smartphone app development)

[0230] TensorFlow (AI algorithm implementation)

[0231] Google Cloud (Cloud Analytics)

[0232] Kubernetes (cloud server management)

[0233] Bluetooth API (data transfer)

[0234] Specific example explanation

[0235] Factory workers wear smartwatches, and robots collect their biometric data. This data is transmitted to a local server via Bluetooth and then uploaded to a cloud server. AI algorithms on the cloud server analyze the data and monitor the workers' health. If any abnormalities are detected, they are notified in real time via digital signs in the factory and a smartphone app.

[0236] Example prompt sentence:

[0237] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[0238] In this way, it is possible to provide real-time care for workers in factories and provide a safe and efficient working environment.

[0239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0240] Step 1:

[0241] When the user wears the smartwatch, biometric data collection begins. The smartwatch continuously measures data such as heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory. The input is the user's physical data, and the output is the data stored in the smartwatch's internal memory.

[0242] Step 2:

[0243] The data stored in the smartwatch is transferred to a local server via Bluetooth or Wi-Fi. Specifically, the smartwatch batch processes the measurement data at regular intervals and transmits the data through communication with the local server. The input is the data in the smartwatch's internal memory, and the output is the data transferred to the local server.

[0244] Step 3:

[0245] The local server temporarily stores the received data and uploads it to the cloud server via the Internet. Data processing on the local server includes data formatting and format conversion. The input is the data transferred to the local server, and the output is the data uploaded to the cloud server.

[0246] Step 4:

[0247] The cloud server analyzes the received data using an AI algorithm (using TensorFlow). Specifically, it performs pattern recognition and anomaly detection. The input is the biometric data uploaded to the cloud server, and the output is the analysis results, i.e., an evaluation of health status and anomaly detection results.

[0248] Step 5:

[0249] The cloud server generates health advice and warnings for the user based on the analysis results. Specifically, it uses a generative AI model to create appropriate messages based on the analysis results. The input is the analysis results, and the output is health advice and warning messages.

[0250] Step 6:

[0251] The generated health advice and warning messages are notified to users in real time using a smartphone app or digital signs in the factory. Specifically, the cloud server issues the notifications and sends them to each device via a messaging interface. The input is the health advice or warning message, and the output is the notification sent to the user.

[0252] Step 7:

[0253] The cloud server accumulates long-term data, periodically analyzes it, and sends preventive alerts to users. This includes trend analysis of past data and anomaly detection. The input is long-term accumulated biometric data, and the output is preventive alerts.

[0254] A specific example includes steps to generate the following prompt statement if the user's heart rate increases by more than 20% of normal:

[0255] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0258] System Configuration

[0259] The system includes the following main components to collect and analyze users' biometric and emotional data and provide health advice:

[0260] 1. User device (smartwatch)

[0261] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0262] 3. Cloud Server

[0263] 4. User device (smartphone app)

[0264] 5. Messaging interface (e.g., LINE)

[0265] 6. Emotion Engine

[0266] Program processing

[0267] Data collection

[0268] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory. In addition, a smartphone app collects emotional data using the user's facial photos and voice data.

[0269] Data Transfer

[0270] The biometric data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user enters photos of their meals or self-reported data into the app, the data is also sent to the cloud server.

[0271] Data analysis

[0272] The cloud server analyzes all received data. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Food data is analyzed using image recognition technology to calculate calories and nutritional balance. The emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies.

[0273] Generating health advice

[0274] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. If the emotion engine evaluates a user's stress level as high, it will suggest relaxation methods and mental care advice. The advice content is also customized based on the user's emotional state.

[0275] Real-time notifications

[0276] The cloud server then sends the generated health advice to the user in real time via messaging interfaces such as LINE. For example, if the emotion engine detects that the user's stress level is high, it will send a message such as "Take 10 deep breaths and relax."

[0277] Supporting preventative care and access to healthcare

[0278] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user experiences insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[0279] Specific examples

[0280] 1. Data Collection and Transfer

[0281] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0282] Device (smartphone app): The user takes a photo of their meal and the data is sent to a cloud server via the smartphone app. The app also captures and records a photo of the user's face and a short voice message, which collects emotional data.

[0283] 2. Data analysis and health advice generation

[0284] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0285] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[0286] Server and Emotion Engine: The emotion engine analyzes facial photos and audio data to assess the user's emotional state, for example, assessing stress levels based on the frequency of smiles and tone of voice.

[0287] 3. Real-time notifications and preventative care

[0288] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[0289] Server: If the user's emotional state is unstable, send a notification saying "Take 10 deep breaths and relax."

[0290] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0291] 4. Supporting access to medical care

[0292] Device (smartphone app): When the user receives a prevention alert and opens the app, they search for nearby medical institutions, select an appropriate medical institution, and make an appointment.

[0293] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[0294] The processing flow will be explained below.

[0295] Step 1:

[0296] Device (smartwatch): The user wears a smartwatch to measure data such as heart rate, number of steps, sleep time, and activity level. The smartwatch continuously measures this data and stores it in its internal memory for a certain period of time (for example, one day or several hours).

[0297] Step 2:

[0298] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi. The transfer occurs automatically at specific time intervals, and the data is deleted from the smartwatch's internal memory after transfer.

[0299] Step 3:

[0300] Device (smartphone app): The smartphone app uploads the data received from the smartwatch to the cloud server. When the upload is complete, the user is notified. Also, when the user eats a meal, the smartphone app takes a photo of the meal, adds a brief note (including ingredients and the name of the dish), and sends this to the cloud server.

[0301] Step 4:

[0302] User: Emotion data is collected by users taking photos of their faces and recording voice messages using a smartphone app. These data are also sent to the cloud server.

[0303] Step 5:

[0304] Server: The cloud server securely stores all data received from the smartwatch and smartphone, and saves it in a database for analysis, along with timestamps and user identification information.

[0305] Step 6:

[0306] Server: The cloud server performs the data analysis. Heart rate data is analyzed by an AI algorithm to calculate the average heart rate, resting heart rate, maximum and minimum heart rate over the past 24 hours, and detect abnormalities. Sleep data is also calculated to evaluate the ratio of deep sleep to light sleep and evaluate sleep quality.

[0307] Step 7:

[0308] Server: When analyzing meal data, image recognition technology is used to identify ingredients in photos of meals, and the calories and major nutrients of each ingredient are retrieved from a database to calculate total calories and nutritional balance.

[0309] Step 8:

[0310] Emotion engine: The emotion engine on the cloud server analyzes facial photos and voice data to assess the user's emotional state. For example, it calculates the user's stress level and emotional tendencies (happiness, sadness, anger, etc.) through facial expression analysis and tone of voice analysis.

[0311] Step 9:

[0312] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if a user is lacking in nutrients, the server will provide advice such as, "You are lacking in vitamin D, so try incorporating fish and mushrooms into your diet." Based on the results from the emotion engine, if the user's stress level is high, the server will suggest relaxation methods.

[0313] Step 10:

[0314] Server: The cloud server generates health advice and sends it to the user in real time via a messaging interface such as LINE. For example, a notification such as "You haven't exercised much today, so try stretching for 15 minutes" is sent.

[0315] Step 11:

[0316] Server: The cloud server analyzes the user's data over a long period of time and generates preventive alerts if signs of a deterioration in health are detected. For example, if the user has not had enough sleep for several days in a row, the server will send a message saying, "We recommend that you consult a doctor."

[0317] Step 12:

[0318] Device (smartphone app): The user receives a preventive alert and opens the smartphone app. Within the app, they search for a nearby medical institution, select an appropriate medical institution, and make an appointment. Once the appointment is completed, a reminder notification is sent to the user.

[0319] In this way, the system continuously monitors the user's health and emotional data and provides appropriate advice and preventative care in real time, helping to improve the user's health and quality of life.

[0320] Example 2

[0321] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0322] Current health management systems primarily measure and analyze biometric information, but few systems comprehensively analyze users' emotional and dietary data to provide health advice. Furthermore, there is a lack of real-time preventive alert notifications based on the user's health and emotional state, making it difficult for users to receive timely health improvement measures.

[0323] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0324] In this invention, the server includes a communication means for receiving measurement data from a device that measures the user's biological information, an analysis means for analyzing the received measurement data and the user's facial photograph and voice data, and a messaging interface for providing and transmitting health advice to the user in real time based on the analysis results. This makes it possible to comprehensively analyze the user's biological information, dietary data, and emotional state, provide the user with optimal health advice in real time, and promptly notify the user of preventive medical alerts.

[0325] "User's biological information" refers to data measured directly from the user's body, such as the user's heart rate, sleep duration, and activity level.

[0326] "Measurement data" refers to data collected by various sensors, including biometric information and emotional data of the user.

[0327] "Communication means" refers to technology that has the function of transferring data from devices such as smartwatches and smartphone apps to cloud servers, and includes Bluetooth and Wi-Fi.

[0328] "Analysis means" refers to the AI ​​algorithms and analysis programs used to analyze data received on the cloud server.

[0329] A "user's face photo" is image data of the user's face taken using a smartphone app.

[0330] "Voice data" refers to data of a user's voice recorded using a smartphone app.

[0331] "Health advice" refers to recommendations and advice regarding health management that is provided to the user based on the results of analysis by the analysis means.

[0332] "Notification means" refers to technology that has the function of notifying users of the generated health advice, and includes messaging services such as LINE.

[0333] A "messaging interface" is an interface for sending notifications from a cloud server to a user in real time, and includes messaging services that use APIs.

[0334] "Dietary data" refers to photos of meals taken by the user and data on the contents of meals self-reported by the user.

[0335] "Calories" refers to the amount of energy calculated from dietary data.

[0336] "Nutritional balance" refers to the proportion of nutrients contained in food.

[0337] A "preventive alert" is a notification of preventive medical advice or caution generated by the analysis means.

[0338] MODE FOR CARRYING OUT THE INVENTION

[0339] The following describes in detail the specific embodiments of the present invention. The present invention is a system for collecting and analyzing biometric and emotional data of a user and providing health advice. The main components of the system are as follows:

[0340] 1. User device (smartwatch)

[0341] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0342] 3. Cloud Server

[0343] 4. User device (smartphone app)

[0344] 5. Messaging interface (e.g., LINE)

[0345] 6. Emotion Engine

[0346] Data collection

[0347] When a user wears a smartwatch, the collection of biometric information begins. The device (smartwatch) continuously measures the user's heart rate, sleep time, and activity level, and stores the data in its internal memory. In addition, the device (smartphone app) collects emotional data using the user's facial photos and voice data. For example, if a user wears a smartwatch with a heart rate monitoring function all day, heart rate data is automatically recorded. In addition, the user can take photos of their meals, and the data is sent to a cloud server via the smartphone app.

[0348] Data Transfer

[0349] The biometric data stored in the device (smartwatch) is periodically transferred to the device (smartphone app) via Bluetooth or Wi-Fi. The smartphone app then uploads this data to a cloud server. For example, the smartphone app periodically opens a Bluetooth connection to receive data and sends the data to the cloud server using the HTTP protocol.

[0350] Data analysis

[0351] The server receives the data sent to the cloud and analyzes it using AI algorithms. For example, heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies in the data. Meal data is used to calculate calories and nutritional balance using image recognition technology. An emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies. Specifically, the server analyzes the data using an AI algorithm built in Python and stores the results in a database. Examples of prompts include "Analyze today's heart rate data and tell me if there are any abnormalities" and "Calculate calories and nutritional balance from this meal photo."

[0352] Generating health advice

[0353] The server generates optimal health advice for the user based on the results of data analysis. For example, if a user is lacking in a particular nutrient, the server will suggest corresponding ingredients and recipes. If the user's stress level is high, relaxation methods will be suggested. Specifically, the server uses a pre-configured rule-based engine to select appropriate advice from a database. An example of a prompt could be, "Please assess the user's emotional state based on this facial photo and voice data."

[0354] Real-time notifications

[0355] The server sends the generated health advice to the user via a messaging interface such as LINE. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent. Specifically, the server uses the LINE API to generate a message and send it to the user. An example of a prompt is "Generate health advice from long-term sleep data."

[0356] Supporting preventative care and access to healthcare

[0357] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has had insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app. Specifically, the smartphone app uses GPS to list nearby medical institutions and connects with the reservation system to complete the appointment.

[0358] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[0359] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0360] Step 1: The user puts on the smartwatch.

[0361] Input: User's biological information (heart rate, sleep time, activity level)

[0362] How it works: The smartwatch's sensors measure the user's pulse and record it as heart rate data, while the accelerometer measures activity and estimates sleep time.

[0363] Output: Biometric data stored in internal memory

[0364] Step 2: The user takes a photo of their face using a smartphone app and records a voice message.

[0365] Input: User's face photo data, voice data

[0366] What it does: The smartphone camera takes a picture of the user's face, the microphone records their voice, and the app temporarily stores this data.

[0367] Output: Temporarily saved face photo data and audio data

[0368] Step 3: Transfer the biometric data stored on the smartwatch to your smartphone.

[0369] Input: Biometric data stored in the smartwatch's internal memory

[0370] What it does: Transfers smartwatch data to your smartphone via Bluetooth or Wi-Fi connection.

[0371] Output: Biometric data stored on a smartphone

[0372] Step 4: The smartphone app uploads all collected data to the cloud server.

[0373] Input: Biometric data, facial photo data, and voice data stored on the smartphone

[0374] Specific operation: The smartphone app sends data to the cloud server using the HTTP protocol.

[0375] Output: Biometric data, facial photo data, and voice data stored on a cloud server

[0376] Step 5: The cloud server analyzes the received data.

[0377] Input: Biometric data, facial photo data, and voice data stored on the cloud server

[0378] How it works: An AI algorithm on a cloud server analyzes heart rate data to detect 24-hour average heart rate and abnormalities. Image recognition technology is used to calculate calories and nutritional balance from photos of meals. An emotion engine also analyzes facial photos and voice data to assess stress levels and emotional states.

[0379] Output: Analysis results (health status, nutritional balance, emotional state)

[0380] Step 6: The cloud server generates health advice based on the analysis results.

[0381] Input: Analysis results (health status, nutritional balance, emotional state)

[0382] How it works: The cloud server uses a pre-configured rule-based engine to generate appropriate health advice.

[0383] Output: Generated health advice

[0384] Step 7: The cloud server notifies the user of the generated health advice in real time.

[0385] Input: Generated health advice

[0386] Specific operation: The cloud server generates a message using the LINE API and sends it to the user. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent.

[0387] Output: Health advice sent to the user

[0388] Step 8: The cloud server analyzes the long-term data and sends proactive alerts as needed.

[0389] Input: Long-term biometric data, emotional data

[0390] How it works: The cloud server analyzes past data trends and detects chronic problems. For example, if you experience insufficient sleep for several consecutive days, it generates a notification suggesting that you seek medical advice.

[0391] Output: Preventive alerts sent to users

[0392] Step 9: The user makes an appointment with a medical institution using the smartphone app.

[0393] Input: User's current location, preventative alerts

[0394] Specific operation: The smartphone app uses GPS to search for nearby medical institutions and connects with the reservation system to complete the appointment.

[0395] Output: Information about the medical institution where the reservation was completed

[0396] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice in real time, and quickly access medical care if needed.

[0397] (Application example 2)

[0398] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0399] Conventional health management systems provide health advice based on a user's biometric information, but do not offer service suggestions based on the customer's real-time emotional state, making it difficult to improve customer satisfaction, especially in brick-and-mortar stores. Furthermore, they lack the functionality to optimize services by utilizing in-store environmental data. This invention aims to solve these problems by providing a system that collects and analyzes customer emotional and environmental data and offers service suggestions based on that data.

[0400] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for measuring the user's biological information, means for receiving measurement data and environmental data from the device, means for analyzing the received measurement data and environmental data, means for providing health advice and service suggestions to the user based on the analysis results, and a messaging interface for transmitting the health advice and service suggestions. This makes it possible to improve the efficiency of store operations and customer satisfaction.

[0401] "Biometric information" refers to data relating to the user's health condition and bodily functions, and specifically includes heart rate, sleep duration, activity level, and the like.

[0402] "Communication means" refers to a means for transmitting measurement data from a device that measures biological information to other devices or servers, and utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[0403] The "analysis means" is a technology that processes the received measurement data and environmental data to evaluate and judge the user's health and emotional state, and uses artificial intelligence and machine learning algorithms.

[0404] "Notification means" refers to technology for notifying users and staff in real time of appropriate health advice and service suggestions based on the analysis results.

[0405] A "messaging interface" is a means of communication for conveying notifications to users and staff, and utilizes messaging applications such as LINE and WhatsApp.

[0406] "Service suggestions" are specific instructions and advice provided based on the user's emotional state and health status obtained by the analysis means, and are intended to improve the quality of service in physical stores.

[0407] "Environmental data" refers to information about factors that affect customer comfort and service provision, such as temperature, humidity, lighting, and foot traffic within the store.

[0408] "Real-time notification" is a technology that quickly conveys information to users and staff based on results obtained instantly by analytical means.

[0409] The present invention is a system for collecting and analyzing biometric and emotional data of users, and providing health advice and service suggestions based on the collected data. The system aims to improve customer satisfaction in brick-and-mortar stores and consists of several main components.

[0410] System Configuration

[0411] The main components of the system are:

[0412] 1. User device (smart glasses)

[0413] 2. Communication method (Wi-Fi)

[0414] 3. Cloud Server

[0415] 4. Notification devices (smartphone apps, messaging apps)

[0416] 5. Analysis engine (emotion engine and environmental data engine)

[0417] How to use

[0418] Data collection

[0419] When users (customers) wear smart glasses, their facial photos and facial expression data are collected by a camera. In-store environmental data (temperature, humidity, and traffic flow) is collected by sensors, and this data is transferred from the smart glasses to a cloud server in real time.

[0420] Data analysis

[0421] The cloud server analyzes the received biometric information and environmental data. It uses the Face Recognition API and Sentiment Analysis API to collect user emotional data from facial photos and facial expression data. It analyzes environmental data obtained from IoT devices and adjusts the store environment as needed.

[0422] Providing health advice and service suggestions

[0423] The cloud server generates advice and service suggestions for the user's health condition based on the analyzed data. Specifically, if the user is feeling stressed, it will provide advice on relaxation methods and mental care. In addition, store staff will be notified of the customer's service suggestions in real time.

[0424] Real-time notifications

[0425] The cloud server then sends the generated health advice and service suggestions to a smartphone app or messaging app via Wi-Fi. For example, if a customer is detected as feeling stressed, a specific service suggestion such as "Please serve hot tea to help the customer relax" can be sent to store staff.

[0426] Hardware and software used

[0427] Hardware: Smart glasses (camera, built-in sensors), IoT sensors (temperature, humidity, people flow sensors)

[0428] Software: Face Recognition API, Sentiment Analysis API, IoT Connectivity (Arduino, Raspberry Pi)

[0429] Cloud Platform: AWS (Amazon Web Services), Azure

[0430] Specific examples and examples of prompts for generative AI models

[0431] Examples:

[0432] If a customer smiles at the entrance, "The customer is relaxed. Please maintain the atmosphere in the store."

[0433] If the customer looks unhappy, say, "The customer's stress level is high. I suggest offering them a welcome drink."

[0434] Example prompts for generative AI models:

[0435] plaintext

[0436] "We want to develop an application that analyzes the stress levels of customers and suggests appropriate services based on the results. The input data will be photos of the customer's face taken with smart glasses and data on the in-store environment. We will use the Sentiment Analysis API for emotion analysis and suggest services in real time."

[0437] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0438] Step 1:

[0439] The user device (smart glasses) collects the user's facial photograph and facial expression data. The collected data includes the user's facial image and facial expression information such as smile or anger. The smart glasses temporarily store the captured image and facial expression data in their internal memory.

[0440] Step 2:

[0441] The communication method (Wi-Fi) periodically transfers the data stored in the smart glasses to a cloud server. The transferred data includes facial images, facial expression data, and environmental data (temperature, humidity, traffic flow, etc.). Once the cloud server has received the data, it proceeds to the next analysis step.

[0442] Step 3:

[0443] The cloud server uses the Face Recognition API to analyze the user's emotional data based on the received facial images. Specifically, it uses information obtained from facial expressions to evaluate the user's stress level and relaxation level. The input data is facial images and facial expression data, and the output data is an emotional evaluation report.

[0444] Step 4:

[0445] The cloud server analyzes the received environmental data. It evaluates the comfort level within the store based on the temperature, humidity, and people flow data received from the IoT devices. If necessary, it also generates adjustment instructions to optimize the environmental conditions. The input data are temperature, humidity, and people flow data, and the output data is an environmental assessment report.

[0446] Step 5:

[0447] The cloud server integrates the emotion data and environmental data to generate health advice and service suggestions for users and staff. For example, if the user is feeling stressed, a specific service suggestion is generated, such as "Please serve hot tea to help the customer relax." The input data are the emotion evaluation report and the environmental evaluation report, and the output data are the service suggestion message.

[0448] Step 6:

[0449] The cloud server notifies the generated health advice and service suggestions in real time. Notifications are sent via smartphone apps and messaging apps. Timely and appropriate service suggestions are communicated to staff through visual and audio notifications. The input data is the service suggestion message, and the output data is the notification message.

[0450] Through the above processing steps, users and staff can receive appropriate health advice and service suggestions in real time, which improves customer satisfaction and contributes to more efficient store operations.

[0451] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0452] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0453] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0454] [Second embodiment]

[0455] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0456] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0457] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0458] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0459] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0460] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0461] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0462] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0463] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0464] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0465] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0466] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0467] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0468] System Configuration

[0469] The system includes the following main components to collect and analyze a user's biometric information and provide health advice:

[0470] 1. User device (smartwatch)

[0471] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0472] 3. Cloud Server

[0473] 4. User device (smartphone app)

[0474] 5. Messaging interface (e.g., LINE)

[0475] Program processing

[0476] Data collection

[0477] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[0478] Data Transfer

[0479] The data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user takes a photo of their meal and enters the meal data into the app, it is also sent to the cloud server.

[0480] Data analysis

[0481] The cloud server analyzes all the data it receives. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Dietary data is then analyzed using image recognition technology to calculate calories and nutritional balance.

[0482] Generating health advice

[0483] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if the user's heart rate or stress level is abnormal, it will suggest measures to improve the situation or relaxation methods.

[0484] Real-time notifications

[0485] The cloud server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[0486] Supporting preventative care and access to healthcare

[0487] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user has consistently been experiencing insufficient sleep, it will send a notification saying, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[0488] Specific examples

[0489] 1. Data Collection and Transfer

[0490] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0491] Terminal (smartphone app): The user takes a photo of the meal and the data is sent to the cloud server via the smartphone app.

[0492] 2. Data analysis and health advice generation

[0493] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0494] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[0495] 3. Real-time notifications and preventative care

[0496] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[0497] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0498] 4. Supporting access to medical care

[0499] Device (smartphone app): The user opens the app, selects from the provided list of medical institutions, and makes an appointment.

[0500] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[0501] The processing flow will be explained below.

[0502] Step 1:

[0503] Device (Smartwatch): The user wears a smartwatch to monitor their heart rate, steps, sleep, and activity. The smartwatch continuously measures these data and temporarily stores them in the device's internal memory.

[0504] Step 2:

[0505] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi at pre-set intervals, such as every 10 or 30 minutes.

[0506] Step 3:

[0507] Terminal (smartphone app): The smartphone imports the received data into the app and uploads it to the cloud server. The user is notified when the upload is complete.

[0508] Step 4:

[0509] User: When eating a meal, the user takes a photo of the meal using a smartphone app and adds a brief note (including ingredients and the name of the dish). After adding the information, the data is sent to the cloud server.

[0510] Step 5:

[0511] Server: The cloud server stores the received biometric information and dietary data and saves it in an analysis database. The database also records hourly data.

[0512] Step 6:

[0513] Server: Analyzes the stored data in real time. For heart rate data, it calculates the average heart rate and maximum and minimum heart rates over the past 24 hours, and for sleep data, it analyzes the quality of sleep (the ratio of deep sleep to light sleep).

[0514] Step 7:

[0515] Server: Analyzes photos of meal data using image recognition technology to identify ingredients. Retrieves the calories and nutrients of ingredients from a database and calculates total calorie and macronutrient intake.

[0516] Step 8:

[0517] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if excessive calorie intake is detected, it will suggest ingredients and recipes to reduce calorie intake.

[0518] Step 9:

[0519] Server: The generated health advice is sent to the user in real time via a messaging interface such as LINE. For example, a message such as "Since you haven't exercised much today, try doing 15 minutes of stretching" is delivered.

[0520] Step 10:

[0521] Server: Analyzes user data over a long period of time and generates preventative alerts if signs of a worsening health condition are detected. For example, if a user has not had enough sleep for several consecutive days, a notification will be sent stating that "consultation with a medical institution is recommended."

[0522] Step 11:

[0523] Device (smartphone app): The user receives a preventive alert, opens the app, searches for nearby medical institutions within the app, selects an appropriate medical institution, and makes an appointment.

[0524] Through these steps, users can understand their health status in real time and receive appropriate health advice, enabling them to manage their health and take preventative medicine.

[0525] Example 1

[0526] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0527] Health management has become increasingly important in recent years, and health management systems that utilize biometric information have been attracting attention. However, existing systems are limited in the amount of biometric information they can acquire, making it difficult to achieve comprehensive health management for users. Furthermore, dietary data acquisition and analysis are insufficient, making it difficult to properly evaluate a user's nutritional balance. Furthermore, real-time health advice and preventative alerts are often not provided, hindering users' prompt health management.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0529] In this invention, the server includes a device for measuring a user's biometric information, memory means for temporarily storing the data, communication means, analysis means, image recognition means, notification means, and a messaging interface, which enable comprehensive collection and management of biometric information, detailed analysis of dietary data, and provision of real-time health advice and preventative alerts.

[0530] "Devices that measure biological information" are devices that measure biological data such as a user's heart rate, sleep time, and activity level, and generally refer to smartwatches and fitness trackers with heart rate monitoring functions.

[0531] "Memory means" refers to a storage device for temporarily storing measured biometric information, and corresponds to the internal memory of a smartwatch or smartphone.

[0532] "Communication means" refers to a device for transmitting measured data to other devices or servers, and includes wireless communication technologies such as Bluetooth and Wi-Fi.

[0533] "Analysis means" refers to means for analyzing received measurement data, and includes artificial intelligence algorithms and data analysis software.

[0534] "Image recognition means" refers to means for analyzing dietary data, and includes image analysis technology for analyzing photos of meals to calculate calories and nutrients.

[0535] "Notification means" refers to a means for providing health advice to users based on the analysis results, and includes smartphone notifications and message sending functions.

[0536] "Messaging interface" refers to a communication means for sending health advice to users, and includes messaging applications such as LINE and email.

[0537] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments for carrying out the present invention will be described in detail below.

[0538] System configuration

[0539] The system includes the following main components for collecting and analyzing a user's biometric information and providing health advice:

[0540] 1. User device (smartwatch)

[0541] This is a device that measures biometric information such as heart rate, sleep time, and activity level.

[0542] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0543] It is a means for transmitting measurement data to other devices or servers.

[0544] 3. Cloud Server

[0545] This server analyzes the received data and provides health advice to users. The analysis uses artificial intelligence algorithms and image analysis technology.

[0546] 4. User device (smartphone app)

[0547] It is a means for uploading data transferred from a smartwatch to a cloud server.

[0548] 5. Messaging interface (e.g., LINE)

[0549] It is a means for sending health advice to users.

[0550] Program processing

[0551] Data collection

[0552] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[0553] Data Transfer

[0554] The data stored on the device (smartwatch) is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi. The device (smartphone app) then uploads the data to a cloud server. When the user takes photos of their meals and enters them into the app, the meal data is also sent to the cloud server.

[0555] Data analysis

[0556] The server analyzes all the data it receives. Heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies. Dietary data is analyzed using image recognition technology to calculate calories and nutritional balance.

[0557] Generating health advice

[0558] The server generates optimal health advice for the user based on the analysis results. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if heart rate or stress levels are abnormal, it will suggest measures to improve or relaxation methods.

[0559] Real-time notifications

[0560] The server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[0561] Supporting preventative care and access to healthcare

[0562] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has consistently been experiencing insufficient sleep, the server will send a notification saying, "We recommend that you consult a medical institution." The user can easily make an appointment with a medical institution using the device (smartphone app).

[0563] Specific examples

[0564] 1. Data Collection and Transfer

[0565] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0566] Users take photos of their meals and the data is sent to a cloud server via a smartphone app.

[0567] 2. Data analysis and health advice generation

[0568] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0569] Server: Analyzes dietary data, identifies nutrient deficiencies, and suggests corresponding ingredient lists and recipes.

[0570] 3. Real-time notifications and preventative care

[0571] Server: Sends a message to the user via LINE, such as "We recommend walking for 20 minutes today."

[0572] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0573] 4. Supporting access to medical care

[0574] The user opens the smartphone app, selects from the provided list of medical institutions, and makes an appointment.

[0575] Examples of prompt statements

[0576] Prompt text when user enters meal data:

[0577] Take a photo of what you had for dinner last night and send it to us along with the details of the meal. For example, write "Salad, steak, and rice."

[0578] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[0579] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0580] Step 1:

[0581] Input: The user puts on the smartwatch.

[0582] How it works: When a user wears a smartwatch with heart rate monitoring throughout the day, it automatically starts collecting heart rate, vital signs, and sleep data. The data is temporarily stored in the smartwatch's internal memory.

[0583] Output: Temporarily stored heart rate, vital activity and sleep data.

[0584] Step 2:

[0585] Input: Biometric data stored on the smartwatch.

[0586] Specific operation: The device (smartwatch) periodically transfers data to the user's smartphone via Bluetooth or Wi-Fi. During the transfer process, the user checks the smartphone's connection and pairs it with the smartwatch if necessary.

[0587] Output: Biometric data transmitted to a smartphone.

[0588] Step 3:

[0589] Input: Biometric data transferred to smartphone.

[0590] Specific operation: The device (smartphone app) uploads all biometric data, including data entered into the app by the user after taking photos of their meals, to a cloud server. The user can initiate the upload by pressing the "Data Sync" button in the app.

[0591] Output: Biometric data and dietary data uploaded to a cloud server.

[0592] Step 4:

[0593] Input: Biometric data and dietary data uploaded to a cloud server.

[0594] How it works: The server uses AI algorithms to analyze the received data. It analyzes the heart rate data to detect the average heart rate over 24 hours and fluctuations in heart rate during sleep. It also uses image recognition technology to calculate calories and nutritional balance from the dietary data.

[0595] Output: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[0596] Step 5:

[0597] Input: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[0598] How it works: Based on the analysis results, the server generates optimal health advice for the user. For example, if your heart rate is too high, it will suggest relaxation techniques, or if you are lacking certain nutrients, it will suggest the necessary ingredients and recipes.

[0599] Output: The generated health advice.

[0600] Step 6:

[0601] Input: Generated health advice.

[0602] Specific operation: The server notifies the user of health advice in real time, sending messages such as "We recommend walking for 20 minutes today" via a messaging interface such as LINE.

[0603] Output: Health advice notification sent to user.

[0604] Step 7:

[0605] Input: Analysis results of long-term biometric data.

[0606] Specific operation: The server analyzes data over a long period of time, and if it detects an abnormality, such as persistent lack of sleep, it generates and sends a preventative alert such as "We recommend that you consult a medical institution."

[0607] Output: Proactive alerts sent to users.

[0608] Step 8:

[0609] Input: Medical consultation advice sent to the user.

[0610] Specific operation: The user opens the smartphone app and makes an appointment by selecting from the provided list of medical institutions. The user selects the "Medical Institution Appointment" tab in the app and specifies the desired date and time.

[0611] Output: Medical appointment.

[0612] (Application example 1)

[0613] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0614] This invention relates to a system for monitoring worker health and ensuring a safe working environment. In particular, it aims to prevent health problems caused by overwork and stress by monitoring workers' biological information in real time and providing appropriate health advice and preventive measures based on the results. Another objective is to improve worker productivity and safety by improving the working environment.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0616] In this invention, the server includes a means for measuring a user's biometric information, a means for receiving measurement data from the device, a means for analyzing the received measurement data, a means for providing health advice to the user based on the analysis results, a means for transmitting the health advice, a means for monitoring the worker's health status based on the user's biometric information, a means for improving the work environment based on the monitoring results, and a means for displaying the advice in real time. This allows for continuous monitoring of the worker's health status and immediate notification of any abnormalities. Furthermore, by providing appropriate health advice and preventive alerts, health problems caused by overwork and stress can be prevented and a safe work environment can be maintained.

[0617] "User" means an individual who utilizes the System to provide biometric information and receive health monitoring and advice.

[0618] "Biometric information" refers to data about your physical condition, such as your heart rate, activity level, and body temperature.

[0619] A "measuring device" is a device for measuring a user's biometric information, and includes, for example, a smartwatch or fitness tracker.

[0620] "Communication means" refers to a means for transmitting data acquired from a measuring device to the system, and may use Bluetooth, Wi-Fi, etc.

[0621] The "analysis means" is a means for evaluating and analyzing the health condition based on the received biometric information data.

[0622] "Notification means" refers to a means for notifying the user of health advice and warnings based on the analysis results.

[0623] A "messaging interface" is a means of communication for sending notifications and advice to users, and includes, for example, LINE and email.

[0624] "Monitoring means" refers to a means for continuously checking the health status of a user based on their biometric information and detecting any abnormalities or changes.

[0625] "Work environment improvement measures" are measures to provide users with specific advice and measures to improve their work environment based on the monitoring results.

[0626] A "display device" is a device that displays advice and warnings to users in real time, and includes digital signs in factories and smartphones.

[0627] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0628] System Configuration

[0629] The system includes the following main components to collect and analyze users' biometric information and provide advice to improve the working environment:

[0630] 1. Measuring device (smartwatch): worn by the user.

[0631] 2. Communication means (Bluetooth, Wi-Fi): Receives and transfers measurement data.

[0632] 3. Local server: Installed within the factory.

[0633] 4. Cloud server: Analyzes the data.

[0634] 5. Smartphone app: Notifies users.

[0635] 6. Display device (digital signage): displays advice in real time.

[0636] 7. Robots: Monitor the health of workers in factories.

[0637] Program processing

[0638] Data collection

[0639] Once the user wears the smartwatch, biometric data collection begins: the smartwatch continuously measures the user's heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory.

[0640] Data Transfer

[0641] The collected data is transferred to a local server via Bluetooth or Wi-Fi, and then uploaded from the local server to a cloud server.

[0642] Data analysis

[0643] The cloud server analyzes all received data and uses AI algorithms (using TensorFlow) to detect patterns and abnormalities in biometric information.

[0644] Generating Advice

[0645] Based on the analysis results, the cloud server generates optimal health advice for the user, for example, recommending rest if stress levels are high.

[0646] Real-time notifications

[0647] The cloud server then sends the generated health advice to users in real time via a smartphone app or digital signage.

[0648] Supporting preventative care and access to healthcare

[0649] The cloud server analyzes long-term data and sends preventative alerts as needed, including automatic notification to medical institutions if necessary.

[0650] Hardware and software used

[0651] Hardware:

[0652] Smartwatch

[0653] Local Server

[0654] Digital signage in factories

[0655] robot

[0656] software:

[0657] Android SDK (smartphone app development)

[0658] TensorFlow (AI algorithm implementation)

[0659] Google Cloud (Cloud Analytics)

[0660] Kubernetes (cloud server management)

[0661] Bluetooth API (data transfer)

[0662] Specific example explanation

[0663] Factory workers wear smartwatches, and robots collect their biometric data. This data is transmitted to a local server via Bluetooth and then uploaded to a cloud server. AI algorithms on the cloud server analyze the data and monitor the workers' health. If any abnormalities are detected, they are notified in real time via digital signs in the factory and a smartphone app.

[0664] Example prompt sentence:

[0665] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[0666] In this way, it is possible to provide real-time care for workers in factories and provide a safe and efficient working environment.

[0667] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0668] Step 1:

[0669] When the user wears the smartwatch, biometric data collection begins. The smartwatch continuously measures data such as heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory. The input is the user's physical data, and the output is the data stored in the smartwatch's internal memory.

[0670] Step 2:

[0671] The data stored in the smartwatch is transferred to a local server via Bluetooth or Wi-Fi. Specifically, the smartwatch batch processes the measurement data at regular intervals and transmits the data through communication with the local server. The input is the data in the smartwatch's internal memory, and the output is the data transferred to the local server.

[0672] Step 3:

[0673] The local server temporarily stores the received data and uploads it to the cloud server via the Internet. Data processing on the local server includes data formatting and format conversion. The input is the data transferred to the local server, and the output is the data uploaded to the cloud server.

[0674] Step 4:

[0675] The cloud server analyzes the received data using an AI algorithm (using TensorFlow). Specifically, it performs pattern recognition and anomaly detection. The input is the biometric data uploaded to the cloud server, and the output is the analysis results, i.e., an evaluation of health status and anomaly detection results.

[0676] Step 5:

[0677] The cloud server generates health advice and warnings for the user based on the analysis results. Specifically, it uses a generative AI model to create appropriate messages based on the analysis results. The input is the analysis results, and the output is health advice and warning messages.

[0678] Step 6:

[0679] The generated health advice and warning messages are notified to users in real time using a smartphone app or digital signs in the factory. Specifically, the cloud server issues the notifications and sends them to each device via a messaging interface. The input is the health advice or warning message, and the output is the notification sent to the user.

[0680] Step 7:

[0681] The cloud server accumulates long-term data, periodically analyzes it, and sends preventive alerts to users. This includes trend analysis of past data and anomaly detection. The input is long-term accumulated biometric data, and the output is preventive alerts.

[0682] A specific example includes steps to generate the following prompt statement if the user's heart rate increases by more than 20% of normal:

[0683] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[0684] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0685] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0686] System Configuration

[0687] The system includes the following main components to collect and analyze users' biometric and emotional data and provide health advice:

[0688] 1. User device (smartwatch)

[0689] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0690] 3. Cloud Server

[0691] 4. User device (smartphone app)

[0692] 5. Messaging interface (e.g., LINE)

[0693] 6. Emotion Engine

[0694] Program processing

[0695] Data collection

[0696] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory. In addition, a smartphone app collects emotional data using the user's facial photos and voice data.

[0697] Data Transfer

[0698] The biometric data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user enters photos of their meals or self-reported data into the app, the data is also sent to the cloud server.

[0699] Data analysis

[0700] The cloud server analyzes all received data. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Food data is analyzed using image recognition technology to calculate calories and nutritional balance. The emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies.

[0701] Generating health advice

[0702] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. If the emotion engine evaluates a user's stress level as high, it will suggest relaxation methods and mental care advice. The advice content is also customized based on the user's emotional state.

[0703] Real-time notifications

[0704] The cloud server then sends the generated health advice to the user in real time via messaging interfaces such as LINE. For example, if the emotion engine detects that the user's stress level is high, it will send a message such as "Take 10 deep breaths and relax."

[0705] Supporting preventative care and access to healthcare

[0706] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user experiences insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[0707] Specific examples

[0708] 1. Data Collection and Transfer

[0709] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0710] Device (smartphone app): The user takes a photo of their meal and the data is sent to a cloud server via the smartphone app. The app also captures and records a photo of the user's face and a short voice message, which collects emotional data.

[0711] 2. Data analysis and health advice generation

[0712] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0713] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[0714] Server and Emotion Engine: The emotion engine analyzes facial photos and audio data to assess the user's emotional state, for example, assessing stress levels based on the frequency of smiles and tone of voice.

[0715] 3. Real-time notifications and preventative care

[0716] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[0717] Server: If the user's emotional state is unstable, send a notification saying "Take 10 deep breaths and relax."

[0718] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0719] 4. Supporting access to medical care

[0720] Device (smartphone app): When the user receives a prevention alert and opens the app, they search for nearby medical institutions, select an appropriate medical institution, and make an appointment.

[0721] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[0722] The processing flow will be explained below.

[0723] Step 1:

[0724] Device (smartwatch): The user wears a smartwatch to measure data such as heart rate, number of steps, sleep time, and activity level. The smartwatch continuously measures this data and stores it in its internal memory for a certain period of time (for example, one day or several hours).

[0725] Step 2:

[0726] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi. The transfer occurs automatically at specific time intervals, and the data is deleted from the smartwatch's internal memory after transfer.

[0727] Step 3:

[0728] Device (smartphone app): The smartphone app uploads the data received from the smartwatch to the cloud server. When the upload is complete, the user is notified. Also, when the user eats a meal, the smartphone app takes a photo of the meal, adds a brief note (including ingredients and the name of the dish), and sends this to the cloud server.

[0729] Step 4:

[0730] User: Emotion data is collected by users taking photos of their faces and recording voice messages using a smartphone app. These data are also sent to the cloud server.

[0731] Step 5:

[0732] Server: The cloud server securely stores all data received from the smartwatch and smartphone, and saves it in a database for analysis, along with timestamps and user identification information.

[0733] Step 6:

[0734] Server: The cloud server performs the data analysis. Heart rate data is analyzed by an AI algorithm to calculate the average heart rate, resting heart rate, maximum and minimum heart rate over the past 24 hours, and detect abnormalities. Sleep data is also calculated to evaluate the ratio of deep sleep to light sleep and evaluate sleep quality.

[0735] Step 7:

[0736] Server: When analyzing meal data, image recognition technology is used to identify ingredients in photos of meals, and the calories and major nutrients of each ingredient are retrieved from a database to calculate total calories and nutritional balance.

[0737] Step 8:

[0738] Emotion engine: The emotion engine on the cloud server analyzes facial photos and voice data to assess the user's emotional state. For example, it calculates the user's stress level and emotional tendencies (happiness, sadness, anger, etc.) through facial expression analysis and tone of voice analysis.

[0739] Step 9:

[0740] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if a user is lacking in nutrients, the server will provide advice such as, "You are lacking in vitamin D, so try incorporating fish and mushrooms into your diet." Based on the results from the emotion engine, if the user's stress level is high, the server will suggest relaxation methods.

[0741] Step 10:

[0742] Server: The cloud server generates health advice and sends it to the user in real time via a messaging interface such as LINE. For example, a notification such as "You haven't exercised much today, so try stretching for 15 minutes" is sent.

[0743] Step 11:

[0744] Server: The cloud server analyzes the user's data over a long period of time and generates preventive alerts if signs of a deterioration in health are detected. For example, if the user has not had enough sleep for several days in a row, the server will send a message saying, "We recommend that you consult a doctor."

[0745] Step 12:

[0746] Device (smartphone app): The user receives a preventive alert and opens the smartphone app. Within the app, they search for a nearby medical institution, select an appropriate medical institution, and make an appointment. Once the appointment is completed, a reminder notification is sent to the user.

[0747] In this way, the system continuously monitors the user's health and emotional data and provides appropriate advice and preventative care in real time, helping to improve the user's health and quality of life.

[0748] Example 2

[0749] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0750] Current health management systems primarily measure and analyze biometric information, but few systems comprehensively analyze users' emotional and dietary data to provide health advice. Furthermore, there is a lack of real-time preventive alert notifications based on the user's health and emotional state, making it difficult for users to receive timely health improvement measures.

[0751] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0752] In this invention, the server includes a communication means for receiving measurement data from a device that measures the user's biological information, an analysis means for analyzing the received measurement data and the user's facial photograph and voice data, and a messaging interface for providing and transmitting health advice to the user in real time based on the analysis results. This makes it possible to comprehensively analyze the user's biological information, dietary data, and emotional state, provide the user with optimal health advice in real time, and promptly notify the user of preventive medical alerts.

[0753] "User's biological information" refers to data measured directly from the user's body, such as the user's heart rate, sleep duration, and activity level.

[0754] "Measurement data" refers to data collected by various sensors, including biometric information and emotional data of the user.

[0755] "Communication means" refers to technology that has the function of transferring data from devices such as smartwatches and smartphone apps to cloud servers, and includes Bluetooth and Wi-Fi.

[0756] "Analysis means" refers to the AI ​​algorithms and analysis programs used to analyze data received on the cloud server.

[0757] A "user's face photo" is image data of the user's face taken using a smartphone app.

[0758] "Voice data" refers to data of a user's voice recorded using a smartphone app.

[0759] "Health advice" refers to recommendations and advice regarding health management that is provided to the user based on the results of analysis by the analysis means.

[0760] "Notification means" refers to technology that has the function of notifying users of the generated health advice, and includes messaging services such as LINE.

[0761] A "messaging interface" is an interface for sending notifications from a cloud server to a user in real time, and includes messaging services that use APIs.

[0762] "Dietary data" refers to photos of meals taken by the user and data on the contents of meals self-reported by the user.

[0763] "Calories" refers to the amount of energy calculated from dietary data.

[0764] "Nutritional balance" refers to the proportion of nutrients contained in food.

[0765] A "preventive alert" is a notification of preventive medical advice or caution generated by the analysis means.

[0766] MODE FOR CARRYING OUT THE INVENTION

[0767] The following describes in detail the specific embodiments of the present invention. The present invention is a system for collecting and analyzing biometric and emotional data of a user and providing health advice. The main components of the system are as follows:

[0768] 1. User device (smartwatch)

[0769] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0770] 3. Cloud Server

[0771] 4. User device (smartphone app)

[0772] 5. Messaging interface (e.g., LINE)

[0773] 6. Emotion Engine

[0774] Data collection

[0775] When a user wears a smartwatch, the collection of biometric information begins. The device (smartwatch) continuously measures the user's heart rate, sleep time, and activity level, and stores the data in its internal memory. In addition, the device (smartphone app) collects emotional data using the user's facial photos and voice data. For example, if a user wears a smartwatch with a heart rate monitoring function all day, heart rate data is automatically recorded. In addition, the user can take photos of their meals, and the data is sent to a cloud server via the smartphone app.

[0776] Data Transfer

[0777] The biometric data stored in the device (smartwatch) is periodically transferred to the device (smartphone app) via Bluetooth or Wi-Fi. The smartphone app then uploads this data to a cloud server. For example, the smartphone app periodically opens a Bluetooth connection to receive data and sends the data to the cloud server using the HTTP protocol.

[0778] Data analysis

[0779] The server receives the data sent to the cloud and analyzes it using AI algorithms. For example, heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies in the data. Meal data is used to calculate calories and nutritional balance using image recognition technology. An emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies. Specifically, the server analyzes the data using an AI algorithm built in Python and stores the results in a database. Examples of prompts include "Analyze today's heart rate data and tell me if there are any abnormalities" and "Calculate calories and nutritional balance from this meal photo."

[0780] Generating health advice

[0781] The server generates optimal health advice for the user based on the results of data analysis. For example, if a user is lacking in a particular nutrient, the server will suggest corresponding ingredients and recipes. If the user's stress level is high, relaxation methods will be suggested. Specifically, the server uses a pre-configured rule-based engine to select appropriate advice from a database. An example of a prompt could be, "Please assess the user's emotional state based on this facial photo and voice data."

[0782] Real-time notifications

[0783] The server sends the generated health advice to the user via a messaging interface such as LINE. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent. Specifically, the server uses the LINE API to generate a message and send it to the user. An example of a prompt is "Generate health advice from long-term sleep data."

[0784] Supporting preventative care and access to healthcare

[0785] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has had insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app. Specifically, the smartphone app uses GPS to list nearby medical institutions and connects with the reservation system to complete the appointment.

[0786] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[0787] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0788] Step 1: The user puts on the smartwatch.

[0789] Input: User's biological information (heart rate, sleep time, activity level)

[0790] How it works: The smartwatch's sensors measure the user's pulse and record it as heart rate data, while the accelerometer measures activity and estimates sleep time.

[0791] Output: Biometric data stored in internal memory

[0792] Step 2: The user takes a photo of their face using a smartphone app and records a voice message.

[0793] Input: User's face photo data, voice data

[0794] What it does: The smartphone camera takes a picture of the user's face, the microphone records their voice, and the app temporarily stores this data.

[0795] Output: Temporarily saved face photo data and audio data

[0796] Step 3: Transfer the biometric data stored on the smartwatch to your smartphone.

[0797] Input: Biometric data stored in the smartwatch's internal memory

[0798] What it does: Transfers smartwatch data to your smartphone via Bluetooth or Wi-Fi connection.

[0799] Output: Biometric data stored on a smartphone

[0800] Step 4: The smartphone app uploads all collected data to the cloud server.

[0801] Input: Biometric data, facial photo data, and voice data stored on the smartphone

[0802] Specific operation: The smartphone app sends data to the cloud server using the HTTP protocol.

[0803] Output: Biometric data, facial photo data, and voice data stored on a cloud server

[0804] Step 5: The cloud server analyzes the received data.

[0805] Input: Biometric data, facial photo data, and voice data stored on the cloud server

[0806] How it works: An AI algorithm on a cloud server analyzes heart rate data to detect 24-hour average heart rate and abnormalities. Image recognition technology is used to calculate calories and nutritional balance from photos of meals. An emotion engine also analyzes facial photos and voice data to assess stress levels and emotional states.

[0807] Output: Analysis results (health status, nutritional balance, emotional state)

[0808] Step 6: The cloud server generates health advice based on the analysis results.

[0809] Input: Analysis results (health status, nutritional balance, emotional state)

[0810] How it works: The cloud server uses a pre-configured rule-based engine to generate appropriate health advice.

[0811] Output: Generated health advice

[0812] Step 7: The cloud server notifies the user of the generated health advice in real time.

[0813] Input: Generated health advice

[0814] Specific operation: The cloud server generates a message using the LINE API and sends it to the user. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent.

[0815] Output: Health advice sent to the user

[0816] Step 8: The cloud server analyzes the long-term data and sends proactive alerts as needed.

[0817] Input: Long-term biometric data, emotional data

[0818] How it works: The cloud server analyzes past data trends and detects chronic problems. For example, if you experience insufficient sleep for several consecutive days, it generates a notification suggesting that you seek medical advice.

[0819] Output: Preventive alerts sent to users

[0820] Step 9: The user makes an appointment with a medical institution using the smartphone app.

[0821] Input: User's current location, preventative alerts

[0822] Specific operation: The smartphone app uses GPS to search for nearby medical institutions and connects with the reservation system to complete the appointment.

[0823] Output: Information about the medical institution where the reservation was completed

[0824] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice in real time, and quickly access medical care if needed.

[0825] (Application example 2)

[0826] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] Conventional health management systems provide health advice based on a user's biometric information, but do not offer service suggestions based on the customer's real-time emotional state, making it difficult to improve customer satisfaction, especially in brick-and-mortar stores. Furthermore, they lack the functionality to optimize services by utilizing in-store environmental data. This invention aims to solve these problems by providing a system that collects and analyzes customer emotional and environmental data and offers service suggestions based on that data.

[0828] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for measuring the user's biological information, means for receiving measurement data and environmental data from the device, means for analyzing the received measurement data and environmental data, means for providing health advice and service suggestions to the user based on the analysis results, and a messaging interface for transmitting the health advice and service suggestions. This makes it possible to improve the efficiency of store operations and customer satisfaction.

[0829] "Biometric information" refers to data relating to the user's health condition and bodily functions, and specifically includes heart rate, sleep duration, activity level, and the like.

[0830] "Communication means" refers to a means for transmitting measurement data from a device that measures biological information to other devices or servers, and utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[0831] The "analysis means" is a technology that processes the received measurement data and environmental data to evaluate and judge the user's health and emotional state, and uses artificial intelligence and machine learning algorithms.

[0832] "Notification means" refers to technology for notifying users and staff in real time of appropriate health advice and service suggestions based on the analysis results.

[0833] A "messaging interface" is a means of communication for conveying notifications to users and staff, and utilizes messaging applications such as LINE and WhatsApp.

[0834] "Service suggestions" are specific instructions and advice provided based on the user's emotional state and health status obtained by the analysis means, and are intended to improve the quality of service in physical stores.

[0835] "Environmental data" refers to information about factors that affect customer comfort and service provision, such as temperature, humidity, lighting, and foot traffic within the store.

[0836] "Real-time notification" is a technology that quickly conveys information to users and staff based on results obtained instantly by analytical means.

[0837] The present invention is a system for collecting and analyzing biometric and emotional data of users, and providing health advice and service suggestions based on the collected data. The system aims to improve customer satisfaction in brick-and-mortar stores and consists of several main components.

[0838] System Configuration

[0839] The main components of the system are:

[0840] 1. User device (smart glasses)

[0841] 2. Communication method (Wi-Fi)

[0842] 3. Cloud Server

[0843] 4. Notification devices (smartphone apps, messaging apps)

[0844] 5. Analysis engine (emotion engine and environmental data engine)

[0845] How to use

[0846] Data collection

[0847] When users (customers) wear smart glasses, their facial photos and facial expression data are collected by a camera. In-store environmental data (temperature, humidity, and traffic flow) is collected by sensors, and this data is transferred from the smart glasses to a cloud server in real time.

[0848] Data analysis

[0849] The cloud server analyzes the received biometric information and environmental data. It uses the Face Recognition API and Sentiment Analysis API to collect user emotional data from facial photos and facial expression data. It analyzes environmental data obtained from IoT devices and adjusts the store environment as needed.

[0850] Providing health advice and service suggestions

[0851] The cloud server generates advice and service suggestions for the user's health condition based on the analyzed data. Specifically, if the user is feeling stressed, it will provide advice on relaxation methods and mental care. In addition, store staff will be notified of the customer's service suggestions in real time.

[0852] Real-time notifications

[0853] The cloud server then sends the generated health advice and service suggestions to a smartphone app or messaging app via Wi-Fi. For example, if a customer is detected as feeling stressed, a specific service suggestion such as "Please serve hot tea to help the customer relax" can be sent to store staff.

[0854] Hardware and software used

[0855] Hardware: Smart glasses (camera, built-in sensors), IoT sensors (temperature, humidity, people flow sensors)

[0856] Software: Face Recognition API, Sentiment Analysis API, IoT Connectivity (Arduino, Raspberry Pi)

[0857] Cloud Platform: AWS (Amazon Web Services), Azure

[0858] Specific examples and examples of prompts for generative AI models

[0859] Examples:

[0860] If a customer smiles at the entrance, "The customer is relaxed. Please maintain the atmosphere in the store."

[0861] If the customer looks unhappy, say, "The customer's stress level is high. I suggest offering them a welcome drink."

[0862] Example prompts for generative AI models:

[0863] plaintext

[0864] "We want to develop an application that analyzes the stress levels of customers and suggests appropriate services based on the results. The input data will be photos of the customer's face taken with smart glasses and data on the in-store environment. We will use the Sentiment Analysis API for emotion analysis and suggest services in real time."

[0865] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0866] Step 1:

[0867] The user device (smart glasses) collects the user's facial photograph and facial expression data. The collected data includes the user's facial image and facial expression information such as smile or anger. The smart glasses temporarily store the captured image and facial expression data in their internal memory.

[0868] Step 2:

[0869] The communication method (Wi-Fi) periodically transfers the data stored in the smart glasses to a cloud server. The transferred data includes facial images, facial expression data, and environmental data (temperature, humidity, traffic flow, etc.). Once the cloud server has received the data, it proceeds to the next analysis step.

[0870] Step 3:

[0871] The cloud server uses the Face Recognition API to analyze the user's emotional data based on the received facial images. Specifically, it uses information obtained from facial expressions to evaluate the user's stress level and relaxation level. The input data is facial images and facial expression data, and the output data is an emotional evaluation report.

[0872] Step 4:

[0873] The cloud server analyzes the received environmental data. It evaluates the comfort level within the store based on the temperature, humidity, and people flow data received from the IoT devices. If necessary, it also generates adjustment instructions to optimize the environmental conditions. The input data are temperature, humidity, and people flow data, and the output data is an environmental assessment report.

[0874] Step 5:

[0875] The cloud server integrates the emotion data and environmental data to generate health advice and service suggestions for users and staff. For example, if the user is feeling stressed, a specific service suggestion is generated, such as "Please serve hot tea to help the customer relax." The input data are the emotion evaluation report and the environmental evaluation report, and the output data are the service suggestion message.

[0876] Step 6:

[0877] The cloud server notifies the generated health advice and service suggestions in real time. Notifications are sent via smartphone apps and messaging apps. Timely and appropriate service suggestions are communicated to staff through visual and audio notifications. The input data is the service suggestion message, and the output data is the notification message.

[0878] Through the above processing steps, users and staff can receive appropriate health advice and service suggestions in real time, which improves customer satisfaction and contributes to more efficient store operations.

[0879] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0880] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0881] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0882] [Third embodiment]

[0883] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0884] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0885] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0886] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0887] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0888] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0889] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0890] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0891] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0892] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0893] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0894] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0895] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[0896] System Configuration

[0897] The system includes the following main components to collect and analyze a user's biometric information and provide health advice:

[0898] 1. User device (smartwatch)

[0899] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0900] 3. Cloud Server

[0901] 4. User device (smartphone app)

[0902] 5. Messaging interface (e.g., LINE)

[0903] Program processing

[0904] Data collection

[0905] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[0906] Data Transfer

[0907] The data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user takes a photo of their meal and enters the meal data into the app, it is also sent to the cloud server.

[0908] Data analysis

[0909] The cloud server analyzes all the data it receives. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Dietary data is then analyzed using image recognition technology to calculate calories and nutritional balance.

[0910] Generating health advice

[0911] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if the user's heart rate or stress level is abnormal, it will suggest measures to improve the situation or relaxation methods.

[0912] Real-time notifications

[0913] The cloud server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[0914] Supporting preventative care and access to healthcare

[0915] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user has consistently been experiencing insufficient sleep, it will send a notification saying, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[0916] Specific examples

[0917] 1. Data Collection and Transfer

[0918] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0919] Terminal (smartphone app): The user takes a photo of the meal and the data is sent to the cloud server via the smartphone app.

[0920] 2. Data analysis and health advice generation

[0921] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0922] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[0923] 3. Real-time notifications and preventative care

[0924] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[0925] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[0926] 4. Supporting access to medical care

[0927] Device (smartphone app): The user opens the app, selects from the provided list of medical institutions, and makes an appointment.

[0928] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[0929] The processing flow will be explained below.

[0930] Step 1:

[0931] Device (Smartwatch): The user wears a smartwatch to monitor their heart rate, steps, sleep, and activity. The smartwatch continuously measures these data and temporarily stores them in the device's internal memory.

[0932] Step 2:

[0933] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi at pre-set intervals, such as every 10 or 30 minutes.

[0934] Step 3:

[0935] Terminal (smartphone app): The smartphone imports the received data into the app and uploads it to the cloud server. The user is notified when the upload is complete.

[0936] Step 4:

[0937] User: When eating a meal, the user takes a photo of the meal using a smartphone app and adds a brief note (including ingredients and the name of the dish). After adding the information, the data is sent to the cloud server.

[0938] Step 5:

[0939] Server: The cloud server stores the received biometric information and dietary data and saves it in an analysis database. The database also records hourly data.

[0940] Step 6:

[0941] Server: Analyzes the stored data in real time. For heart rate data, it calculates the average heart rate and maximum and minimum heart rates over the past 24 hours, and for sleep data, it analyzes the quality of sleep (the ratio of deep sleep to light sleep).

[0942] Step 7:

[0943] Server: Analyzes photos of meal data using image recognition technology to identify ingredients. Retrieves the calories and nutrients of ingredients from a database and calculates total calorie and macronutrient intake.

[0944] Step 8:

[0945] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if excessive calorie intake is detected, it will suggest ingredients and recipes to reduce calorie intake.

[0946] Step 9:

[0947] Server: The generated health advice is sent to the user in real time via a messaging interface such as LINE. For example, a message such as "Since you haven't exercised much today, try doing 15 minutes of stretching" is delivered.

[0948] Step 10:

[0949] Server: Analyzes user data over a long period of time and generates preventative alerts if signs of a worsening health condition are detected. For example, if a user has not had enough sleep for several consecutive days, a notification will be sent stating that "consultation with a medical institution is recommended."

[0950] Step 11:

[0951] Device (smartphone app): The user receives a preventive alert, opens the app, searches for nearby medical institutions within the app, selects an appropriate medical institution, and makes an appointment.

[0952] Through these steps, users can understand their health status in real time and receive appropriate health advice, enabling them to manage their health and take preventative medicine.

[0953] Example 1

[0954] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] Health management has become increasingly important in recent years, and health management systems that utilize biometric information have been attracting attention. However, existing systems are limited in the amount of biometric information they can acquire, making it difficult to achieve comprehensive health management for users. Furthermore, dietary data acquisition and analysis are insufficient, making it difficult to properly evaluate a user's nutritional balance. Furthermore, real-time health advice and preventative alerts are often not provided, hindering users' prompt health management.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0957] In this invention, the server includes a device for measuring a user's biometric information, memory means for temporarily storing the data, communication means, analysis means, image recognition means, notification means, and a messaging interface, which enable comprehensive collection and management of biometric information, detailed analysis of dietary data, and provision of real-time health advice and preventative alerts.

[0958] "Devices that measure biological information" are devices that measure biological data such as a user's heart rate, sleep time, and activity level, and generally refer to smartwatches and fitness trackers with heart rate monitoring functions.

[0959] "Memory means" refers to a storage device for temporarily storing measured biometric information, and corresponds to the internal memory of a smartwatch or smartphone.

[0960] "Communication means" refers to a device for transmitting measured data to other devices or servers, and includes wireless communication technologies such as Bluetooth and Wi-Fi.

[0961] "Analysis means" refers to means for analyzing received measurement data, and includes artificial intelligence algorithms and data analysis software.

[0962] "Image recognition means" refers to means for analyzing dietary data, and includes image analysis technology for analyzing photos of meals to calculate calories and nutrients.

[0963] "Notification means" refers to a means for providing health advice to users based on the analysis results, and includes smartphone notifications and message sending functions.

[0964] "Messaging interface" refers to a communication means for sending health advice to users, and includes messaging applications such as LINE and email.

[0965] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments for carrying out the present invention will be described in detail below.

[0966] System configuration

[0967] The system includes the following main components for collecting and analyzing a user's biometric information and providing health advice:

[0968] 1. User device (smartwatch)

[0969] This is a device that measures biometric information such as heart rate, sleep time, and activity level.

[0970] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[0971] It is a means for transmitting measurement data to other devices or servers.

[0972] 3. Cloud Server

[0973] This server analyzes the received data and provides health advice to users. The analysis uses artificial intelligence algorithms and image analysis technology.

[0974] 4. User device (smartphone app)

[0975] It is a means for uploading data transferred from a smartwatch to a cloud server.

[0976] 5. Messaging interface (e.g., LINE)

[0977] It is a means for sending health advice to users.

[0978] Program processing

[0979] Data collection

[0980] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[0981] Data Transfer

[0982] The data stored on the device (smartwatch) is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi. The device (smartphone app) then uploads the data to a cloud server. When the user takes photos of their meals and enters them into the app, the meal data is also sent to the cloud server.

[0983] Data analysis

[0984] The server analyzes all the data it receives. Heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies. Dietary data is analyzed using image recognition technology to calculate calories and nutritional balance.

[0985] Generating health advice

[0986] The server generates optimal health advice for the user based on the analysis results. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if heart rate or stress levels are abnormal, it will suggest measures to improve or relaxation methods.

[0987] Real-time notifications

[0988] The server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[0989] Supporting preventative care and access to healthcare

[0990] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has consistently been experiencing insufficient sleep, the server will send a notification saying, "We recommend that you consult a medical institution." The user can easily make an appointment with a medical institution using the device (smartphone app).

[0991] Specific examples

[0992] 1. Data Collection and Transfer

[0993] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[0994] Users take photos of their meals and the data is sent to a cloud server via a smartphone app.

[0995] 2. Data analysis and health advice generation

[0996] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[0997] Server: Analyzes dietary data, identifies nutrient deficiencies, and suggests corresponding ingredient lists and recipes.

[0998] 3. Real-time notifications and preventative care

[0999] Server: Sends a message to the user via LINE, such as "We recommend walking for 20 minutes today."

[1000] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[1001] 4. Supporting access to medical care

[1002] The user opens the smartphone app, selects from the provided list of medical institutions, and makes an appointment.

[1003] Examples of prompt statements

[1004] Prompt text when user enters meal data:

[1005] Take a photo of what you had for dinner last night and send it to us along with the details of the meal. For example, write "Salad, steak, and rice."

[1006] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[1007] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1008] Step 1:

[1009] Input: The user puts on the smartwatch.

[1010] How it works: When a user wears a smartwatch with heart rate monitoring throughout the day, it automatically starts collecting heart rate, vital signs, and sleep data. The data is temporarily stored in the smartwatch's internal memory.

[1011] Output: Temporarily stored heart rate, vital activity and sleep data.

[1012] Step 2:

[1013] Input: Biometric data stored on the smartwatch.

[1014] Specific operation: The device (smartwatch) periodically transfers data to the user's smartphone via Bluetooth or Wi-Fi. During the transfer process, the user checks the smartphone's connection and pairs it with the smartwatch if necessary.

[1015] Output: Biometric data transmitted to a smartphone.

[1016] Step 3:

[1017] Input: Biometric data transferred to smartphone.

[1018] Specific operation: The device (smartphone app) uploads all biometric data, including data entered into the app by the user after taking photos of their meals, to a cloud server. The user can initiate the upload by pressing the "Data Sync" button in the app.

[1019] Output: Biometric data and dietary data uploaded to a cloud server.

[1020] Step 4:

[1021] Input: Biometric data and dietary data uploaded to a cloud server.

[1022] How it works: The server uses AI algorithms to analyze the received data. It analyzes the heart rate data to detect the average heart rate over 24 hours and fluctuations in heart rate during sleep. It also uses image recognition technology to calculate calories and nutritional balance from the dietary data.

[1023] Output: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[1024] Step 5:

[1025] Input: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[1026] How it works: Based on the analysis results, the server generates optimal health advice for the user. For example, if your heart rate is too high, it will suggest relaxation techniques, or if you are lacking certain nutrients, it will suggest the necessary ingredients and recipes.

[1027] Output: The generated health advice.

[1028] Step 6:

[1029] Input: Generated health advice.

[1030] Specific operation: The server notifies the user of health advice in real time, sending messages such as "We recommend walking for 20 minutes today" via a messaging interface such as LINE.

[1031] Output: Health advice notification sent to user.

[1032] Step 7:

[1033] Input: Analysis results of long-term biometric data.

[1034] Specific operation: The server analyzes data over a long period of time, and if it detects an abnormality, such as persistent lack of sleep, it generates and sends a preventative alert such as "We recommend that you consult a medical institution."

[1035] Output: Proactive alerts sent to users.

[1036] Step 8:

[1037] Input: Medical consultation advice sent to the user.

[1038] Specific operation: The user opens the smartphone app and makes an appointment by selecting from the provided list of medical institutions. The user selects the "Medical Institution Appointment" tab in the app and specifies the desired date and time.

[1039] Output: Medical appointment.

[1040] (Application example 1)

[1041] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1042] This invention relates to a system for monitoring worker health and ensuring a safe working environment. In particular, it aims to prevent health problems caused by overwork and stress by monitoring workers' biological information in real time and providing appropriate health advice and preventive measures based on the results. Another objective is to improve worker productivity and safety by improving the working environment.

[1043] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1044] In this invention, the server includes a means for measuring a user's biometric information, a means for receiving measurement data from the device, a means for analyzing the received measurement data, a means for providing health advice to the user based on the analysis results, a means for transmitting the health advice, a means for monitoring the worker's health status based on the user's biometric information, a means for improving the work environment based on the monitoring results, and a means for displaying the advice in real time. This allows for continuous monitoring of the worker's health status and immediate notification of any abnormalities. Furthermore, by providing appropriate health advice and preventive alerts, health problems caused by overwork and stress can be prevented and a safe work environment can be maintained.

[1045] "User" means an individual who utilizes the System to provide biometric information and receive health monitoring and advice.

[1046] "Biometric information" refers to data about your physical condition, such as your heart rate, activity level, and body temperature.

[1047] A "measuring device" is a device for measuring a user's biometric information, and includes, for example, a smartwatch or fitness tracker.

[1048] "Communication means" refers to a means for transmitting data acquired from a measuring device to the system, and may use Bluetooth, Wi-Fi, etc.

[1049] The "analysis means" is a means for evaluating and analyzing the health condition based on the received biometric information data.

[1050] "Notification means" refers to a means for notifying the user of health advice and warnings based on the analysis results.

[1051] A "messaging interface" is a means of communication for sending notifications and advice to users, and includes, for example, LINE and email.

[1052] "Monitoring means" refers to a means for continuously checking the health status of a user based on their biometric information and detecting any abnormalities or changes.

[1053] "Work environment improvement measures" are measures to provide users with specific advice and measures to improve their work environment based on the monitoring results.

[1054] A "display device" is a device that displays advice and warnings to users in real time, and includes digital signs in factories and smartphones.

[1055] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[1056] System Configuration

[1057] The system includes the following main components to collect and analyze users' biometric information and provide advice to improve the working environment:

[1058] 1. Measuring device (smartwatch): worn by the user.

[1059] 2. Communication means (Bluetooth, Wi-Fi): Receives and transfers measurement data.

[1060] 3. Local server: Installed within the factory.

[1061] 4. Cloud server: Analyzes the data.

[1062] 5. Smartphone app: Notifies users.

[1063] 6. Display device (digital signage): displays advice in real time.

[1064] 7. Robots: Monitor the health of workers in factories.

[1065] Program processing

[1066] Data collection

[1067] Once the user wears the smartwatch, biometric data collection begins: the smartwatch continuously measures the user's heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory.

[1068] Data Transfer

[1069] The collected data is transferred to a local server via Bluetooth or Wi-Fi, and then uploaded from the local server to a cloud server.

[1070] Data analysis

[1071] The cloud server analyzes all received data and uses AI algorithms (using TensorFlow) to detect patterns and abnormalities in biometric information.

[1072] Generating Advice

[1073] Based on the analysis results, the cloud server generates optimal health advice for the user, for example, recommending rest if stress levels are high.

[1074] Real-time notifications

[1075] The cloud server then sends the generated health advice to users in real time via a smartphone app or digital signage.

[1076] Supporting preventative care and access to healthcare

[1077] The cloud server analyzes long-term data and sends preventative alerts as needed, including automatic notification to medical institutions if necessary.

[1078] Hardware and software used

[1079] Hardware:

[1080] Smartwatch

[1081] Local Server

[1082] Digital signage in factories

[1083] robot

[1084] software:

[1085] Android SDK (smartphone app development)

[1086] TensorFlow (AI algorithm implementation)

[1087] Google Cloud (Cloud Analytics)

[1088] Kubernetes (cloud server management)

[1089] Bluetooth API (data transfer)

[1090] Specific example explanation

[1091] Factory workers wear smartwatches, and robots collect their biometric data. This data is transmitted to a local server via Bluetooth and then uploaded to a cloud server. AI algorithms on the cloud server analyze the data and monitor the workers' health. If any abnormalities are detected, they are notified in real time via digital signs in the factory and a smartphone app.

[1092] Example prompt sentence:

[1093] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[1094] In this way, it is possible to provide real-time care for workers in factories and provide a safe and efficient working environment.

[1095] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1096] Step 1:

[1097] When the user wears the smartwatch, biometric data collection begins. The smartwatch continuously measures data such as heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory. The input is the user's physical data, and the output is the data stored in the smartwatch's internal memory.

[1098] Step 2:

[1099] The data stored in the smartwatch is transferred to a local server via Bluetooth or Wi-Fi. Specifically, the smartwatch batch processes the measurement data at regular intervals and transmits the data through communication with the local server. The input is the data in the smartwatch's internal memory, and the output is the data transferred to the local server.

[1100] Step 3:

[1101] The local server temporarily stores the received data and uploads it to the cloud server via the Internet. Data processing on the local server includes data formatting and format conversion. The input is the data transferred to the local server, and the output is the data uploaded to the cloud server.

[1102] Step 4:

[1103] The cloud server analyzes the received data using an AI algorithm (using TensorFlow). Specifically, it performs pattern recognition and anomaly detection. The input is the biometric data uploaded to the cloud server, and the output is the analysis results, i.e., an evaluation of health status and anomaly detection results.

[1104] Step 5:

[1105] The cloud server generates health advice and warnings for the user based on the analysis results. Specifically, it uses a generative AI model to create appropriate messages based on the analysis results. The input is the analysis results, and the output is health advice and warning messages.

[1106] Step 6:

[1107] The generated health advice and warning messages are notified to users in real time using a smartphone app or digital signs in the factory. Specifically, the cloud server issues the notifications and sends them to each device via a messaging interface. The input is the health advice or warning message, and the output is the notification sent to the user.

[1108] Step 7:

[1109] The cloud server accumulates long-term data, periodically analyzes it, and sends preventive alerts to users. This includes trend analysis of past data and anomaly detection. The input is long-term accumulated biometric data, and the output is preventive alerts.

[1110] A specific example includes steps to generate the following prompt statement if the user's heart rate increases by more than 20% of normal:

[1111] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[1112] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1113] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[1114] System Configuration

[1115] The system includes the following main components to collect and analyze users' biometric and emotional data and provide health advice:

[1116] 1. User device (smartwatch)

[1117] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[1118] 3. Cloud Server

[1119] 4. User device (smartphone app)

[1120] 5. Messaging interface (e.g., LINE)

[1121] 6. Emotion Engine

[1122] Program processing

[1123] Data collection

[1124] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory. In addition, a smartphone app collects emotional data using the user's facial photos and voice data.

[1125] Data Transfer

[1126] The biometric data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user enters photos of their meals or self-reported data into the app, the data is also sent to the cloud server.

[1127] Data analysis

[1128] The cloud server analyzes all received data. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Food data is analyzed using image recognition technology to calculate calories and nutritional balance. The emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies.

[1129] Generating health advice

[1130] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. If the emotion engine evaluates a user's stress level as high, it will suggest relaxation methods and mental care advice. The advice content is also customized based on the user's emotional state.

[1131] Real-time notifications

[1132] The cloud server then sends the generated health advice to the user in real time via messaging interfaces such as LINE. For example, if the emotion engine detects that the user's stress level is high, it will send a message such as "Take 10 deep breaths and relax."

[1133] Supporting preventative care and access to healthcare

[1134] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user experiences insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[1135] Specific examples

[1136] 1. Data Collection and Transfer

[1137] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[1138] Device (smartphone app): The user takes a photo of their meal and the data is sent to a cloud server via the smartphone app. The app also captures and records a photo of the user's face and a short voice message, which collects emotional data.

[1139] 2. Data analysis and health advice generation

[1140] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[1141] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[1142] Server and Emotion Engine: The emotion engine analyzes facial photos and audio data to assess the user's emotional state, for example, assessing stress levels based on the frequency of smiles and tone of voice.

[1143] 3. Real-time notifications and preventative care

[1144] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[1145] Server: If the user's emotional state is unstable, send a notification saying "Take 10 deep breaths and relax."

[1146] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[1147] 4. Supporting access to medical care

[1148] Device (smartphone app): When the user receives a prevention alert and opens the app, they search for nearby medical institutions, select an appropriate medical institution, and make an appointment.

[1149] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[1150] The processing flow will be explained below.

[1151] Step 1:

[1152] Device (smartwatch): The user wears a smartwatch to measure data such as heart rate, number of steps, sleep time, and activity level. The smartwatch continuously measures this data and stores it in its internal memory for a certain period of time (for example, one day or several hours).

[1153] Step 2:

[1154] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi. The transfer occurs automatically at specific time intervals, and the data is deleted from the smartwatch's internal memory after transfer.

[1155] Step 3:

[1156] Device (smartphone app): The smartphone app uploads the data received from the smartwatch to the cloud server. When the upload is complete, the user is notified. Also, when the user eats a meal, the smartphone app takes a photo of the meal, adds a brief note (including ingredients and the name of the dish), and sends this to the cloud server.

[1157] Step 4:

[1158] User: Emotion data is collected by users taking photos of their faces and recording voice messages using a smartphone app. These data are also sent to the cloud server.

[1159] Step 5:

[1160] Server: The cloud server securely stores all data received from the smartwatch and smartphone, and saves it in a database for analysis, along with timestamps and user identification information.

[1161] Step 6:

[1162] Server: The cloud server performs the data analysis. Heart rate data is analyzed by an AI algorithm to calculate the average heart rate, resting heart rate, maximum and minimum heart rate over the past 24 hours, and detect abnormalities. Sleep data is also calculated to evaluate the ratio of deep sleep to light sleep and evaluate sleep quality.

[1163] Step 7:

[1164] Server: When analyzing meal data, image recognition technology is used to identify ingredients in photos of meals, and the calories and major nutrients of each ingredient are retrieved from a database to calculate total calories and nutritional balance.

[1165] Step 8:

[1166] Emotion engine: The emotion engine on the cloud server analyzes facial photos and voice data to assess the user's emotional state. For example, it calculates the user's stress level and emotional tendencies (happiness, sadness, anger, etc.) through facial expression analysis and tone of voice analysis.

[1167] Step 9:

[1168] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if a user is lacking in nutrients, the server will provide advice such as, "You are lacking in vitamin D, so try incorporating fish and mushrooms into your diet." Based on the results from the emotion engine, if the user's stress level is high, the server will suggest relaxation methods.

[1169] Step 10:

[1170] Server: The cloud server generates health advice and sends it to the user in real time via a messaging interface such as LINE. For example, a notification such as "You haven't exercised much today, so try stretching for 15 minutes" is sent.

[1171] Step 11:

[1172] Server: The cloud server analyzes the user's data over a long period of time and generates preventive alerts if signs of a deterioration in health are detected. For example, if the user has not had enough sleep for several days in a row, the server will send a message saying, "We recommend that you consult a doctor."

[1173] Step 12:

[1174] Device (smartphone app): The user receives a preventive alert and opens the smartphone app. Within the app, they search for a nearby medical institution, select an appropriate medical institution, and make an appointment. Once the appointment is completed, a reminder notification is sent to the user.

[1175] In this way, the system continuously monitors the user's health and emotional data and provides appropriate advice and preventative care in real time, helping to improve the user's health and quality of life.

[1176] Example 2

[1177] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1178] Current health management systems primarily measure and analyze biometric information, but few systems comprehensively analyze users' emotional and dietary data to provide health advice. Furthermore, there is a lack of real-time preventive alert notifications based on the user's health and emotional state, making it difficult for users to receive timely health improvement measures.

[1179] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1180] In this invention, the server includes a communication means for receiving measurement data from a device that measures the user's biological information, an analysis means for analyzing the received measurement data and the user's facial photograph and voice data, and a messaging interface for providing and transmitting health advice to the user in real time based on the analysis results. This makes it possible to comprehensively analyze the user's biological information, dietary data, and emotional state, provide the user with optimal health advice in real time, and promptly notify the user of preventive medical alerts.

[1181] "User's biological information" refers to data measured directly from the user's body, such as the user's heart rate, sleep duration, and activity level.

[1182] "Measurement data" refers to data collected by various sensors, including biometric information and emotional data of the user.

[1183] "Communication means" refers to technology that has the function of transferring data from devices such as smartwatches and smartphone apps to cloud servers, and includes Bluetooth and Wi-Fi.

[1184] "Analysis means" refers to the AI ​​algorithms and analysis programs used to analyze data received on the cloud server.

[1185] A "user's face photo" is image data of the user's face taken using a smartphone app.

[1186] "Voice data" refers to data of a user's voice recorded using a smartphone app.

[1187] "Health advice" refers to recommendations and advice regarding health management that is provided to the user based on the results of analysis by the analysis means.

[1188] "Notification means" refers to technology that has the function of notifying users of the generated health advice, and includes messaging services such as LINE.

[1189] A "messaging interface" is an interface for sending notifications from a cloud server to a user in real time, and includes messaging services that use APIs.

[1190] "Dietary data" refers to photos of meals taken by the user and data on the contents of meals self-reported by the user.

[1191] "Calories" refers to the amount of energy calculated from dietary data.

[1192] "Nutritional balance" refers to the proportion of nutrients contained in food.

[1193] A "preventive alert" is a notification of preventive medical advice or caution generated by the analysis means.

[1194] MODE FOR CARRYING OUT THE INVENTION

[1195] The following describes in detail the specific embodiments of the present invention. The present invention is a system for collecting and analyzing biometric and emotional data of a user and providing health advice. The main components of the system are as follows:

[1196] 1. User device (smartwatch)

[1197] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[1198] 3. Cloud Server

[1199] 4. User device (smartphone app)

[1200] 5. Messaging interface (e.g., LINE)

[1201] 6. Emotion Engine

[1202] Data collection

[1203] When a user wears a smartwatch, the collection of biometric information begins. The device (smartwatch) continuously measures the user's heart rate, sleep time, and activity level, and stores the data in its internal memory. In addition, the device (smartphone app) collects emotional data using the user's facial photos and voice data. For example, if a user wears a smartwatch with a heart rate monitoring function all day, heart rate data is automatically recorded. In addition, the user can take photos of their meals, and the data is sent to a cloud server via the smartphone app.

[1204] Data Transfer

[1205] The biometric data stored in the device (smartwatch) is periodically transferred to the device (smartphone app) via Bluetooth or Wi-Fi. The smartphone app then uploads this data to a cloud server. For example, the smartphone app periodically opens a Bluetooth connection to receive data and sends the data to the cloud server using the HTTP protocol.

[1206] Data analysis

[1207] The server receives the data sent to the cloud and analyzes it using AI algorithms. For example, heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies in the data. Meal data is used to calculate calories and nutritional balance using image recognition technology. An emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies. Specifically, the server analyzes the data using an AI algorithm built in Python and stores the results in a database. Examples of prompts include "Analyze today's heart rate data and tell me if there are any abnormalities" and "Calculate calories and nutritional balance from this meal photo."

[1208] Generating health advice

[1209] The server generates optimal health advice for the user based on the results of data analysis. For example, if a user is lacking in a particular nutrient, the server will suggest corresponding ingredients and recipes. If the user's stress level is high, relaxation methods will be suggested. Specifically, the server uses a pre-configured rule-based engine to select appropriate advice from a database. An example of a prompt could be, "Please assess the user's emotional state based on this facial photo and voice data."

[1210] Real-time notifications

[1211] The server sends the generated health advice to the user via a messaging interface such as LINE. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent. Specifically, the server uses the LINE API to generate a message and send it to the user. An example of a prompt is "Generate health advice from long-term sleep data."

[1212] Supporting preventative care and access to healthcare

[1213] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has had insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app. Specifically, the smartphone app uses GPS to list nearby medical institutions and connects with the reservation system to complete the appointment.

[1214] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[1215] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1216] Step 1: The user puts on the smartwatch.

[1217] Input: User's biological information (heart rate, sleep time, activity level)

[1218] How it works: The smartwatch's sensors measure the user's pulse and record it as heart rate data, while the accelerometer measures activity and estimates sleep time.

[1219] Output: Biometric data stored in internal memory

[1220] Step 2: The user takes a photo of their face using a smartphone app and records a voice message.

[1221] Input: User's face photo data, voice data

[1222] What it does: The smartphone camera takes a picture of the user's face, the microphone records their voice, and the app temporarily stores this data.

[1223] Output: Temporarily saved face photo data and audio data

[1224] Step 3: Transfer the biometric data stored on the smartwatch to your smartphone.

[1225] Input: Biometric data stored in the smartwatch's internal memory

[1226] What it does: Transfers smartwatch data to your smartphone via Bluetooth or Wi-Fi connection.

[1227] Output: Biometric data stored on a smartphone

[1228] Step 4: The smartphone app uploads all collected data to the cloud server.

[1229] Input: Biometric data, facial photo data, and voice data stored on the smartphone

[1230] Specific operation: The smartphone app sends data to the cloud server using the HTTP protocol.

[1231] Output: Biometric data, facial photo data, and voice data stored on a cloud server

[1232] Step 5: The cloud server analyzes the received data.

[1233] Input: Biometric data, facial photo data, and voice data stored on the cloud server

[1234] How it works: An AI algorithm on a cloud server analyzes heart rate data to detect 24-hour average heart rate and abnormalities. Image recognition technology is used to calculate calories and nutritional balance from photos of meals. An emotion engine also analyzes facial photos and voice data to assess stress levels and emotional states.

[1235] Output: Analysis results (health status, nutritional balance, emotional state)

[1236] Step 6: The cloud server generates health advice based on the analysis results.

[1237] Input: Analysis results (health status, nutritional balance, emotional state)

[1238] How it works: The cloud server uses a pre-configured rule-based engine to generate appropriate health advice.

[1239] Output: Generated health advice

[1240] Step 7: The cloud server notifies the user of the generated health advice in real time.

[1241] Input: Generated health advice

[1242] Specific operation: The cloud server generates a message using the LINE API and sends it to the user. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent.

[1243] Output: Health advice sent to the user

[1244] Step 8: The cloud server analyzes the long-term data and sends proactive alerts as needed.

[1245] Input: Long-term biometric data, emotional data

[1246] How it works: The cloud server analyzes past data trends and detects chronic problems. For example, if you experience insufficient sleep for several consecutive days, it generates a notification suggesting that you seek medical advice.

[1247] Output: Preventive alerts sent to users

[1248] Step 9: The user makes an appointment with a medical institution using the smartphone app.

[1249] Input: User's current location, preventative alerts

[1250] Specific operation: The smartphone app uses GPS to search for nearby medical institutions and connects with the reservation system to complete the appointment.

[1251] Output: Information about the medical institution where the reservation was completed

[1252] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice in real time, and quickly access medical care if needed.

[1253] (Application example 2)

[1254] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1255] Conventional health management systems provide health advice based on a user's biometric information, but do not offer service suggestions based on the customer's real-time emotional state, making it difficult to improve customer satisfaction, especially in brick-and-mortar stores. Furthermore, they lack the functionality to optimize services by utilizing in-store environmental data. This invention aims to solve these problems by providing a system that collects and analyzes customer emotional and environmental data and offers service suggestions based on that data.

[1256] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for measuring the user's biological information, means for receiving measurement data and environmental data from the device, means for analyzing the received measurement data and environmental data, means for providing health advice and service suggestions to the user based on the analysis results, and a messaging interface for transmitting the health advice and service suggestions. This makes it possible to improve the efficiency of store operations and customer satisfaction.

[1257] "Biometric information" refers to data relating to the user's health condition and bodily functions, and specifically includes heart rate, sleep duration, activity level, and the like.

[1258] "Communication means" refers to a means for transmitting measurement data from a device that measures biological information to other devices or servers, and utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[1259] The "analysis means" is a technology that processes the received measurement data and environmental data to evaluate and judge the user's health and emotional state, and uses artificial intelligence and machine learning algorithms.

[1260] "Notification means" refers to technology for notifying users and staff in real time of appropriate health advice and service suggestions based on the analysis results.

[1261] A "messaging interface" is a means of communication for conveying notifications to users and staff, and utilizes messaging applications such as LINE and WhatsApp.

[1262] "Service suggestions" are specific instructions and advice provided based on the user's emotional state and health status obtained by the analysis means, and are intended to improve the quality of service in physical stores.

[1263] "Environmental data" refers to information about factors that affect customer comfort and service provision, such as temperature, humidity, lighting, and foot traffic within the store.

[1264] "Real-time notification" is a technology that quickly conveys information to users and staff based on results obtained instantly by analytical means.

[1265] The present invention is a system for collecting and analyzing biometric and emotional data of users, and providing health advice and service suggestions based on the collected data. The system aims to improve customer satisfaction in brick-and-mortar stores and consists of several main components.

[1266] System Configuration

[1267] The main components of the system are:

[1268] 1. User device (smart glasses)

[1269] 2. Communication method (Wi-Fi)

[1270] 3. Cloud Server

[1271] 4. Notification devices (smartphone apps, messaging apps)

[1272] 5. Analysis engine (emotion engine and environmental data engine)

[1273] How to use

[1274] Data collection

[1275] When users (customers) wear smart glasses, their facial photos and facial expression data are collected by a camera. In-store environmental data (temperature, humidity, and traffic flow) is collected by sensors, and this data is transferred from the smart glasses to a cloud server in real time.

[1276] Data analysis

[1277] The cloud server analyzes the received biometric information and environmental data. It uses the Face Recognition API and Sentiment Analysis API to collect user emotional data from facial photos and facial expression data. It analyzes environmental data obtained from IoT devices and adjusts the store environment as needed.

[1278] Providing health advice and service suggestions

[1279] The cloud server generates advice and service suggestions for the user's health condition based on the analyzed data. Specifically, if the user is feeling stressed, it will provide advice on relaxation methods and mental care. In addition, store staff will be notified of the customer's service suggestions in real time.

[1280] Real-time notifications

[1281] The cloud server then sends the generated health advice and service suggestions to a smartphone app or messaging app via Wi-Fi. For example, if a customer is detected as feeling stressed, a specific service suggestion such as "Please serve hot tea to help the customer relax" can be sent to store staff.

[1282] Hardware and software used

[1283] Hardware: Smart glasses (camera, built-in sensors), IoT sensors (temperature, humidity, people flow sensors)

[1284] Software: Face Recognition API, Sentiment Analysis API, IoT Connectivity (Arduino, Raspberry Pi)

[1285] Cloud Platform: AWS (Amazon Web Services), Azure

[1286] Specific examples and examples of prompts for generative AI models

[1287] Examples:

[1288] If a customer smiles at the entrance, "The customer is relaxed. Please maintain the atmosphere in the store."

[1289] If the customer looks unhappy, say, "The customer's stress level is high. I suggest offering them a welcome drink."

[1290] Example prompts for generative AI models:

[1291] plaintext

[1292] "We want to develop an application that analyzes the stress levels of customers and suggests appropriate services based on the results. The input data will be photos of the customer's face taken with smart glasses and data on the in-store environment. We will use the Sentiment Analysis API for emotion analysis and suggest services in real time."

[1293] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1294] Step 1:

[1295] The user device (smart glasses) collects the user's facial photograph and facial expression data. The collected data includes the user's facial image and facial expression information such as smile or anger. The smart glasses temporarily store the captured image and facial expression data in their internal memory.

[1296] Step 2:

[1297] The communication method (Wi-Fi) periodically transfers the data stored in the smart glasses to a cloud server. The transferred data includes facial images, facial expression data, and environmental data (temperature, humidity, traffic flow, etc.). Once the cloud server has received the data, it proceeds to the next analysis step.

[1298] Step 3:

[1299] The cloud server uses the Face Recognition API to analyze the user's emotional data based on the received facial images. Specifically, it uses information obtained from facial expressions to evaluate the user's stress level and relaxation level. The input data is facial images and facial expression data, and the output data is an emotional evaluation report.

[1300] Step 4:

[1301] The cloud server analyzes the received environmental data. It evaluates the comfort level within the store based on the temperature, humidity, and people flow data received from the IoT devices. If necessary, it also generates adjustment instructions to optimize the environmental conditions. The input data are temperature, humidity, and people flow data, and the output data is an environmental assessment report.

[1302] Step 5:

[1303] The cloud server integrates the emotion data and environmental data to generate health advice and service suggestions for users and staff. For example, if the user is feeling stressed, a specific service suggestion is generated, such as "Please serve hot tea to help the customer relax." The input data are the emotion evaluation report and the environmental evaluation report, and the output data are the service suggestion message.

[1304] Step 6:

[1305] The cloud server notifies the generated health advice and service suggestions in real time. Notifications are sent via smartphone apps and messaging apps. Timely and appropriate service suggestions are communicated to staff through visual and audio notifications. The input data is the service suggestion message, and the output data is the notification message.

[1306] Through the above processing steps, users and staff can receive appropriate health advice and service suggestions in real time, which improves customer satisfaction and contributes to more efficient store operations.

[1307] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1308] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1309] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1310] [Fourth embodiment]

[1311] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1312] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1313] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1314] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1315] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1316] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1317] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1318] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1319] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1320] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1321] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1322] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1323] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1324] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[1325] System Configuration

[1326] The system includes the following main components to collect and analyze a user's biometric information and provide health advice:

[1327] 1. User device (smartwatch)

[1328] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[1329] 3. Cloud Server

[1330] 4. User device (smartphone app)

[1331] 5. Messaging interface (e.g., LINE)

[1332] Program processing

[1333] Data collection

[1334] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep duration, and activity level, and temporarily stores the data in its internal memory.

[1335] Data Transfer

[1336] The data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user takes a photo of their meal and enters the meal data into the app, it is also sent to the cloud server.

[1337] Data analysis

[1338] The cloud server analyzes all the data it receives. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Dietary data is then analyzed using image recognition technology to calculate calories and nutritional balance.

[1339] Generating health advice

[1340] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if the user's heart rate or stress level is abnormal, it will suggest measures to improve the situation or relaxation methods.

[1341] Real-time notifications

[1342] The cloud server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[1343] Supporting preventative care and access to healthcare

[1344] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user has consistently been experiencing insufficient sleep, it will send a notification saying, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[1345] Specific examples

[1346] 1. Data Collection and Transfer

[1347] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[1348] Terminal (smartphone app): The user takes a photo of the meal and the data is sent to the cloud server via the smartphone app.

[1349] 2. Data analysis and health advice generation

[1350] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[1351] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[1352] 3. Real-time notifications and preventative care

[1353] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[1354] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[1355] 4. Supporting access to medical care

[1356] Device (smartphone app): The user opens the app, selects from the provided list of medical institutions, and makes an appointment.

[1357] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[1358] The processing flow will be explained below.

[1359] Step 1:

[1360] Device (Smartwatch): The user wears a smartwatch to monitor their heart rate, steps, sleep, and activity. The smartwatch continuously measures these data and temporarily stores them in the device's internal memory.

[1361] Step 2:

[1362] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi at pre-set intervals, such as every 10 or 30 minutes.

[1363] Step 3:

[1364] Terminal (smartphone app): The smartphone imports the received data into the app and uploads it to the cloud server. The user is notified when the upload is complete.

[1365] Step 4:

[1366] User: When eating a meal, the user takes a photo of the meal using a smartphone app and adds a brief note (including ingredients and the name of the dish). After adding the information, the data is sent to the cloud server.

[1367] Step 5:

[1368] Server: The cloud server stores the received biometric information and dietary data and saves it in an analysis database. The database also records hourly data.

[1369] Step 6:

[1370] Server: Analyzes the stored data in real time. For heart rate data, it calculates the average heart rate and maximum and minimum heart rates over the past 24 hours, and for sleep data, it analyzes the quality of sleep (the ratio of deep sleep to light sleep).

[1371] Step 7:

[1372] Server: Analyzes photos of meal data using image recognition technology to identify ingredients. Retrieves the calories and nutrients of ingredients from a database and calculates total calorie and macronutrient intake.

[1373] Step 8:

[1374] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if excessive calorie intake is detected, it will suggest ingredients and recipes to reduce calorie intake.

[1375] Step 9:

[1376] Server: The generated health advice is sent to the user in real time via a messaging interface such as LINE. For example, a message such as "Since you haven't exercised much today, try doing 15 minutes of stretching" is delivered.

[1377] Step 10:

[1378] Server: Analyzes user data over a long period of time and generates preventative alerts if signs of a worsening health condition are detected. For example, if a user has not had enough sleep for several consecutive days, a notification will be sent stating that "consultation with a medical institution is recommended."

[1379] Step 11:

[1380] Device (smartphone app): The user receives a preventive alert, opens the app, searches for nearby medical institutions within the app, selects an appropriate medical institution, and makes an appointment.

[1381] Through these steps, users can understand their health status in real time and receive appropriate health advice, enabling them to manage their health and take preventative medicine.

[1382] Example 1

[1383] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1384] Health management has become increasingly important in recent years, and health management systems that utilize biometric information have been attracting attention. However, existing systems are limited in the amount of biometric information they can acquire, making it difficult to achieve comprehensive health management for users. Furthermore, dietary data acquisition and analysis are insufficient, making it difficult to properly evaluate a user's nutritional balance. Furthermore, real-time health advice and preventative alerts are often not provided, hindering users' prompt health management.

[1385] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1386] In this invention, the server includes a device for measuring a user's biometric information, memory means for temporarily storing the data, communication means, analysis means, image recognition means, notification means, and a messaging interface, which enable comprehensive collection and management of biometric information, detailed analysis of dietary data, and provision of real-time health advice and preventative alerts.

[1387] "Devices that measure biological information" are devices that measure biological data such as a user's heart rate, sleep time, and activity level, and generally refer to smartwatches and fitness trackers with heart rate monitoring functions.

[1388] "Memory means" refers to a storage device for temporarily storing measured biometric information, and corresponds to the internal memory of a smartwatch or smartphone.

[1389] "Communication means" refers to a device for transmitting measured data to other devices or servers, and includes wireless communication technologies such as Bluetooth and Wi-Fi.

[1390] "Analysis means" refers to means for analyzing received measurement data, and includes artificial intelligence algorithms and data analysis software.

[1391] "Image recognition means" refers to means for analyzing dietary data, and includes image analysis technology for analyzing photos of meals to calculate calories and nutrients.

[1392] "Notification means" refers to a means for providing health advice to users based on the analysis results, and includes smartphone notifications and message sending functions.

[1393] "Messaging interface" refers to a communication means for sending health advice to users, and includes messaging applications such as LINE and email.

[1394] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments for carrying out the present invention will be described in detail below.

[1395] System configuration

[1396] The system includes the following main components for collecting and analyzing a user's biometric information and providing health advice:

[1397] 1. User device (smartwatch)

[1398] This is a device that measures biometric information such as heart rate, sleep time, and activity level.

[1399] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[1400] It is a means for transmitting measurement data to other devices or servers.

[1401] 3. Cloud Server

[1402] This server analyzes the received data and provides health advice to users. The analysis uses artificial intelligence algorithms and image analysis technology.

[1403] 4. User device (smartphone app)

[1404] It is a means for uploading data transferred from a smartwatch to a cloud server.

[1405] 5. Messaging interface (e.g., LINE)

[1406] It is a means for sending health advice to users.

[1407] Program processing

[1408] Data collection

[1409] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory.

[1410] Data Transfer

[1411] The data stored on the device (smartwatch) is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi. The device (smartphone app) then uploads the data to a cloud server. When the user takes photos of their meals and enters them into the app, the meal data is also sent to the cloud server.

[1412] Data analysis

[1413] The server analyzes all the data it receives. Heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies. Dietary data is analyzed using image recognition technology to calculate calories and nutritional balance.

[1414] Generating health advice

[1415] The server generates optimal health advice for the user based on the analysis results. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. Also, if heart rate or stress levels are abnormal, it will suggest measures to improve or relaxation methods.

[1416] Real-time notifications

[1417] The server then notifies the user of the generated health advice in real time via messaging interfaces such as LINE, allowing the user to take appropriate action immediately.

[1418] Supporting preventative care and access to healthcare

[1419] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has consistently been experiencing insufficient sleep, the server will send a notification saying, "We recommend that you consult a medical institution." The user can easily make an appointment with a medical institution using their device (smartphone app).

[1420] Specific examples

[1421] 1. Data Collection and Transfer

[1422] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[1423] Users take photos of their meals and the data is sent to a cloud server via a smartphone app.

[1424] 2. Data analysis and health advice generation

[1425] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[1426] Server: Analyzes dietary data, identifies nutrient deficiencies, and suggests corresponding ingredient lists and recipes.

[1427] 3. Real-time notifications and preventative care

[1428] Server: Sends a message to the user via LINE, such as "We recommend walking for 20 minutes today."

[1429] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[1430] 4. Supporting access to medical care

[1431] The user opens the smartphone app, selects from the provided list of medical institutions, and makes an appointment.

[1432] Examples of prompt statements

[1433] Prompt text when user enters meal data:

[1434] Take a photo of what you had for dinner last night and send it to us along with the details of the meal. For example, write "Salad, steak, and rice."

[1435] This system allows users to accurately understand their health condition on a daily basis, receive personalized health advice, and quickly access medical institutions when necessary, thereby contributing to extending healthy life expectancy.

[1436] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1437] Step 1:

[1438] Input: The user puts on the smartwatch.

[1439] How it works: When a user wears a smartwatch with heart rate monitoring throughout the day, it automatically starts collecting heart rate, vital signs, and sleep data. The data is temporarily stored in the smartwatch's internal memory.

[1440] Output: Temporarily stored heart rate, vital activity and sleep data.

[1441] Step 2:

[1442] Input: Biometric data stored on the smartwatch.

[1443] Specific operation: The device (smartwatch) periodically transfers data to the user's smartphone via Bluetooth or Wi-Fi. During the transfer process, the user checks the smartphone's connection and pairs it with the smartwatch if necessary.

[1444] Output: Biometric data transmitted to a smartphone.

[1445] Step 3:

[1446] Input: Biometric data transferred to smartphone.

[1447] Specific operation: The device (smartphone app) uploads all biometric data, including data entered into the app by the user after taking photos of their meals, to a cloud server. The user can initiate the upload by pressing the "Data Sync" button in the app.

[1448] Output: Biometric data and dietary data uploaded to a cloud server.

[1449] Step 4:

[1450] Input: Biometric data and dietary data uploaded to a cloud server.

[1451] How it works: The server uses AI algorithms to analyze the received data. It analyzes the heart rate data to detect the average heart rate over 24 hours and fluctuations in heart rate during sleep. It also uses image recognition technology to calculate calories and nutritional balance from the dietary data.

[1452] Output: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[1453] Step 5:

[1454] Input: Analyzed heart rate data, sleep data, calorie and nutritional balance data.

[1455] How it works: Based on the analysis results, the server generates optimal health advice for the user. For example, if your heart rate is too high, it will suggest relaxation techniques, or if you are lacking certain nutrients, it will suggest the necessary ingredients and recipes.

[1456] Output: The generated health advice.

[1457] Step 6:

[1458] Input: Generated health advice.

[1459] Specific operation: The server notifies the user of health advice in real time, sending messages such as "We recommend walking for 20 minutes today" via a messaging interface such as LINE.

[1460] Output: Health advice notification sent to user.

[1461] Step 7:

[1462] Input: Analysis results of long-term biometric data.

[1463] Specific operation: The server analyzes data over a long period of time, and if it detects an abnormality, such as persistent lack of sleep, it generates and sends a preventative alert such as "We recommend that you consult a medical institution."

[1464] Output: Proactive alerts sent to users.

[1465] Step 8:

[1466] Input: Medical consultation advice sent to the user.

[1467] Specific operation: The user opens the smartphone app and makes an appointment by selecting from the provided list of medical institutions. The user selects the "Medical Institution Appointment" tab in the app and specifies the desired date and time.

[1468] Output: Medical appointment.

[1469] (Application example 1)

[1470] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1471] This invention relates to a system for monitoring worker health and ensuring a safe working environment. In particular, it aims to prevent health problems caused by overwork and stress by monitoring workers' biological information in real time and providing appropriate health advice and preventive measures based on the results. Another objective is to improve worker productivity and safety by improving the working environment.

[1472] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1473] In this invention, the server includes a means for measuring a user's biometric information, a means for receiving measurement data from the device, a means for analyzing the received measurement data, a means for providing health advice to the user based on the analysis results, a means for transmitting the health advice, a means for monitoring the worker's health status based on the user's biometric information, a means for improving the work environment based on the monitoring results, and a means for displaying the advice in real time. This allows for continuous monitoring of the worker's health status and immediate notification of any abnormalities. Furthermore, by providing appropriate health advice and preventive alerts, health problems caused by overwork and stress can be prevented and a safe work environment can be maintained.

[1474] "User" means an individual who utilizes the System to provide biometric information and receive health monitoring and advice.

[1475] "Biometric information" refers to data about your physical condition, such as your heart rate, activity level, and body temperature.

[1476] A "measuring device" is a device for measuring a user's biometric information, and includes, for example, a smartwatch or fitness tracker.

[1477] "Communication means" refers to a means for transmitting data acquired from a measuring device to the system, and may use Bluetooth, Wi-Fi, etc.

[1478] The "analysis means" is a means for evaluating and analyzing the health condition based on the received biometric information data.

[1479] "Notification means" refers to a means for notifying the user of health advice and warnings based on the analysis results.

[1480] A "messaging interface" is a means of communication for sending notifications and advice to users, and includes, for example, LINE and email.

[1481] "Monitoring means" refers to a means for continuously checking the health status of a user based on their biometric information and detecting any abnormalities or changes.

[1482] "Work environment improvement measures" are measures to provide users with specific advice and measures to improve their work environment based on the monitoring results.

[1483] A "display device" is a device that displays advice and warnings to users in real time, and includes digital signs in factories and smartphones.

[1484] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[1485] System Configuration

[1486] The system includes the following main components to collect and analyze users' biometric information and provide advice to improve the working environment:

[1487] 1. Measuring device (smartwatch): worn by the user.

[1488] 2. Communication means (Bluetooth, Wi-Fi): Receives and transfers measurement data.

[1489] 3. Local server: Installed within the factory.

[1490] 4. Cloud server: Analyzes the data.

[1491] 5. Smartphone app: Notifies users.

[1492] 6. Display device (digital signage): displays advice in real time.

[1493] 7. Robots: Monitor the health of workers in factories.

[1494] Program processing

[1495] Data collection

[1496] Once the user wears the smartwatch, biometric data collection begins: the smartwatch continuously measures the user's heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory.

[1497] Data Transfer

[1498] The collected data is transferred to a local server via Bluetooth or Wi-Fi, and then uploaded from the local server to a cloud server.

[1499] Data analysis

[1500] The cloud server analyzes all received data and uses AI algorithms (using TensorFlow) to detect patterns and abnormalities in biometric information.

[1501] Generating Advice

[1502] Based on the analysis results, the cloud server generates optimal health advice for the user, for example, recommending rest if stress levels are high.

[1503] Real-time notifications

[1504] The cloud server then sends the generated health advice to users in real time via a smartphone app or digital signage.

[1505] Supporting preventative care and access to healthcare

[1506] The cloud server analyzes long-term data and sends preventative alerts as needed, including automatic notification to medical institutions if necessary.

[1507] Hardware and software used

[1508] Hardware:

[1509] Smartwatch

[1510] Local Server

[1511] Digital signage in factories

[1512] robot

[1513] software:

[1514] Android SDK (smartphone app development)

[1515] TensorFlow (AI algorithm implementation)

[1516] Google Cloud (Cloud Analytics)

[1517] Kubernetes (cloud server management)

[1518] Bluetooth API (data transfer)

[1519] Specific example explanation

[1520] Factory workers wear smartwatches, and robots collect their biometric data. This data is transmitted to a local server via Bluetooth and then uploaded to a cloud server. AI algorithms on the cloud server analyze the data and monitor the workers' health. If any abnormalities are detected, they are notified in real time via digital signs in the factory and a smartphone app.

[1521] Example prompt sentence:

[1522] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[1523] In this way, it is possible to provide real-time care for workers in factories and provide a safe and efficient working environment.

[1524] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1525] Step 1:

[1526] When the user wears the smartwatch, biometric data collection begins. The smartwatch continuously measures data such as heart rate, activity level, and body temperature, and temporarily stores the data in its internal memory. The input is the user's physical data, and the output is the data stored in the smartwatch's internal memory.

[1527] Step 2:

[1528] The data stored in the smartwatch is transferred to a local server via Bluetooth or Wi-Fi. Specifically, the smartwatch batch processes the measurement data at regular intervals and transmits the data through communication with the local server. The input is the data in the smartwatch's internal memory, and the output is the data transferred to the local server.

[1529] Step 3:

[1530] The local server temporarily stores the received data and uploads it to the cloud server via the Internet. Data processing on the local server includes data formatting and format conversion. The input is the data transferred to the local server, and the output is the data uploaded to the cloud server.

[1531] Step 4:

[1532] The cloud server analyzes the received data using an AI algorithm (using TensorFlow). Specifically, it performs pattern recognition and anomaly detection. The input is the biometric data uploaded to the cloud server, and the output is the analysis results, i.e., an evaluation of health status and anomaly detection results.

[1533] Step 5:

[1534] The cloud server generates health advice and warnings for the user based on the analysis results. Specifically, it uses a generative AI model to create appropriate messages based on the analysis results. The input is the analysis results, and the output is health advice and warning messages.

[1535] Step 6:

[1536] The generated health advice and warning messages are notified to users in real time using a smartphone app or digital signs in the factory. Specifically, the cloud server issues the notifications and sends them to each device via a messaging interface. The input is the health advice or warning message, and the output is the notification sent to the user.

[1537] Step 7:

[1538] The cloud server accumulates long-term data, periodically analyzes it, and sends preventive alerts to users. This includes trend analysis of past data and anomaly detection. The input is long-term accumulated biometric data, and the output is preventive alerts.

[1539] A specific example includes steps to generate the following prompt statement if the user's heart rate increases by more than 20% of normal:

[1540] "Your heart rate has increased by more than 20% above normal. If this condition continues it may be detrimental to your health, so we recommend that you take a 15-minute break."

[1541] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1542] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to a specific system configuration and a method of using the system.

[1543] System Configuration

[1544] The system includes the following main components to collect and analyze users' biometric and emotional data and provide health advice:

[1545] 1. User device (smartwatch)

[1546] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[1547] 3. Cloud Server

[1548] 4. User device (smartphone app)

[1549] 5. Messaging interface (e.g., LINE)

[1550] 6. Emotion Engine

[1551] Program processing

[1552] Data collection

[1553] When a user wears a smartwatch, biometric data collection begins. The smartwatch continuously measures the user's heart rate, sleep time, and activity level, and temporarily stores the data in its internal memory. In addition, a smartphone app collects emotional data using the user's facial photos and voice data.

[1554] Data Transfer

[1555] The biometric data stored in the smartwatch is periodically transferred to the user's smartphone via Bluetooth or Wi-Fi and uploaded to a cloud server via a smartphone app. When the user enters photos of their meals or self-reported data into the app, the data is also sent to the cloud server.

[1556] Data analysis

[1557] The cloud server analyzes all received data. In particular, heart rate and sleep data are analyzed using AI algorithms to detect patterns and anomalies. Food data is analyzed using image recognition technology to calculate calories and nutritional balance. The emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies.

[1558] Generating health advice

[1559] Based on the analysis results, the cloud server generates optimal health advice for the user. For example, if dietary data indicates a deficiency in a particular nutrient, it will suggest ingredients and recipes containing the necessary nutrients. If the emotion engine evaluates a user's stress level as high, it will suggest relaxation methods and mental care advice. The advice content is also customized based on the user's emotional state.

[1560] Real-time notifications

[1561] The cloud server then sends the generated health advice to the user in real time via messaging interfaces such as LINE. For example, if the emotion engine detects that the user's stress level is high, it will send a message such as "Take 10 deep breaths and relax."

[1562] Supporting preventative care and access to healthcare

[1563] The cloud server analyzes long-term data and sends preventative alerts as needed. For example, if a user experiences insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app.

[1564] Specific examples

[1565] 1. Data Collection and Transfer

[1566] Device (smartwatch): When a user wears a smartwatch with heart rate monitoring function all day, heart rate data is automatically recorded.

[1567] Device (smartphone app): The user takes a photo of their meal and the data is sent to a cloud server via the smartphone app. The app also captures and records a photo of the user's face and a short voice message, which collects emotional data.

[1568] 2. Data analysis and health advice generation

[1569] Server: The cloud server analyzes the user's heart rate data, monitors the 24-hour average heart rate and heart rate fluctuations during sleep, and generates an alert if an abnormality is detected.

[1570] Server: Analyzes dietary data, identifies nutrient deficiencies from the diet, and suggests corresponding ingredient lists and recipes.

[1571] Server and Emotion Engine: The emotion engine analyzes facial photos and audio data to assess the user's emotional state, for example, assessing stress levels based on the frequency of smiles and tone of voice.

[1572] 3. Real-time notifications and preventative care

[1573] Server: Based on the analysis results, a message such as "We recommend walking for 20 minutes today" is sent to the user via LINE.

[1574] Server: If the user's emotional state is unstable, send a notification saying "Take 10 deep breaths and relax."

[1575] Server: Monitors the user's sleep data over a long period of time, and if chronic sleep deprivation is detected, sends a notification stating, "Consider consulting a medical professional."

[1576] 4. Supporting access to medical care

[1577] Device (smartphone app): When the user receives a prevention alert and opens the app, they search for nearby medical institutions, select an appropriate medical institution, and make an appointment.

[1578] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[1579] The processing flow will be explained below.

[1580] Step 1:

[1581] Device (smartwatch): The user wears a smartwatch to measure data such as heart rate, number of steps, sleep time, and activity level. The smartwatch continuously measures this data and stores it in its internal memory for a certain period of time (for example, one day or several hours).

[1582] Step 2:

[1583] Device (smartwatch): The smartwatch periodically transfers data to the smartphone via Bluetooth or Wi-Fi. The transfer occurs automatically at specific time intervals, and the data is deleted from the smartwatch's internal memory after transfer.

[1584] Step 3:

[1585] Device (smartphone app): The smartphone app uploads the data received from the smartwatch to the cloud server. When the upload is complete, the user is notified. Also, when the user eats a meal, the smartphone app takes a photo of the meal, adds a brief note (including ingredients and the name of the dish), and sends this to the cloud server.

[1586] Step 4:

[1587] User: Emotion data is collected by users taking photos of their faces and recording voice messages using a smartphone app. These data are also sent to the cloud server.

[1588] Step 5:

[1589] Server: The cloud server securely stores all data received from the smartwatch and smartphone, and saves it in a database for analysis, along with timestamps and user identification information.

[1590] Step 6:

[1591] Server: The cloud server performs the data analysis. Heart rate data is analyzed by an AI algorithm to calculate the average heart rate, resting heart rate, maximum and minimum heart rate over the past 24 hours, and detect abnormalities. Sleep data is also calculated to evaluate the ratio of deep sleep to light sleep and evaluate sleep quality.

[1592] Step 7:

[1593] Server: When analyzing meal data, image recognition technology is used to identify ingredients in photos of meals, and the calories and major nutrients of each ingredient are retrieved from a database to calculate total calories and nutritional balance.

[1594] Step 8:

[1595] Emotion engine: The emotion engine on the cloud server analyzes facial photos and voice data to assess the user's emotional state. For example, it calculates the user's stress level and emotional tendencies (happiness, sadness, anger, etc.) through facial expression analysis and tone of voice analysis.

[1596] Step 9:

[1597] Server: Based on the analysis results, the AI ​​algorithm generates optimal health advice for the user. For example, if a user is lacking in nutrients, the server will provide advice such as, "You are lacking in vitamin D, so try incorporating fish and mushrooms into your diet." Based on the results from the emotion engine, if the user's stress level is high, the server will suggest relaxation methods.

[1598] Step 10:

[1599] Server: The cloud server generates health advice and sends it to the user in real time via a messaging interface such as LINE. For example, a notification such as "You haven't exercised much today, so try stretching for 15 minutes" is sent.

[1600] Step 11:

[1601] Server: The cloud server analyzes the user's data over a long period of time and generates preventive alerts if signs of a deterioration in health are detected. For example, if the user has not had enough sleep for several days in a row, the server will send a message saying, "We recommend that you consult a doctor."

[1602] Step 12:

[1603] Device (smartphone app): The user receives a preventive alert and opens the smartphone app. Within the app, they search for a nearby medical institution, select an appropriate medical institution, and make an appointment. Once the appointment is completed, a reminder notification is sent to the user.

[1604] In this way, the system continuously monitors the user's health and emotional data and provides appropriate advice and preventative care in real time, helping to improve the user's health and quality of life.

[1605] Example 2

[1606] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1607] Current health management systems primarily measure and analyze biometric information, but few systems comprehensively analyze users' emotional and dietary data to provide health advice. Furthermore, there is a lack of real-time preventive alert notifications based on the user's health and emotional state, making it difficult for users to receive timely health improvement measures.

[1608] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1609] In this invention, the server includes a communication means for receiving measurement data from a device that measures the user's biological information, an analysis means for analyzing the received measurement data and the user's facial photograph and voice data, and a messaging interface for providing and transmitting health advice to the user in real time based on the analysis results. This makes it possible to comprehensively analyze the user's biological information, dietary data, and emotional state, provide the user with optimal health advice in real time, and promptly notify the user of preventive medical alerts.

[1610] "User's biological information" refers to data measured directly from the user's body, such as the user's heart rate, sleep duration, and activity level.

[1611] "Measurement data" refers to data collected by various sensors, including biometric information and emotional data of the user.

[1612] "Communication means" refers to technology that has the function of transferring data from devices such as smartwatches and smartphone apps to cloud servers, and includes Bluetooth and Wi-Fi.

[1613] "Analysis means" refers to the AI ​​algorithms and analysis programs used to analyze data received on the cloud server.

[1614] A "user's face photo" is image data of the user's face taken using a smartphone app.

[1615] "Voice data" refers to data of a user's voice recorded using a smartphone app.

[1616] "Health advice" refers to recommendations and advice regarding health management that is provided to the user based on the results of analysis by the analysis means.

[1617] "Notification means" refers to technology that has the function of notifying users of the generated health advice, and includes messaging services such as LINE.

[1618] A "messaging interface" is an interface for sending notifications from a cloud server to a user in real time, and includes messaging services that use APIs.

[1619] "Dietary data" refers to photos of meals taken by the user and data on the contents of meals self-reported by the user.

[1620] "Calories" refers to the amount of energy calculated from dietary data.

[1621] "Nutritional balance" refers to the proportion of nutrients contained in food.

[1622] A "preventive alert" is a notification of preventive medical advice or caution generated by the analysis means.

[1623] MODE FOR CARRYING OUT THE INVENTION

[1624] The following describes in detail the specific embodiments of the present invention. The present invention is a system for collecting and analyzing biometric and emotional data of a user and providing health advice. The main components of the system are as follows:

[1625] 1. User device (smartwatch)

[1626] 2. Communication means (Bluetooth, Wi-Fi, etc.)

[1627] 3. Cloud Server

[1628] 4. User device (smartphone app)

[1629] 5. Messaging interface (e.g., LINE)

[1630] 6. Emotion Engine

[1631] Data collection

[1632] When a user wears a smartwatch, the collection of biometric information begins. The device (smartwatch) continuously measures the user's heart rate, sleep time, and activity level, and stores the data in its internal memory. In addition, the device (smartphone app) collects emotional data using the user's facial photos and voice data. For example, if a user wears a smartwatch with a heart rate monitoring function all day, heart rate data is automatically recorded. In addition, the user can take photos of their meals, and the data is sent to a cloud server via the smartphone app.

[1633] Data Transfer

[1634] The biometric data stored in the device (smartwatch) is periodically transferred to the device (smartphone app) via Bluetooth or Wi-Fi. The smartphone app then uploads this data to a cloud server. For example, the smartphone app periodically opens a Bluetooth connection to receive data and sends the data to the cloud server using the HTTP protocol.

[1635] Data analysis

[1636] The server receives the data sent to the cloud and analyzes it using AI algorithms. For example, heart rate and sleep data are analyzed by AI algorithms to detect patterns and anomalies in the data. Meal data is used to calculate calories and nutritional balance using image recognition technology. An emotion engine identifies the user's emotions based on facial photos and voice data, and evaluates stress levels and emotional tendencies. Specifically, the server analyzes the data using an AI algorithm built in Python and stores the results in a database. Examples of prompts include "Analyze today's heart rate data and tell me if there are any abnormalities" and "Calculate calories and nutritional balance from this meal photo."

[1637] Generating health advice

[1638] The server generates optimal health advice for the user based on the results of data analysis. For example, if a user is lacking in a particular nutrient, the server will suggest corresponding ingredients and recipes. If the user's stress level is high, relaxation methods will be suggested. Specifically, the server uses a pre-configured rule-based engine to select appropriate advice from a database. An example of a prompt could be, "Please assess the user's emotional state based on this facial photo and voice data."

[1639] Real-time notifications

[1640] The server sends the generated health advice to the user via a messaging interface such as LINE. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent. Specifically, the server uses the LINE API to generate a message and send it to the user. An example of a prompt is "Generate health advice from long-term sleep data."

[1641] Supporting preventative care and access to healthcare

[1642] The server analyzes long-term data and sends preventative alerts as necessary. For example, if a user has had insufficient sleep for several consecutive days, a notification will be sent stating, "We recommend that you consult a medical institution." Users can also easily make appointments with medical institutions using a smartphone app. Specifically, the smartphone app uses GPS to list nearby medical institutions and connects with the reservation system to complete the appointment.

[1643] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice, and quickly access medical institutions when necessary, contributing to improving their health and quality of life.

[1644] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1645] Step 1: The user puts on the smartwatch.

[1646] Input: User's biological information (heart rate, sleep time, activity level)

[1647] How it works: The smartwatch's sensors measure the user's pulse and record it as heart rate data, while the accelerometer measures activity and estimates sleep time.

[1648] Output: Biometric data stored in internal memory

[1649] Step 2: The user takes a photo of their face using a smartphone app and records a voice message.

[1650] Input: User's face photo data, voice data

[1651] What it does: The smartphone camera takes a picture of the user's face, the microphone records their voice, and the app temporarily stores this data.

[1652] Output: Temporarily saved face photo data and audio data

[1653] Step 3: Transfer the biometric data stored on the smartwatch to your smartphone.

[1654] Input: Biometric data stored in the smartwatch's internal memory

[1655] What it does: Transfers smartwatch data to your smartphone via Bluetooth or Wi-Fi connection.

[1656] Output: Biometric data stored on a smartphone

[1657] Step 4: The smartphone app uploads all collected data to the cloud server.

[1658] Input: Biometric data, facial photo data, and voice data stored on the smartphone

[1659] Specific operation: The smartphone app sends data to the cloud server using the HTTP protocol.

[1660] Output: Biometric data, facial photo data, and voice data stored on a cloud server

[1661] Step 5: The cloud server analyzes the received data.

[1662] Input: Biometric data, facial photo data, and voice data stored on the cloud server

[1663] How it works: An AI algorithm on a cloud server analyzes heart rate data to detect 24-hour average heart rate and abnormalities. Image recognition technology is used to calculate calories and nutritional balance from photos of meals. An emotion engine also analyzes facial photos and voice data to assess stress levels and emotional states.

[1664] Output: Analysis results (health status, nutritional balance, emotional state)

[1665] Step 6: The cloud server generates health advice based on the analysis results.

[1666] Input: Analysis results (health status, nutritional balance, emotional state)

[1667] How it works: The cloud server uses a pre-configured rule-based engine to generate appropriate health advice.

[1668] Output: Generated health advice

[1669] Step 7: The cloud server notifies the user of the generated health advice in real time.

[1670] Input: Generated health advice

[1671] Specific operation: The cloud server generates a message using the LINE API and sends it to the user. For example, if a high stress level is detected, a message such as "Take 10 deep breaths and relax" is sent.

[1672] Output: Health advice sent to the user

[1673] Step 8: The cloud server analyzes the long-term data and sends proactive alerts as needed.

[1674] Input: Long-term biometric data, emotional data

[1675] How it works: The cloud server analyzes past data trends and detects chronic problems. For example, if you experience insufficient sleep for several consecutive days, it generates a notification suggesting that you seek medical advice.

[1676] Output: Preventive alerts sent to users

[1677] Step 9: The user makes an appointment with a medical institution using the smartphone app.

[1678] Input: User's current location, preventative alerts

[1679] Specific operation: The smartphone app uses GPS to search for nearby medical institutions and connects with the reservation system to complete the appointment.

[1680] Output: Information about the medical institution where the reservation was completed

[1681] This allows users to accurately understand their health and emotional state, receive personalized health and mental care advice in real time, and quickly access medical care if needed.

[1682] (Application example 2)

[1683] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] Conventional health management systems provide health advice based on a user's biometric information, but do not offer service suggestions based on the customer's real-time emotional state, making it difficult to improve customer satisfaction, especially in brick-and-mortar stores. Furthermore, they lack the functionality to optimize services by utilizing in-store environmental data. This invention aims to solve these problems by providing a system that collects and analyzes customer emotional and environmental data and offers service suggestions based on that data.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for measuring the user's biological information, means for receiving measurement data and environmental data from the device, means for analyzing the received measurement data and environmental data, means for providing health advice and service suggestions to the user based on the analysis results, and a messaging interface for transmitting the health advice and service suggestions. This makes it possible to improve the efficiency of store operations and customer satisfaction.

[1686] "Biometric information" refers to data relating to the user's health condition and bodily functions, and specifically includes heart rate, sleep duration, activity level, and the like.

[1687] "Communication means" refers to a means for transmitting measurement data from a device that measures biological information to other devices or servers, and utilizes wireless communication technologies such as Bluetooth and Wi-Fi.

[1688] The "analysis means" is a technology that processes the received measurement data and environmental data to evaluate and judge the user's health and emotional state, and uses artificial intelligence and machine learning algorithms.

[1689] "Notification means" refers to technology for notifying users and staff in real time of appropriate health advice and service suggestions based on the analysis results.

[1690] A "messaging interface" is a means of communication for conveying notifications to users and staff, and utilizes messaging applications such as LINE and WhatsApp.

[1691] "Service suggestions" are specific instructions and advice provided based on the user's emotional state and health status obtained by the analysis means, and are intended to improve the quality of service in physical stores.

[1692] "Environmental data" refers to information about factors that affect customer comfort and service provision, such as temperature, humidity, lighting, and foot traffic within the store.

[1693] "Real-time notification" is a technology that quickly conveys information to users and staff based on results obtained instantly by analytical means.

[1694] The present invention is a system for collecting and analyzing biometric and emotional data of users, and providing health advice and service suggestions based on the collected data. The system aims to improve customer satisfaction in brick-and-mortar stores and consists of several main components.

[1695] System Configuration

[1696] The main components of the system are:

[1697] 1. User device (smart glasses)

[1698] 2. Communication method (Wi-Fi)

[1699] 3. Cloud Server

[1700] 4. Notification devices (smartphone apps, messaging apps)

[1701] 5. Analysis engine (emotion engine and environmental data engine)

[1702] How to use

[1703] Data collection

[1704] When users (customers) wear smart glasses, their facial photos and facial expression data are collected by a camera. In-store environmental data (temperature, humidity, and traffic flow) is collected by sensors, and this data is transferred from the smart glasses to a cloud server in real time.

[1705] Data analysis

[1706] The cloud server analyzes the received biometric information and environmental data. It uses the Face Recognition API and Sentiment Analysis API to collect user emotional data from facial photos and facial expression data. It analyzes environmental data obtained from IoT devices and adjusts the store environment as needed.

[1707] Providing health advice and service suggestions

[1708] The cloud server generates advice and service suggestions for the user's health condition based on the analyzed data. Specifically, if the user is feeling stressed, it will provide advice on relaxation methods and mental care. In addition, store staff will be notified of the customer's service suggestions in real time.

[1709] Real-time notifications

[1710] The cloud server then sends the generated health advice and service suggestions to a smartphone app or messaging app via Wi-Fi. For example, if a customer is detected as feeling stressed, a specific service suggestion such as "Please serve hot tea to help the customer relax" can be sent to store staff.

[1711] Hardware and software used

[1712] Hardware: Smart glasses (camera, built-in sensors), IoT sensors (temperature, humidity, people flow sensors)

[1713] Software: Face Recognition API, Sentiment Analysis API, IoT Connectivity (Arduino, Raspberry Pi)

[1714] Cloud Platform: AWS (Amazon Web Services), Azure

[1715] Specific examples and examples of prompts for generative AI models

[1716] Examples:

[1717] If a customer smiles at the entrance, "The customer is relaxed. Please maintain the atmosphere in the store."

[1718] If the customer looks unhappy, say, "The customer's stress level is high. I suggest offering them a welcome drink."

[1719] Example prompts for generative AI models:

[1720] plaintext

[1721] "We want to develop an application that analyzes the stress levels of customers and suggests appropriate services based on the results. The input data will be photos of the customer's face taken with smart glasses and data on the in-store environment. We will use the Sentiment Analysis API for emotion analysis and suggest services in real time."

[1722] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1723] Step 1:

[1724] The user device (smart glasses) collects the user's facial photograph and facial expression data. The collected data includes the user's facial image and facial expression information such as smile or anger. The smart glasses temporarily store the captured image and facial expression data in their internal memory.

[1725] Step 2:

[1726] The communication method (Wi-Fi) periodically transfers the data stored in the smart glasses to a cloud server. The transferred data includes facial images, facial expression data, and environmental data (temperature, humidity, traffic flow, etc.). Once the cloud server has received the data, it proceeds to the next analysis step.

[1727] Step 3:

[1728] The cloud server uses the Face Recognition API to analyze the user's emotional data based on the received facial images. Specifically, it uses information obtained from facial expressions to evaluate the user's stress level and relaxation level. The input data is facial images and facial expression data, and the output data is an emotional evaluation report.

[1729] Step 4:

[1730] The cloud server analyzes the received environmental data. It evaluates the comfort level within the store based on the temperature, humidity, and people flow data received from the IoT devices. If necessary, it also generates adjustment instructions to optimize the environmental conditions. The input data are temperature, humidity, and people flow data, and the output data is an environmental assessment report.

[1731] Step 5:

[1732] The cloud server integrates the emotion data and environmental data to generate health advice and service suggestions for users and staff. For example, if the user is feeling stressed, a specific service suggestion is generated, such as "Please serve hot tea to help the customer relax." The input data are the emotion evaluation report and the environmental evaluation report, and the output data are the service suggestion message.

[1733] Step 6:

[1734] The cloud server notifies the generated health advice and service suggestions in real time. Notifications are sent via smartphone apps and messaging apps. Timely and appropriate service suggestions are communicated to staff through visual and audio notifications. The input data is the service suggestion message, and the output data is the notification message.

[1735] Through the above processing steps, users and staff can receive appropriate health advice and service suggestions in real time, which improves customer satisfaction and contributes to more efficient store operations.

[1736] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1737] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1738] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1739] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1740] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1741] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1742] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1743] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1744] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1745] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1746] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1747] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1748] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1749] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1750] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1751] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1752] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1753] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1754] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1755] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1756] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1757] The following is further disclosed regarding the above embodiment.

[1758] (Claim 1)

[1759] a device for measuring biometric information of a user;

[1760] a communication means for receiving measurement data from the device;

[1761] analysis means for analyzing the received measurement data;

[1762] a notification means for providing health advice to the user based on the analysis results;

[1763] The system includes a messaging interface for sending the health advice.

[1764] (Claim 2)

[1765] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing the dietary data of the user and calculating calories and nutritional balance.

[1766] (Claim 3)

[1767] 10. The system of claim 1, wherein the notification means includes means for sending preventative alerts in real time based on the user's health status.

[1768] (Claim 4)

[1769] 2. The system of claim 1, wherein the analyzing means includes means for analyzing the user's sleep data and generating advice for improving sleep quality.

[1770] (Claim 5)

[1771] 2. The system of claim 1, wherein the analyzing means includes means for analyzing the user's stress level and providing counseling for stress management.

[1772] "Example 1"

[1773] (Claim 1)

[1774] a device for measuring biometric information of a user;

[1775] memory means for temporarily storing data;

[1776] a communication means for receiving measurement data from the device;

[1777] analysis means for analyzing the received measurement data;

[1778] Image recognition means for analyzing dietary data;

[1779] a notification means for providing health advice to the user based on the analysis results;

[1780] The system includes a messaging interface for sending the health advice.

[1781] (Claim 2)

[1782] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing the dietary data of the user and calculating calories and nutritional balance.

[1783] (Claim 3)

[1784] 10. The system of claim 1, wherein the notification means includes means for sending preventative alerts in real time based on the user's health status.

[1785] "Application Example 1"

[1786] (Claim 1)

[1787] A device for measuring biometric information of a user;

[1788] a communication means for receiving measurement data from the device;

[1789] analysis means for analyzing the received measurement data;

[1790] a notification means for providing health advice to the user based on the analysis results;

[1791] a messaging interface for sending said health advice;

[1792] a means for monitoring the health status of workers based on the biometric information of the users;

[1793] means for providing advice to improve the working environment based on the monitoring results;

[1794] a display device that displays the advice in real time;

[1795] A system including:

[1796] (Claim 2)

[1797] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing the dietary data of the user and calculating calories and nutritional balance.

[1798] (Claim 3)

[1799] 10. The system of claim 1, wherein the notification means includes means for sending preventative alerts in real time based on the user's health status.

[1800] "Example 2: Combining Emotion Engines"

[1801] (Claim 1)

[1802] a device for measuring biometric information of a user;

[1803] a communication means for receiving measurement data from the device;

[1804] an analysis means for analyzing the received measurement data and the user's facial photograph and voice data;

[1805] a notification means for providing health advice to the user based on the analysis results;

[1806] The system includes a messaging interface for transmitting the health advice in real time.

[1807] (Claim 2)

[1808] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing the dietary data of the user and calculating calories and nutritional balance.

[1809] (Claim 3)

[1810] 10. The system of claim 1, wherein the notification means includes means for sending preventative alerts in real time based on the user's health and emotional state.

[1811] "Application example 2 when combining emotion engines"

[1812] (Claim 1)

[1813] a device for measuring biometric information of a user;

[1814] a communication means for receiving measurement data from the device;

[1815] an analysis means for analyzing the received measurement data and environmental data;

[1816] a notification means for providing health advice and service suggestions to the user based on the analysis results;

[1817] A system including a messaging interface for transmitting said health advice and service offers.

[1818] (Claim 2)

[1819] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing a facial photograph of the user to collect emotional data and generate service proposals.

[1820] (Claim 3)

[1821] 2. The system of claim 1, wherein the notification means includes means for visually and audibly notifying the user of the service offer in real time. [Explanation of symbols]

[1822] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a device for measuring biometric information of a user; a communication means for receiving measurement data from the device; analysis means for analyzing the received measurement data; a notification means for providing health advice to the user based on the analysis results; The system includes a messaging interface for sending the health advice.

2. 2. The system according to claim 1, wherein the analyzing means includes means for analyzing the dietary data of the user and calculating calories and nutritional balance.

3. The system of claim 1 , wherein the notification means includes means for sending preventative alerts in real time based on the user's health status.

4. The system of claim 1 , wherein the analysis means includes means for analyzing the user's sleep data and generating advice for improving the quality of sleep.

5. 2. The system of claim 1, wherein the analyzing means includes means for analyzing the user's stress level and providing counseling for stress management.

Images