System

Abstract

A system is provided.SOLUTION: A system comprising: sensor means for measuring a body fat percentage and a skin temperature of a user; communication means for transmitting the measurements to a server; means at the server for transmitting the measurements to a generating AI for analysis; means for transmitting feedback and advice generated by the generating AI back to a device; and means at the device for visually displaying the feedback and advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's busy lifestyles, it is difficult for individual users to properly and continuously manage their health status in their daily lives. In particular, it is difficult to improve lifestyle habits without daily checking basic health indicators such as body fat percentage and skin temperature. In addition, due to a lack of health data, it is difficult for users to receive specific feedback and advice to change their behavior. [Means for solving the problem]

[0005] The present invention provides a system that includes a sensor means for measuring a user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means in the server for transmitting the measurement data to a generating AI for analysis, a means for transmitting the feedback and advice generated by the generating AI back to a terminal, and a means for visually displaying the feedback and advice on the terminal. This allows users to easily check their health status in front of a bathroom mirror and continuously manage their health in their daily lives. Through analysis and personalized advice by the generating AI, users can receive intuitive and useful feedback for making specific behavioral changes.

[0006] "User" refers to an individual who uses this system and is the subject of obtaining measurement data of body fat percentage and skin temperature.

[0007] "Body fat percentage" is an indicator that shows the percentage of body fat in a user's total body weight, and is important data for evaluating health status.

[0008] "Skin temperature" is a value that measures the temperature of the user's skin, and is data used to determine their health condition and physical condition.

[0009] The term "sensor means" refers to a device installed to measure body fat percentage and skin temperature, and has the function of acquiring this data from the user's body.

[0010] "Communication means" refers to a device or method for transmitting measured data to a server, enabling real-time data transfer.

[0011] "Server" refers to the central computer system that receives the measurement data and sends it to the generating AI for analysis.

[0012] "Generative AI" refers to artificial intelligence algorithms and software that analyzes the data it receives and generates feedback and advice based on the user's health status.

[0013] "Analysis means" refers to the process of data analysis by the generation AI installed on the server, which has the function of evaluating the user's health condition.

[0014] "Feedback" refers to information generated by the AI ​​for the user based on the analysis results, including evaluations and advice regarding body fat percentage and skin temperature.

[0015] "Advice" refers to information containing specific action suggestions to improve the user's health, and is provided as feedback by the generative AI.

[0016] The "terminal" refers to the IoT mirror device installed in the bathroom, which has the function of displaying measurement data and feedback and advice from the generating AI to the user.

[0017] "Visual display means" refers to methods and devices for displaying feedback and advice to the user at the terminal in an easy-to-view format. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means on the server for having the generated AI analyze the data, a means for transmitting feedback and advice generated by the generated AI back to the terminal, and a means for visually displaying the feedback and advice on the terminal.

[0040] System Components and Functions

[0041] Sensor Means

[0042] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[0043] communication means

[0044] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[0045] Server and Generating AI

[0046] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[0047] Send feedback and advice

[0048] The feedback and advice generated by the AI ​​is then sent back to the device via the server. For example, if it determines that the body fat percentage is on the rise, the AI ​​might generate the advice, "Your body fat percentage has been increasing recently. Consider increasing your exercise."

[0049] Terminals and Visual Displays

[0050] The device visually displays this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[0051] Specific examples

[0052] Below are some concrete examples of how this system works in a user's daily life:

[0053] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0054] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0055] 3. The measurement data is sent to the server via a communication means.

[0056] 4. The server stores this data and sends it to the generating AI.

[0057] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0058] 6. Feedback from the generated AI is sent to the device via the server.

[0059] 7. When the user stands in front of the mirror again, the mirror will display the AI's advice, along with "Body fat percentage 20%, skin temperature 36.5 degrees."

[0060] In this way, the present invention helps users manage their health on a daily basis, allowing users to easily check their own health status and receive advice tailored to their needs.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] A user stands in front of the IoT mirror in the bathroom in the morning.

[0064] Step 2:

[0065] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[0066] Step 3:

[0067] The device sends the measured data (body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[0068] Step 4:

[0069] The server stores the received measurement data in a database.

[0070] Step 5:

[0071] The server prepares the saved data to be sent to the generation AI.

[0072] Step 6:

[0073] The generating AI analyzes the received data and past data to assess the user's health status.

[0074] Step 7:

[0075] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[0076] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[0077] Step 8:

[0078] The generated AI sends the generated feedback and advice to the server.

[0079] Step 9:

[0080] The server sends the feedback and advice received from the generated AI to the terminal.

[0081] Step 10:

[0082] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[0083] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "Your body fat percentage has increased recently. Consider increasing your exercise."

[0084] Step 11:

[0085] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[0086] Example: A user decides to add a morning jog.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] In today's modern living environment, it is important to monitor individual health conditions in real time and receive appropriate feedback and advice, but the systems required to do so are complex and difficult to use.In addition, the lack of analytical tools to provide personalized health advice makes it difficult to provide accurate health management.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, a means for automatically transmitting multiple pieces of data measured by the sensor means to the server, and a means for evaluating the user's health condition with the generation AI and generating advice based on past and current data. This enables users to easily monitor their own health condition in real time in their daily lives and receive personalized feedback and advice.

[0092] The "sensor means" is a device that measures the user's body fat percentage and skin temperature.

[0093] The "communication means" is a network device for transmitting measurement data to a server.

[0094] The "server" is a computer system that receives measurement data, sends it to the generation AI, and sends the analysis results to the terminal.

[0095] "Generative AI" is artificial intelligence that analyzes the data it receives and generates feedback and advice about the user's health status.

[0096] A "terminal" is a device that visually displays feedback and advice to a user.

[0097] A "database" is an information system for storing received measurement data.

[0098] A "prompt sentence" is an instruction sentence that the generative AI uses to analyze the user's health condition.

[0099] "Display means" refers to a display device that allows a user to visually confirm information.

[0100] "Health status assessment" means that the generative AI analyzes new and old data to determine the user's physical condition.

[0101] "Feedback" is information that the generative AI provides to the user based on the analysis results.

[0102] "Advice" is specific instructions provided by the generative AI to improve the user's health.

[0103] The above are definitions of important terms contained in the claims.

[0104] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives, and is mainly composed of sensor means, communication means, a server, generation AI, and a terminal. Specific use cases and the hardware and software used are described below.

[0105] Sensor Means

[0106] The sensor means is used to measure the user's body fat percentage and skin temperature. This system uses a general-purpose body fat scale sensor and temperature sensor. For example, a commercially available general-purpose sensor is used as the body fat scale sensor, and a high-precision temperature sensor is used. The sensors are embedded in the IoT mirror and automatically start up and collect data when the user stands in front of the mirror.

[0107] communication means

[0108] The measured data is sent to the server in real time via a communication means. This communication uses common network technologies such as Wi-Fi. Specifically, by using an ESP8266 as a Wi-Fi module, data acquired from the sensor can be sent to the server quickly and reliably.

[0109] Server and Data Analysis

[0110] The server stores the received data and sends it to the generation AI for analysis. The server uses a database management system (e.g., MySQL) to efficiently store the measurement data. The server manages the data using SQL statements such as:

[0111] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[0112] Analysis by generative AI

[0113] The server sends the data to a generator AI, which evaluates the user's health status. The generator AI uses, for example, OpenAI's GPT model. The generator AI analyzes the data using prompts like the following:

[0114] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[0115] The generative AI generates appropriate feedback and advice based on the input data, such as "Your body fat percentage has increased recently. Consider increasing your exercise."

[0116] Send feedback and advice

[0117] The feedback and advice generated by the AI ​​is then sent back to the device via the server, again via the Wi-Fi module.

[0118] Display on device

[0119] The device visually displays this feedback and advice to the user. For example, a Raspberry Pi touchscreen monitor can be used as the display device. When the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[0120] Specifically, it will appear as follows:

[0121] Body fat percentage: 20%

[0122] Skin temperature: 36.5 degrees

[0123] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[0124] In this way, the IoT mirror system provides support for users in managing their health on a daily basis, allowing them to easily monitor their own health status and receive accurate feedback and advice.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1:

[0127] A user stands in front of the IoT mirror in the bathroom in the morning. The sensor means is automatically activated to measure the user's body fat percentage and skin temperature. The input is "user's body fat percentage and skin temperature" and the output is "measured data (e.g. body fat percentage 20%, skin temperature 36.5 degrees)." The sensor uses a body fat scale sensor and a temperature sensor to acquire data.

[0128] Step 2:

[0129] The device transmits the measured data to the server in real time via a communication means. The input is the "measured data (body fat percentage 20%, skin temperature 36.5°C)" and the output is the "data transmitted to the server." A Wi-Fi module (e.g., ESP8266) is used for this communication.

[0130] Step 3:

[0131] The data received by the server is saved in a database. The input is "data sent to the server" (e.g., user ID, measurement date and time, body fat percentage, skin temperature), and the output is "data saved in the database." A MySQL database is used for saving, and the following SQL statement is issued:

[0132] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[0133] Step 4:

[0134] The server formats the data to send to the generation AI. The input is "data stored in the database" (e.g., body fat percentage, skin temperature, past measurement data), and the output is "formatted data" (e.g., data sent with the prompt). The prompt is formatted as follows:

[0135] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[0136] Step 5:

[0137] The generative AI analyzes the data and generates feedback and advice. The input is "formatted data" (e.g., the latest body fat percentage and skin temperature, past data), and the output is "generated feedback and advice." For example, feedback in the form of "Your body fat percentage has increased recently. Please consider increasing your exercise" is generated.

[0138] Step 6:

[0139] The server sends the generated feedback and advice to the terminal. The input is "generated feedback and advice" and the output is "feedback and advice sent to the terminal." This communication is also done via the Wi-Fi module.

[0140] Step 7:

[0141] The terminal displays visual feedback and advice to the user. The input is "feedback and advice sent to the terminal" and the output is "visually displayed feedback and advice". Specifically, it is displayed on the Raspberry Pi's touchscreen monitor as follows:

[0142] Body fat percentage: 20%

[0143] Skin temperature: 36.5 degrees

[0144] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[0145] This allows users to easily check their health status and receive appropriate advice.

[0146] (Application example 1)

[0147] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0148] Conventional health management systems require users to voluntarily enter data, making them difficult to use on a daily basis. Furthermore, there was a lack of a way for fitness facilities to efficiently manage members' health data and provide individualized feedback in real time.

[0149] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0150] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and a means for measuring the member's health data at the fitness facility and providing feedback analyzed by the generation AI in real time. This allows users to easily obtain health data on a daily basis and receive individual health advice at the fitness facility.

[0151] "User" refers to a member of a fitness facility or an individual person who generally uses the system.

[0152] "Body fat percentage" is health data that indicates the percentage of fat in a user's total body weight.

[0153] "Skin temperature" is a measurement of the temperature of the user's skin surface, and is data that indicates part of the user's health condition.

[0154] "Sensor means" refers to an apparatus or device for measuring body fat percentage and skin temperature in real time.

[0155] "Communication means" refers to the technology, including network connections and protocols, for transmitting measurement data to a server.

[0156] "Server" refers to the central management system for receiving measurement data and transmitting it to the storage and generating AI.

[0157] "Generative AI" is an artificial intelligence tool that analyzes measurement data and generates feedback and health advice.

[0158] "Feedback" refers to the evaluations and comments that the generative AI provides to the user based on the analysis results.

[0159] "Advice" refers to specific recommendations or suggestions for action that the generative AI provides to users based on the analysis results.

[0160] "Terminal" refers to the device (smartphone, tablet, IoT mirror, etc.) through which the user visually views feedback and advice.

[0161] "Fitness facilities" refer to facilities where individuals can exercise and manage their health, such as gyms and fitness clubs.

[0162] The present invention relates to a "Gym Health Advisor" system for efficiently managing the health of members at fitness facilities. Specific embodiments for realizing this system will be described below.

[0163] System Overview

[0164] The system uses IoT mirrors installed in fitness facilities to measure members' health data and provide real-time feedback analyzed by generative AI.

[0165] Hardware and software used

[0166] IoT Mirror: Equipped with sensors to measure body fat percentage and skin temperature.

[0167] Communication method: Measurement data is sent to the server via Wi-Fi, Bluetooth, etc.

[0168] Server: Stores measurement data in a database and sends the data to the generation AI.

[0169] Generative AI: Analyzes measurement data and generates feedback and advice. For generative AI, models such as GPT-4 are used.

[0170] Device: A device, such as a smartphone or tablet, that allows members to visually view feedback and advice.

[0171] Program processing overview

[0172] The server receives the user's body fat percentage and skin temperature measurement data sent from the IoT mirror and stores it in a database.The server then sends this measurement data to the generation AI for analysis.

[0173] The Generative AI uses the user's past health data and newly received data to assess their health status and generate personalized feedback and advice, which are then sent to the device via a server and displayed visually on the device.

[0174] Specific examples

[0175] When a user stands in front of the IoT mirror at a fitness facility, the mirror measures their body fat percentage and skin temperature and sends the data to a server in real time. The server receives this data and sends it to the generating AI. The generating AI analyzes the data and generates advice such as, "Your body fat percentage is high. Please review your diet." This feedback and advice is then sent back to the user's smartphone via the server.

[0176] Example prompts for generative AI models

[0177] Analyze the user's body fat percentage (20%) and skin temperature (36.5°C) and generate appropriate feedback and health management advice.

[0178] In this way, the present invention is designed to efficiently manage the health of members at fitness facilities, allowing users to easily check their daily health status and receive appropriate advice in real time.

[0179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0180] Step 1:

[0181] When a user stands in front of an IoT mirror at a fitness facility, the sensor built into the IoT mirror measures body fat percentage and skin temperature. The input is the user's physical data (body fat percentage and skin temperature), and the output is the measured health data. Specifically, the sensor measures the user's body fat percentage as, for example, 20% and records the skin temperature as 36.5 degrees.

[0182] Step 2:

[0183] The measurement data is sent to a server in real time via a communication method. The input is the body fat percentage and skin temperature data obtained from the sensor, and the output is the data sent to the server. Specifically, the IoT mirror uses Wi-Fi and Bluetooth to send data such as a body fat percentage of 20% and a skin temperature of 36.5 degrees to the server.

[0184] Step 3:

[0185] The server stores the received measurement data in a database. The input is the transmitted data (body fat percentage and skin temperature), and the output is the record stored in the database. Specifically, the server uses database software (e.g., MySQL) to store the received data in a specific table.

[0186] Step 4:

[0187] The server sends the saved measurement data to the generation AI for analysis. The input is the saved measurement data and past health data, and the output is the analysis results. Specifically, the server sends the data to a generation AI model (e.g., GPT-4) and performs the analysis using the prompt, "Analyze the user's body fat percentage of 20% and skin temperature of 36.5 degrees, and generate appropriate feedback and health management advice."

[0188] Step 5:

[0189] The server then sends the feedback and advice generated by the generating AI back to the device. The input is the feedback and advice as the analysis results, and the output is the data sent to the user's device. Specifically, the server organizes the information received from the generating AI and sends advice such as, "Your body fat percentage is high. Please review your diet" to the user's smartphone or tablet.

[0190] Step 6:

[0191] The feedback and advice received by the user's device is visually displayed. The input is the feedback and advice sent from the server, and the output is the visual display content. Specifically, the smartphone application displays on the screen "Body fat percentage 20%, skin temperature 36.5°C" along with the advice "Your body fat percentage is high. Please review your diet."

[0192] This process allows users to understand their own health status in real time and receive appropriate advice.

[0193] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0194] This invention combines an IoT mirror system designed to allow users to easily manage their health in their daily lives with an emotion engine that recognizes the user's emotions. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means in the server for transmitting this data to a generation AI for analysis, a means for transmitting feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and an emotion engine that recognizes the user's emotions.

[0195] System Components and Functions

[0196] Sensor Means

[0197] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[0198] communication means

[0199] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[0200] Server and Generating AI

[0201] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[0202] Emotion Engine

[0203] The emotion engine recognizes emotions by analyzing the user's facial expressions and tone of voice. For example, when a user stands in front of a mirror checking their body fat percentage or skin temperature, it can determine whether the user is happy or stressed.

[0204] Generate and send feedback and advice

[0205] The generative AI generates feedback and advice appropriate for the user based on the analysis results. Meanwhile, the emotion engine appropriately modifies the feedback and advice provided by the generative AI based on the user's emotions as recognized. For example, if the user is feeling stressed, the advice may be adjusted to include relaxation techniques.

[0206] Terminals and Visual Displays

[0207] The device will visually display this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with tailored advice provided by the generative AI and emotion engine.

[0208] Specific examples

[0209] Below are some concrete examples of how this system works in a user's daily life:

[0210] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0211] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0212] 3. The measurement data is sent to the server via a communication means.

[0213] 4. The server stores this data and sends it to the generating AI.

[0214] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0215] 6. The emotion engine analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[0216] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion engine, generating adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[0217] 8. Feedback and advice is sent to the device through the server.

[0218] 9. When the user stands in front of the mirror again, the mirror will display "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[0219] In this way, the present invention helps users manage their health on a daily basis, and by combining it with the emotion engine's recognition of the user's emotions, it provides more personalized feedback and supports users in improving their lifestyle habits.

[0220] The processing flow will be explained below.

[0221] Step 1:

[0222] A user stands in front of the IoT mirror in the bathroom in the morning.

[0223] Step 2:

[0224] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[0225] Step 3:

[0226] The device sends the measured data (e.g., body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[0227] Step 4:

[0228] The server stores the received measurement data in a database and also sends the measurement data to the generation AI.

[0229] Step 5:

[0230] The generating AI analyzes the received data and the user's past health data to assess the user's health condition.

[0231] Step 6:

[0232] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[0233] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[0234] Step 7:

[0235] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

[0236] Step 8:

[0237] The server sends the analysis results of the emotion engine to the generation AI.

[0238] Step 9:

[0239] The generative AI adjusts feedback and advice based on the analysis results of the emotion engine.

[0240] For example: If the user is feeling stressed, "I recommend increasing your exercise and taking time to relax."

[0241] Step 10:

[0242] The generated AI sends the generated feedback and advice to the server.

[0243] Step 11:

[0244] The server sends the feedback and advice received from the generated AI to the terminal.

[0245] Step 12:

[0246] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[0247] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax."

[0248] Step 13:

[0249] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[0250] Example: A user decides to incorporate meditation time into their morning jog.

[0251] Example 2

[0252] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0253] Conventional health management systems have difficulty providing personalized feedback and advice because they cannot consider how data such as a user's body fat percentage and skin temperature correlates with the user's emotions and stress levels. Furthermore, even if a user is feeling stressed or frustrated, they cannot generate advice that takes this into account, preventing effective health management.

[0254] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI and analyzing it, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion recognition means for analyzing the user's facial expression and tone of voice to recognize their emotions, and a means for appropriately correcting the feedback and advice provided by the generation AI based on the results of the emotion recognition means. This enables personalized health management that takes the user's emotions into consideration.

[0255] "Sensor means" refers to a device that measures the user's body fat percentage and skin temperature.

[0256] "Communication means" refers to an interface for transmitting measurement data to a server.

[0257] "Generative AI" refers to artificial intelligence that analyzes measurement data on the server and generates feedback and advice.

[0258] "Emotion recognition means" refers to technology that recognizes emotions by analyzing a user's facial expressions and tone of voice.

[0259] "Means for sending feedback and advice back to the terminal" refers to a communication means for sending the feedback and advice generated by the generating AI to the terminal.

[0260] "Means for visually displaying feedback and advice at a terminal" refers to a display function on a terminal for visually displaying feedback and advice to a user.

[0261] "Means for appropriately modifying the feedback and advice provided by the generating AI based on the results of the emotion recognition means" refers to an algorithm that enables the generating AI to modify its feedback and advice in consideration of the user's emotional data obtained by the emotion recognition means.

[0262] The present invention relates to a system that allows users to easily manage their health on a daily basis, and is characterized in that it provides personalized feedback and advice taking into account the user's emotions. Specific implementation methods for this system are described below.

[0263] Sensor Means

[0264] When a user stands in front of the mirror, the sensors built into the IoT mirror automatically measure the user's body fat percentage and skin temperature. The sensors include a bioimpedance sensor to measure body fat percentage and a thermistor to measure skin temperature.

[0265] communication means

[0266] The measurement data is sent to the server via a communication means such as a Wi-Fi module or Bluetooth module, eliminating the need for users to manually input the measurement data.

[0267] Server and Generating AI

[0268] The server stores the received data in a database. The stored data is then sent to the Generator AI for analysis. This Generator AI uses advanced artificial intelligence, such as GPT-4. This Generator AI analyzes past health data and newly received data to assess the user's health status and generate specific feedback and advice.

[0269] emotion recognition means

[0270] The mirror uses deep learning technology to recognize emotions by analyzing the user's facial expressions and tone of voice. A camera and microphone attached to the mirror capture the user's facial expressions and voice for analysis.

[0271] Generate feedback and advice

[0272] The AI ​​then generates feedback and advice for the user based on the analysis results. Furthermore, the emotion recognition system takes the user's emotions into account and modifies the AI's feedback and advice accordingly. For example, if the user is feeling stressed, the AI ​​may add relaxation techniques.

[0273] Viewing feedback and advice

[0274] The generated feedback and advice is sent to the device via the server, and when the user stands in front of the mirror again, the adjusted feedback and advice is displayed on the mirror along with the current values ​​of body fat percentage and skin temperature.

[0275] Specific examples

[0276] Below are some concrete examples of how this system works in a user's daily life:

[0277] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0278] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0279] 3. The measurement data is sent to the server via the communication means.

[0280] 4. The server stores this data and sends it to the generating AI.

[0281] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0282] 6. The emotion recognition means analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[0283] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion recognition means, and generates adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[0284] 8. Feedback and advice is sent to the device through the server.

[0285] 9. When the user stands in front of the mirror again, the mirror displays "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[0286] Prompt Sentence Examples

[0287] Below are some example prompts to be input to the generative AI model:

[0288] The user's body fat percentage has increased recently, so you would like to advise the user to increase their exercise. Additionally, the user is feeling stressed, so you would like to encourage them to take time to relax.

[0289] As a result, the present invention can provide personalized feedback and advice that takes into account the user's emotions, allowing the user to manage their health more effectively.

[0290] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0291] Step 1:

[0292] A user stands in front of the IoT mirror in the bathroom in the morning. Specifically, the user simply stands in front of the mirror to wash their face, and the system automatically activates. The input is the user's presence, and the output is the trigger that activates the sensor.

[0293] Step 2:

[0294] The sensor means measures body fat percentage and skin temperature. Specifically, a bioimpedance sensor mounted on the mirror measures body fat percentage, and a thermistor measures skin temperature. The input is the user's body fat percentage and skin temperature, and the output is these measurement data.

[0295] Step 3:

[0296] The device sends the measurement data to the server. Specifically, the communication means (Wi-Fi module) sends the measurement data (e.g., body fat percentage 20%, skin temperature 36.5°C) to the server in real time. The input is the measurement data, and the output is the data sent to the server.

[0297] Step 4:

[0298] The server stores the received data in a database. Specifically, the server executes a procedure to store the data in a database such as MySQL. The input is the transmitted measurement data, and the output is the stored data.

[0299] Step 5:

[0300] The server sends data to the generation AI. Specifically, the server sends data to the generation AI (e.g., GPT-4) via an API. The input is the stored measurement data, and the output is the data sent to the generation AI.

[0301] Step 6:

[0302] The generating AI analyzes the received data and generates feedback and advice. Specifically, the generating AI analyzes the user's past health data and newly received data. The input is measurement data and past health data, and the output is feedback and advice such as, "Your body fat percentage has been increasing recently. Please consider increasing your exercise."

[0303] Step 7:

[0304] The emotion recognition means recognizes the user's emotions. Specifically, a camera and microphone attached to the mirror capture the user's facial expressions and tone of voice, which are then analyzed by a deep learning model. The input is the user's facial expression and voice data, and the output is the user's emotional data.

[0305] Step 8:

[0306] The generative AI modifies the feedback and advice based on the results of the emotion recognition means. Specifically, the generative AI takes the emotion data into account and modifies the advice to "Increase your exercise and take time to relax." The inputs are the feedback, advice, and emotion data, and the output is the modified feedback and advice.

[0307] Step 9:

[0308] The server sends the feedback and advice to the terminal. As a specific operation, the feedback and advice are sent to the terminal again through the communication means. The input is the modified feedback and advice, and the output is the data sent to the terminal.

[0309] Step 10:

[0310] The device visually displays feedback and advice. Specifically, when the user stands in front of the mirror again, the mirror displays the message "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax." The input is the data sent to the device, and the output is the feedback and advice displayed on the mirror.

[0311] (Application example 2)

[0312] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] In autonomous vehicles, safety and comfort are not sufficiently ensured because the driver's health and emotional state are not monitored in real time. In addition, it is difficult to provide appropriate advice based on the driver's health and emotional state, which may result in the driver's fatigue and stress being overlooked.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0315] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion analysis means for analyzing the driver's facial expression and tone of voice to recognize the emotional state, and a means for appropriately correcting the feedback and advice by the generation AI based on the emotion analysis results. This makes it possible to manage the driver's health and emotional state in real time in an autonomous vehicle and provide appropriate feedback and advice.

[0316] A "user" is a person who utilizes the system to receive feedback on their health and emotional state.

[0317] "Body fat percentage" is a numerical value that indicates the percentage of fat present in the user's body.

[0318] "Skin temperature" is a numerical value measuring the surface temperature of the user's skin.

[0319] "Sensor means" refers to devices and techniques for measuring body fat percentage and skin temperature.

[0320] "Communication means" refers to the communication protocol and device for transmitting the measured data to the server.

[0321] "Generative AI" refers to artificial intelligence technology that analyzes past health data and newly received data to generate appropriate feedback and advice.

[0322] "Emotion analysis means" refers to technology and devices that analyze facial expressions and tone of voice to recognize the user's emotional state.

[0323] "Terminal" refers to a device for visually displaying feedback and advice to a user.

[0324] "Feedback and advice" refers to specific instructions and advice provided to the user based on the results of the generative AI's analysis.

[0325] "Emotion analysis result" refers to the analysis result of the user's emotional state obtained by the emotion analysis means.

[0326] "Cloud server" refers to the computer system that stores the measured data and on which the generative AI and emotion analysis means run.

[0327] The present invention relates to a system for providing real-time monitoring of the driver's well-being and emotional state in an autonomous vehicle. The system includes the following main components:

[0328] Sensor Means

[0329] Health status data collection

[0330] The sensor means is installed in the autonomous vehicle and includes a device for measuring body fat percentage and skin temperature. The body fat percentage sensor and skin temperature sensor measure the driver's health data in real time and acquire the values.

[0331] communication means

[0332] Data transmission

[0333] The acquired body fat percentage and skin temperature data is sent in real time to a cloud server via a communication module in the vehicle (e.g., Bluetooth, Wi-Fi), eliminating the need for the driver to check the measurement data directly.

[0334] Server and Generating AI

[0335] Data analysis

[0336] The cloud server stores the received body fat percentage and skin temperature data and sends it to the generative AI model (e.g., GPT-4), which analyzes this data, evaluates the driver's health status, and generates specific feedback and advice.

[0337] Emotion analysis means

[0338] emotion recognition

[0339] Emotion analysis refers to technology for analyzing the driver's facial expressions and tone of voice. Driver information is collected using the smartphone's camera and microphone, and analyzed by an emotion analysis engine (e.g., Emotion AI SDK). This allows the driver's emotional state (e.g., stress, satisfaction) to be recognized.

[0340] Generate and send feedback and advice

[0341] Feedback Adjustment

[0342] The generative AI will then adjust the feedback and advice appropriately based on the emotional data obtained from the emotion analysis means. For example, if the emotion analysis determines that the driver is feeling stressed, the AI ​​will adjust the advice by adding relaxation techniques.

[0343] Terminals and Visual Displays

[0344] Information presentation

[0345] Final feedback and advice is provided to the driver via a smartphone application, which displays the driver's current body fat percentage and skin temperature, as well as tailored feedback and advice provided by the AI.

[0346] Specific examples

[0347] Below is a concrete example of how this system works within a self-driving vehicle.

[0348] 1. The driver enters the vehicle and the sensors measure his body fat percentage at 19% and his skin temperature at 36.0°C.

[0349] 2. Measurement data is sent in real time to a cloud server via a smartphone.

[0350] 3. The AI ​​analyzes the data stored on the cloud server and generates feedback such as, "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break."

[0351] 4. The emotion analysis engine recognizes fatigue from the driver's facial expression, and the generative AI adjusts the feedback to, "It would be a good idea to exercise and drink a warm drink."

[0352] 5. Calibrated feedback will be displayed on your smartphone.

[0353] Prompt Sentence Examples

[0354] User Health Data:

[0355] Body fat percentage: 19%

[0356] Skin temperature: 36.0 degrees

[0357] User's emotional state: Fatigue

[0358] Generate feedback and advice:

[0359] Your body fat percentage is normal, but your skin temperature may be low. We recommend taking a break.

[0360] Exercise and drink warm drinks.

[0361] As described above, this invention enables drivers to manage their health and monitor their emotional state while in an autonomous vehicle, and provides appropriate feedback and advice, which is expected to significantly improve driving safety and comfort.

[0362] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0363] Step 1:

[0364] Health data collection

[0365] When a user sits in the autonomous vehicle, the sensor measures their body fat percentage and skin temperature, and the measured data is sent to a smartphone.

[0366] Input: User's body fat percentage and skin temperature

[0367] Output: Measurement data (e.g., body fat percentage 19%, skin temperature 36.0°C)

[0368] Step 2:

[0369] Data transmission

[0370] The device (smartphone) transmits the collected body fat percentage and skin temperature data to a cloud server in real time, using Bluetooth and Wi-Fi as the communication method.

[0371] Input: Measurement data (body fat percentage 19%, skin temperature 36.0°C)

[0372] Output: Data sent to the cloud server

[0373] Step 3:

[0374] Data storage and analysis preparation

[0375] The server stores the received data in a database and prepares to send the data to the generation AI.

[0376] Input: Data sent to the cloud server

[0377] Output: Data stored in the database, input data to the generative AI

[0378] Step 4:

[0379] Analysis by generative AI

[0380] The AI ​​analyzes the body fat percentage and skin temperature data sent from the server, as well as past health data, to assess the user's current health condition. Based on the analysis results, it generates specific feedback and advice.

[0381] Input: Data input to the AI ​​(body fat percentage 19%, skin temperature 36.0°C)

[0382] Output: Generated feedback and advice (e.g., "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break.")

[0383] Step 5:

[0384] Emotion recognition using emotion analysis methods

[0385] The smartphone's camera and microphone are used to analyze the user's facial expressions and tone of voice, and an emotion analysis engine (e.g., Emotion AI SDK) recognizes the user's emotional state.

[0386] Input: facial expression data and tone of voice

[0387] Output: Sentiment analysis result (e.g., fatigue)

[0388] Step 6:

[0389] Feedback Adjustment

[0390] The generative AI then adjusts the feedback and advice appropriately based on the emotional data obtained from the emotion analysis tool. For example, if the user is feeling stressed, it will add relaxation techniques.

[0391] Input: Generated feedback and advice, sentiment analysis results

[0392] Output: Tailored feedback and advice (e.g., "You should exercise and drink warm drinks.")

[0393] Step 7:

[0394] Sending feedback and suggestions

[0395] The server sends tailored feedback and advice to the terminal.

[0396] Input: Calibrated feedback and advice

[0397] Output: Feedback and advice sent to the device

[0398] Step 8:

[0399] Information presentation

[0400] The device (smartphone) visually displays tailored feedback and advice to the driver.

[0401] Input: Feedback and advice sent to the device

[0402] Output: Feedback and advice displayed on the smartphone (e.g., "It would be good for you to exercise and drink warm drinks.")

[0403] The above are the specific processing steps of the system that realizes the application example.

[0404] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0405] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0406] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0407] [Second embodiment]

[0408] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0409] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0410] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0411] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0412] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0413] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0414] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0415] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0416] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0417] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0418] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0419] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0420] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means on the server for having the generated AI analyze the data, a means for transmitting feedback and advice generated by the generated AI back to the terminal, and a means for visually displaying the feedback and advice on the terminal.

[0421] System Components and Functions

[0422] Sensor Means

[0423] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[0424] communication means

[0425] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[0426] Server and Generating AI

[0427] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[0428] Send feedback and advice

[0429] The feedback and advice generated by the AI ​​is then sent back to the device via the server. For example, if it determines that the body fat percentage is on the rise, the AI ​​might generate the advice, "Your body fat percentage has been increasing recently. Consider increasing your exercise."

[0430] Terminals and Visual Displays

[0431] The device visually displays this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[0432] Specific examples

[0433] Below are some concrete examples of how this system works in a user's daily life:

[0434] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0435] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0436] 3. The measurement data is sent to the server via a communication means.

[0437] 4. The server stores this data and sends it to the generating AI.

[0438] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0439] 6. Feedback from the generated AI is sent to the device via the server.

[0440] 7. When the user stands in front of the mirror again, the mirror will display the AI's advice, along with "Body fat percentage 20%, skin temperature 36.5 degrees."

[0441] In this way, the present invention helps users manage their health on a daily basis, allowing users to easily check their own health status and receive advice tailored to their needs.

[0442] The processing flow will be explained below.

[0443] Step 1:

[0444] A user stands in front of the IoT mirror in the bathroom in the morning.

[0445] Step 2:

[0446] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[0447] Step 3:

[0448] The device sends the measured data (body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[0449] Step 4:

[0450] The server stores the received measurement data in a database.

[0451] Step 5:

[0452] The server prepares the saved data to be sent to the generation AI.

[0453] Step 6:

[0454] The generating AI analyzes the received data and past data to assess the user's health status.

[0455] Step 7:

[0456] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[0457] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[0458] Step 8:

[0459] The generated AI sends the generated feedback and advice to the server.

[0460] Step 9:

[0461] The server sends the feedback and advice received from the generated AI to the terminal.

[0462] Step 10:

[0463] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[0464] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "Your body fat percentage has increased recently. Consider increasing your exercise."

[0465] Step 11:

[0466] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[0467] Example: A user decides to add a morning jog.

[0468] Example 1

[0469] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0470] In today's modern living environment, it is important to monitor individual health conditions in real time and receive appropriate feedback and advice, but the systems required to do so are complex and difficult to use.In addition, the lack of analytical tools to provide personalized health advice makes it difficult to provide accurate health management.

[0471] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0472] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, a means for automatically transmitting multiple pieces of data measured by the sensor means to the server, and a means for evaluating the user's health condition with the generation AI and generating advice based on past and current data. This enables users to easily monitor their own health condition in real time in their daily lives and receive personalized feedback and advice.

[0473] The "sensor means" is a device that measures the user's body fat percentage and skin temperature.

[0474] The "communication means" is a network device for transmitting measurement data to a server.

[0475] The "server" is a computer system that receives measurement data, sends it to the generation AI, and sends the analysis results to the terminal.

[0476] "Generative AI" is artificial intelligence that analyzes the data it receives and generates feedback and advice about the user's health status.

[0477] A "terminal" is a device that visually displays feedback and advice to a user.

[0478] A "database" is an information system for storing received measurement data.

[0479] A "prompt sentence" is an instruction sentence that the generative AI uses to analyze the user's health condition.

[0480] "Display means" refers to a display device that allows a user to visually confirm information.

[0481] "Health status assessment" means that the generative AI analyzes new and old data to determine the user's physical condition.

[0482] "Feedback" is information that the generative AI provides to the user based on the analysis results.

[0483] "Advice" is specific instructions provided by the generative AI to improve the user's health.

[0484] The above are definitions of important terms contained in the claims.

[0485] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives, and is mainly composed of sensor means, communication means, a server, generation AI, and a terminal. Specific use cases and the hardware and software used are described below.

[0486] Sensor Means

[0487] The sensor means is used to measure the user's body fat percentage and skin temperature. This system uses a general-purpose body fat scale sensor and temperature sensor. For example, a commercially available general-purpose sensor is used as the body fat scale sensor, and a high-precision temperature sensor is used. The sensors are embedded in the IoT mirror and automatically start up and collect data when the user stands in front of the mirror.

[0488] communication means

[0489] The measured data is sent to the server in real time via a communication means. This communication uses common network technologies such as Wi-Fi. Specifically, by using an ESP8266 as a Wi-Fi module, data acquired from the sensor can be sent to the server quickly and reliably.

[0490] Server and Data Analysis

[0491] The server stores the received data and sends it to the generation AI for analysis. The server uses a database management system (e.g., MySQL) to efficiently store the measurement data. The server manages the data using SQL statements such as:

[0492] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[0493] Analysis by generative AI

[0494] The server sends the data to a generator AI, which evaluates the user's health status. The generator AI uses, for example, OpenAI's GPT model. The generator AI analyzes the data using prompts like the following:

[0495] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[0496] The generative AI generates appropriate feedback and advice based on the input data, such as "Your body fat percentage has increased recently. Consider increasing your exercise."

[0497] Send feedback and advice

[0498] The feedback and advice generated by the AI ​​is then sent back to the device via the server, again via the Wi-Fi module.

[0499] Display on device

[0500] The device visually displays this feedback and advice to the user. For example, a Raspberry Pi touchscreen monitor can be used as the display device. When the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[0501] Specifically, it will appear as follows:

[0502] Body fat percentage: 20%

[0503] Skin temperature: 36.5 degrees

[0504] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[0505] In this way, the IoT mirror system provides support for users in managing their health on a daily basis, allowing them to easily monitor their own health status and receive accurate feedback and advice.

[0506] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0507] Step 1:

[0508] A user stands in front of the IoT mirror in the bathroom in the morning. The sensor means is automatically activated to measure the user's body fat percentage and skin temperature. The input is "user's body fat percentage and skin temperature" and the output is "measured data (e.g. body fat percentage 20%, skin temperature 36.5 degrees)." The sensor uses a body fat scale sensor and a temperature sensor to acquire data.

[0509] Step 2:

[0510] The device transmits the measured data to the server in real time via a communication means. The input is the "measured data (body fat percentage 20%, skin temperature 36.5°C)" and the output is the "data transmitted to the server." A Wi-Fi module (e.g., ESP8266) is used for this communication.

[0511] Step 3:

[0512] The data received by the server is saved in a database. The input is "data sent to the server" (e.g., user ID, measurement date and time, body fat percentage, skin temperature), and the output is "data saved in the database." A MySQL database is used for saving, and the following SQL statement is issued:

[0513] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[0514] Step 4:

[0515] The server formats the data to send to the generation AI. The input is "data stored in the database" (e.g., body fat percentage, skin temperature, past measurement data), and the output is "formatted data" (e.g., data sent with the prompt). The prompt is formatted as follows:

[0516] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[0517] Step 5:

[0518] The generative AI analyzes the data and generates feedback and advice. The input is "formatted data" (e.g., the latest body fat percentage and skin temperature, past data), and the output is "generated feedback and advice." For example, feedback in the form of "Your body fat percentage has increased recently. Please consider increasing your exercise" is generated.

[0519] Step 6:

[0520] The server sends the generated feedback and advice to the terminal. The input is "generated feedback and advice" and the output is "feedback and advice sent to the terminal." This communication is also done via the Wi-Fi module.

[0521] Step 7:

[0522] The terminal displays visual feedback and advice to the user. The input is "feedback and advice sent to the terminal" and the output is "visually displayed feedback and advice". Specifically, it is displayed on the Raspberry Pi's touchscreen monitor as follows:

[0523] Body fat percentage: 20%

[0524] Skin temperature: 36.5 degrees

[0525] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[0526] This allows users to easily check their health status and receive appropriate advice.

[0527] (Application example 1)

[0528] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0529] Conventional health management systems require users to voluntarily enter data, making them difficult to use on a daily basis. Furthermore, there was a lack of a way for fitness facilities to efficiently manage members' health data and provide individualized feedback in real time.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0531] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and a means for measuring the member's health data at the fitness facility and providing feedback analyzed by the generation AI in real time. This allows users to easily obtain health data on a daily basis and receive individual health advice at the fitness facility.

[0532] "User" refers to a member of a fitness facility or an individual person who generally uses the system.

[0533] "Body fat percentage" is health data that indicates the percentage of fat in a user's total body weight.

[0534] "Skin temperature" is a measurement of the temperature of the user's skin surface, and is data that indicates part of the user's health condition.

[0535] "Sensor means" refers to an apparatus or device for measuring body fat percentage and skin temperature in real time.

[0536] "Communication means" refers to the technology, including network connections and protocols, for transmitting measurement data to a server.

[0537] "Server" refers to the central management system for receiving measurement data and transmitting it to the storage and generating AI.

[0538] "Generative AI" is an artificial intelligence tool that analyzes measurement data and generates feedback and health advice.

[0539] "Feedback" refers to the evaluations and comments that the generative AI provides to the user based on the analysis results.

[0540] "Advice" refers to specific recommendations or suggestions for action that the generative AI provides to users based on the analysis results.

[0541] "Terminal" refers to the device (smartphone, tablet, IoT mirror, etc.) through which the user visually views feedback and advice.

[0542] "Fitness facilities" refer to facilities where individuals can exercise and manage their health, such as gyms and fitness clubs.

[0543] The present invention relates to a "Gym Health Advisor" system for efficiently managing the health of members at fitness facilities. Specific embodiments for realizing this system will be described below.

[0544] System Overview

[0545] The system uses IoT mirrors installed in fitness facilities to measure members' health data and provide real-time feedback analyzed by generative AI.

[0546] Hardware and software used

[0547] IoT Mirror: Equipped with sensors to measure body fat percentage and skin temperature.

[0548] Communication method: Measurement data is sent to the server via Wi-Fi, Bluetooth, etc.

[0549] Server: Stores measurement data in a database and sends the data to the generation AI.

[0550] Generative AI: Analyzes measurement data and generates feedback and advice. For generative AI, models such as GPT-4 are used.

[0551] Device: A device, such as a smartphone or tablet, that allows members to visually view feedback and advice.

[0552] Program processing overview

[0553] The server receives the user's body fat percentage and skin temperature measurement data sent from the IoT mirror and stores it in a database.The server then sends this measurement data to the generation AI for analysis.

[0554] The Generative AI uses the user's past health data and newly received data to assess their health status and generate personalized feedback and advice, which are then sent to the device via a server and displayed visually on the device.

[0555] Specific examples

[0556] When a user stands in front of the IoT mirror at a fitness facility, the mirror measures their body fat percentage and skin temperature and sends the data to a server in real time. The server receives this data and sends it to the generating AI. The generating AI analyzes the data and generates advice such as, "Your body fat percentage is high. Please review your diet." This feedback and advice is then sent back to the user's smartphone via the server.

[0557] Example prompts for generative AI models

[0558] Analyze the user's body fat percentage (20%) and skin temperature (36.5°C) and generate appropriate feedback and health management advice.

[0559] In this way, the present invention is designed to efficiently manage the health of members at fitness facilities, allowing users to easily check their daily health status and receive appropriate advice in real time.

[0560] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0561] Step 1:

[0562] When a user stands in front of an IoT mirror at a fitness facility, the sensor built into the IoT mirror measures body fat percentage and skin temperature. The input is the user's physical data (body fat percentage and skin temperature), and the output is the measured health data. Specifically, the sensor measures the user's body fat percentage as, for example, 20% and records the skin temperature as 36.5 degrees.

[0563] Step 2:

[0564] The measurement data is sent to a server in real time via a communication method. The input is the body fat percentage and skin temperature data obtained from the sensor, and the output is the data sent to the server. Specifically, the IoT mirror uses Wi-Fi and Bluetooth to send data such as a body fat percentage of 20% and a skin temperature of 36.5 degrees to the server.

[0565] Step 3:

[0566] The server stores the received measurement data in a database. The input is the transmitted data (body fat percentage and skin temperature), and the output is the record stored in the database. Specifically, the server uses database software (e.g., MySQL) to store the received data in a specific table.

[0567] Step 4:

[0568] The server sends the saved measurement data to the generation AI for analysis. The input is the saved measurement data and past health data, and the output is the analysis results. Specifically, the server sends the data to a generation AI model (e.g., GPT-4) and performs the analysis using the prompt, "Analyze the user's body fat percentage of 20% and skin temperature of 36.5 degrees, and generate appropriate feedback and health management advice."

[0569] Step 5:

[0570] The server then sends the feedback and advice generated by the generating AI back to the device. The input is the feedback and advice as the analysis results, and the output is the data sent to the user's device. Specifically, the server organizes the information received from the generating AI and sends advice such as, "Your body fat percentage is high. Please review your diet" to the user's smartphone or tablet.

[0571] Step 6:

[0572] The feedback and advice received by the user's device is visually displayed. The input is the feedback and advice sent from the server, and the output is the visual display content. Specifically, the smartphone application displays on the screen "Body fat percentage 20%, skin temperature 36.5°C" along with the advice "Your body fat percentage is high. Please review your diet."

[0573] This process allows users to understand their own health status in real time and receive appropriate advice.

[0574] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0575] This invention combines an IoT mirror system designed to allow users to easily manage their health in their daily lives with an emotion engine that recognizes the user's emotions. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means in the server for transmitting this data to a generation AI for analysis, a means for transmitting feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and an emotion engine that recognizes the user's emotions.

[0576] System Components and Functions

[0577] Sensor Means

[0578] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[0579] communication means

[0580] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[0581] Server and Generating AI

[0582] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[0583] Emotion Engine

[0584] The emotion engine recognizes emotions by analyzing the user's facial expressions and tone of voice. For example, when a user stands in front of a mirror checking their body fat percentage or skin temperature, it can determine whether the user is happy or stressed.

[0585] Generate and send feedback and advice

[0586] The generative AI generates feedback and advice appropriate for the user based on the analysis results. Meanwhile, the emotion engine appropriately modifies the feedback and advice provided by the generative AI based on the user's emotions as recognized. For example, if the user is feeling stressed, the advice may be adjusted to include relaxation techniques.

[0587] Terminals and Visual Displays

[0588] The device will visually display this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with tailored advice provided by the generative AI and emotion engine.

[0589] Specific examples

[0590] Below are some concrete examples of how this system works in a user's daily life:

[0591] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0592] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0593] 3. The measurement data is sent to the server via a communication means.

[0594] 4. The server stores this data and sends it to the generating AI.

[0595] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0596] 6. The emotion engine analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[0597] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion engine, generating adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[0598] 8. Feedback and advice is sent to the device through the server.

[0599] 9. When the user stands in front of the mirror again, the mirror will display "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[0600] In this way, the present invention helps users manage their health on a daily basis, and by combining it with the emotion engine's recognition of the user's emotions, it provides more personalized feedback and supports users in improving their lifestyle habits.

[0601] The processing flow will be explained below.

[0602] Step 1:

[0603] A user stands in front of the IoT mirror in the bathroom in the morning.

[0604] Step 2:

[0605] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[0606] Step 3:

[0607] The device sends the measured data (e.g., body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[0608] Step 4:

[0609] The server stores the received measurement data in a database and also sends the measurement data to the generation AI.

[0610] Step 5:

[0611] The generating AI analyzes the received data and the user's past health data to assess the user's health condition.

[0612] Step 6:

[0613] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[0614] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[0615] Step 7:

[0616] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

[0617] Step 8:

[0618] The server sends the analysis results of the emotion engine to the generation AI.

[0619] Step 9:

[0620] The generative AI adjusts feedback and advice based on the analysis results of the emotion engine.

[0621] For example: If the user is feeling stressed, "I recommend increasing your exercise and taking time to relax."

[0622] Step 10:

[0623] The generated AI sends the generated feedback and advice to the server.

[0624] Step 11:

[0625] The server sends the feedback and advice received from the generated AI to the terminal.

[0626] Step 12:

[0627] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[0628] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax."

[0629] Step 13:

[0630] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[0631] Example: A user decides to incorporate meditation time into their morning jog.

[0632] Example 2

[0633] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0634] Conventional health management systems have difficulty providing personalized feedback and advice because they cannot consider how data such as a user's body fat percentage and skin temperature correlates with the user's emotions and stress levels. Furthermore, even if a user is feeling stressed or frustrated, they cannot generate advice that takes this into account, preventing effective health management.

[0635] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI and analyzing it, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion recognition means for analyzing the user's facial expression and tone of voice to recognize their emotions, and a means for appropriately correcting the feedback and advice provided by the generation AI based on the results of the emotion recognition means. This enables personalized health management that takes the user's emotions into consideration.

[0636] "Sensor means" refers to a device that measures the user's body fat percentage and skin temperature.

[0637] "Communication means" refers to an interface for transmitting measurement data to a server.

[0638] "Generative AI" refers to artificial intelligence that analyzes measurement data on the server and generates feedback and advice.

[0639] "Emotion recognition means" refers to technology that recognizes emotions by analyzing a user's facial expressions and tone of voice.

[0640] "Means for sending feedback and advice back to the terminal" refers to a communication means for sending the feedback and advice generated by the generating AI to the terminal.

[0641] "Means for visually displaying feedback and advice at a terminal" refers to a display function on a terminal for visually displaying feedback and advice to a user.

[0642] "Means for appropriately modifying the feedback and advice provided by the generating AI based on the results of the emotion recognition means" refers to an algorithm that enables the generating AI to modify its feedback and advice in consideration of the user's emotional data obtained by the emotion recognition means.

[0643] The present invention relates to a system that allows users to easily manage their health on a daily basis, and is characterized in that it provides personalized feedback and advice taking into account the user's emotions. Specific implementation methods for this system are described below.

[0644] Sensor Means

[0645] When a user stands in front of the mirror, the sensors built into the IoT mirror automatically measure the user's body fat percentage and skin temperature. The sensors include a bioimpedance sensor to measure body fat percentage and a thermistor to measure skin temperature.

[0646] communication means

[0647] The measurement data is sent to the server via a communication means such as a Wi-Fi module or Bluetooth module, eliminating the need for users to manually input the measurement data.

[0648] Server and Generating AI

[0649] The server stores the received data in a database. The stored data is then sent to the Generator AI for analysis. This Generator AI uses advanced artificial intelligence, such as GPT-4. This Generator AI analyzes past health data and newly received data to assess the user's health status and generate specific feedback and advice.

[0650] emotion recognition means

[0651] The mirror uses deep learning technology to recognize emotions by analyzing the user's facial expressions and tone of voice. A camera and microphone attached to the mirror capture the user's facial expressions and voice for analysis.

[0652] Generate feedback and advice

[0653] The AI ​​then generates feedback and advice for the user based on the analysis results. Furthermore, the emotion recognition system takes the user's emotions into account and modifies the AI's feedback and advice accordingly. For example, if the user is feeling stressed, the AI ​​may add relaxation techniques.

[0654] Viewing feedback and advice

[0655] The generated feedback and advice is sent to the device via the server, and when the user stands in front of the mirror again, the adjusted feedback and advice is displayed on the mirror along with the current values ​​of body fat percentage and skin temperature.

[0656] Specific examples

[0657] Below are some concrete examples of how this system works in a user's daily life:

[0658] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0659] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0660] 3. The measurement data is sent to the server via the communication means.

[0661] 4. The server stores this data and sends it to the generating AI.

[0662] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0663] 6. The emotion recognition means analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[0664] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion recognition means, and generates adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[0665] 8. Feedback and advice is sent to the device through the server.

[0666] 9. When the user stands in front of the mirror again, the mirror displays "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[0667] Prompt Sentence Examples

[0668] Below are some example prompts to be input to the generative AI model:

[0669] The user's body fat percentage has increased recently, so you would like to advise the user to increase their exercise. Additionally, the user is feeling stressed, so you would like to encourage them to take time to relax.

[0670] As a result, the present invention can provide personalized feedback and advice that takes into account the user's emotions, allowing the user to manage their health more effectively.

[0671] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] A user stands in front of the IoT mirror in the bathroom in the morning. Specifically, the user simply stands in front of the mirror to wash their face, and the system automatically activates. The input is the user's presence, and the output is the trigger that activates the sensor.

[0674] Step 2:

[0675] The sensor means measures body fat percentage and skin temperature. Specifically, a bioimpedance sensor mounted on the mirror measures body fat percentage, and a thermistor measures skin temperature. The input is the user's body fat percentage and skin temperature, and the output is these measurement data.

[0676] Step 3:

[0677] The device sends the measurement data to the server. Specifically, the communication means (Wi-Fi module) sends the measurement data (e.g., body fat percentage 20%, skin temperature 36.5°C) to the server in real time. The input is the measurement data, and the output is the data sent to the server.

[0678] Step 4:

[0679] The server stores the received data in a database. Specifically, the server executes a procedure to store the data in a database such as MySQL. The input is the transmitted measurement data, and the output is the stored data.

[0680] Step 5:

[0681] The server sends data to the generation AI. Specifically, the server sends data to the generation AI (e.g., GPT-4) via an API. The input is the stored measurement data, and the output is the data sent to the generation AI.

[0682] Step 6:

[0683] The generating AI analyzes the received data and generates feedback and advice. Specifically, the generating AI analyzes the user's past health data and newly received data. The input is measurement data and past health data, and the output is feedback and advice such as, "Your body fat percentage has been increasing recently. Please consider increasing your exercise."

[0684] Step 7:

[0685] The emotion recognition means recognizes the user's emotions. Specifically, a camera and microphone attached to the mirror capture the user's facial expressions and tone of voice, which are then analyzed by a deep learning model. The input is the user's facial expression and voice data, and the output is the user's emotional data.

[0686] Step 8:

[0687] The generative AI modifies the feedback and advice based on the results of the emotion recognition means. Specifically, the generative AI takes the emotion data into account and modifies the advice to "Increase your exercise and take time to relax." The inputs are the feedback, advice, and emotion data, and the output is the modified feedback and advice.

[0688] Step 9:

[0689] The server sends the feedback and advice to the terminal. As a specific operation, the feedback and advice are sent to the terminal again through the communication means. The input is the modified feedback and advice, and the output is the data sent to the terminal.

[0690] Step 10:

[0691] The device visually displays feedback and advice. Specifically, when the user stands in front of the mirror again, the mirror displays the message "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax." The input is the data sent to the device, and the output is the feedback and advice displayed on the mirror.

[0692] (Application example 2)

[0693] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0694] In autonomous vehicles, safety and comfort are not sufficiently ensured because the driver's health and emotional state are not monitored in real time. In addition, it is difficult to provide appropriate advice based on the driver's health and emotional state, which may result in the driver's fatigue and stress being overlooked.

[0695] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0696] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion analysis means for analyzing the driver's facial expression and tone of voice to recognize the emotional state, and a means for appropriately correcting the feedback and advice by the generation AI based on the emotion analysis results. This makes it possible to manage the driver's health and emotional state in real time in an autonomous vehicle and provide appropriate feedback and advice.

[0697] A "user" is a person who utilizes the system to receive feedback on their health and emotional state.

[0698] "Body fat percentage" is a numerical value that indicates the percentage of fat present in the user's body.

[0699] "Skin temperature" is a numerical value measuring the surface temperature of the user's skin.

[0700] "Sensor means" refers to devices and techniques for measuring body fat percentage and skin temperature.

[0701] "Communication means" refers to the communication protocol and device for transmitting the measured data to the server.

[0702] "Generative AI" refers to artificial intelligence technology that analyzes past health data and newly received data to generate appropriate feedback and advice.

[0703] "Emotion analysis means" refers to technology and devices that analyze facial expressions and tone of voice to recognize the user's emotional state.

[0704] "Terminal" refers to a device for visually displaying feedback and advice to a user.

[0705] "Feedback and advice" refers to specific instructions and advice provided to the user based on the results of the generative AI's analysis.

[0706] "Emotion analysis result" refers to the analysis result of the user's emotional state obtained by the emotion analysis means.

[0707] "Cloud server" refers to the computer system that stores the measured data and on which the generative AI and emotion analysis means run.

[0708] The present invention relates to a system for providing real-time monitoring of the driver's well-being and emotional state in an autonomous vehicle. The system includes the following main components:

[0709] Sensor Means

[0710] Health status data collection

[0711] The sensor means is installed in the autonomous vehicle and includes a device for measuring body fat percentage and skin temperature. The body fat percentage sensor and skin temperature sensor measure the driver's health data in real time and acquire the values.

[0712] communication means

[0713] Data transmission

[0714] The acquired body fat percentage and skin temperature data is sent in real time to a cloud server via a communication module in the vehicle (e.g., Bluetooth, Wi-Fi), eliminating the need for the driver to check the measurement data directly.

[0715] Server and Generating AI

[0716] Data analysis

[0717] The cloud server stores the received body fat percentage and skin temperature data and sends it to the generative AI model (e.g., GPT-4), which analyzes this data, evaluates the driver's health status, and generates specific feedback and advice.

[0718] Emotion analysis means

[0719] emotion recognition

[0720] Emotion analysis refers to technology for analyzing the driver's facial expressions and tone of voice. Driver information is collected using the smartphone's camera and microphone, and analyzed by an emotion analysis engine (e.g., Emotion AI SDK). This allows the driver's emotional state (e.g., stress, satisfaction) to be recognized.

[0721] Generate and send feedback and advice

[0722] Feedback Adjustment

[0723] The generative AI will then adjust the feedback and advice appropriately based on the emotional data obtained from the emotion analysis means. For example, if the emotion analysis determines that the driver is feeling stressed, the AI ​​will adjust the advice by adding relaxation techniques.

[0724] Terminals and Visual Displays

[0725] Information presentation

[0726] Final feedback and advice is provided to the driver via a smartphone application, which displays the driver's current body fat percentage and skin temperature, as well as tailored feedback and advice provided by the AI.

[0727] Specific examples

[0728] Below is a concrete example of how this system works within a self-driving vehicle.

[0729] 1. The driver enters the vehicle and the sensors measure his body fat percentage at 19% and his skin temperature at 36.0°C.

[0730] 2. Measurement data is sent in real time to a cloud server via a smartphone.

[0731] 3. The AI ​​analyzes the data stored on the cloud server and generates feedback such as, "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break."

[0732] 4. The emotion analysis engine recognizes fatigue from the driver's facial expression, and the generative AI adjusts the feedback to, "It would be a good idea to exercise and drink a warm drink."

[0733] 5. Calibrated feedback will be displayed on your smartphone.

[0734] Prompt Sentence Examples

[0735] User Health Data:

[0736] Body fat percentage: 19%

[0737] Skin temperature: 36.0 degrees

[0738] User's emotional state: Fatigue

[0739] Generate feedback and advice:

[0740] Your body fat percentage is normal, but your skin temperature may be low. We recommend taking a break.

[0741] Exercise and drink warm drinks.

[0742] As described above, this invention enables drivers to manage their health and monitor their emotional state while in an autonomous vehicle, and provides appropriate feedback and advice, which is expected to significantly improve driving safety and comfort.

[0743] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0744] Step 1:

[0745] Health data collection

[0746] When a user sits in the autonomous vehicle, the sensor measures their body fat percentage and skin temperature, and the measured data is sent to a smartphone.

[0747] Input: User's body fat percentage and skin temperature

[0748] Output: Measurement data (e.g., body fat percentage 19%, skin temperature 36.0°C)

[0749] Step 2:

[0750] Data transmission

[0751] The device (smartphone) transmits the collected body fat percentage and skin temperature data to a cloud server in real time, using Bluetooth and Wi-Fi as the communication method.

[0752] Input: Measurement data (body fat percentage 19%, skin temperature 36.0°C)

[0753] Output: Data sent to the cloud server

[0754] Step 3:

[0755] Data storage and analysis preparation

[0756] The server stores the received data in a database and prepares to send the data to the generation AI.

[0757] Input: Data sent to the cloud server

[0758] Output: Data stored in the database, input data to the generative AI

[0759] Step 4:

[0760] Analysis by generative AI

[0761] The AI ​​analyzes the body fat percentage and skin temperature data sent from the server, as well as past health data, to assess the user's current health condition. Based on the analysis results, it generates specific feedback and advice.

[0762] Input: Data input to the AI ​​(body fat percentage 19%, skin temperature 36.0°C)

[0763] Output: Generated feedback and advice (e.g., "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break.")

[0764] Step 5:

[0765] Emotion recognition using emotion analysis methods

[0766] The smartphone's camera and microphone are used to analyze the user's facial expressions and tone of voice, and an emotion analysis engine (e.g., Emotion AI SDK) recognizes the user's emotional state.

[0767] Input: facial expression data and tone of voice

[0768] Output: Sentiment analysis result (e.g., fatigue)

[0769] Step 6:

[0770] Feedback Adjustment

[0771] The generative AI then adjusts the feedback and advice appropriately based on the emotional data obtained from the emotion analysis tool. For example, if the user is feeling stressed, it will add relaxation techniques.

[0772] Input: Generated feedback and advice, sentiment analysis results

[0773] Output: Tailored feedback and advice (e.g., "You should exercise and drink warm drinks.")

[0774] Step 7:

[0775] Sending feedback and suggestions

[0776] The server sends tailored feedback and advice to the terminal.

[0777] Input: Calibrated feedback and advice

[0778] Output: Feedback and advice sent to the device

[0779] Step 8:

[0780] Information presentation

[0781] The device (smartphone) visually displays tailored feedback and advice to the driver.

[0782] Input: Feedback and advice sent to the device

[0783] Output: Feedback and advice displayed on the smartphone (e.g., "It would be good for you to exercise and drink warm drinks.")

[0784] The above are the specific processing steps of the system that realizes the application example.

[0785] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0786] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0787] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0788] [Third embodiment]

[0789] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0790] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0791] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0792] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0793] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0794] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0795] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0796] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0797] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0798] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0799] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0800] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0801] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means on the server for having the generated AI analyze the data, a means for transmitting feedback and advice generated by the generated AI back to the terminal, and a means for visually displaying the feedback and advice on the terminal.

[0802] System Components and Functions

[0803] Sensor Means

[0804] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[0805] communication means

[0806] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[0807] Server and Generating AI

[0808] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[0809] Send feedback and advice

[0810] The feedback and advice generated by the AI ​​is then sent back to the device via the server. For example, if it determines that the body fat percentage is on the rise, the AI ​​might generate the advice, "Your body fat percentage has been increasing recently. Consider increasing your exercise."

[0811] Terminals and Visual Displays

[0812] The device visually displays this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[0813] Specific examples

[0814] Below are some concrete examples of how this system works in a user's daily life:

[0815] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0816] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0817] 3. The measurement data is sent to the server via a communication means.

[0818] 4. The server stores this data and sends it to the generating AI.

[0819] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0820] 6. Feedback from the generated AI is sent to the device via the server.

[0821] 7. When the user stands in front of the mirror again, the mirror will display the AI's advice, along with "Body fat percentage 20%, skin temperature 36.5 degrees."

[0822] In this way, the present invention helps users manage their health on a daily basis, allowing users to easily check their own health status and receive advice tailored to their needs.

[0823] The processing flow will be explained below.

[0824] Step 1:

[0825] A user stands in front of the IoT mirror in the bathroom in the morning.

[0826] Step 2:

[0827] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[0828] Step 3:

[0829] The device sends the measured data (body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[0830] Step 4:

[0831] The server stores the received measurement data in a database.

[0832] Step 5:

[0833] The server prepares the saved data to be sent to the generation AI.

[0834] Step 6:

[0835] The generating AI analyzes the received data and past data to assess the user's health status.

[0836] Step 7:

[0837] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[0838] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[0839] Step 8:

[0840] The generated AI sends the generated feedback and advice to the server.

[0841] Step 9:

[0842] The server sends the feedback and advice received from the generated AI to the terminal.

[0843] Step 10:

[0844] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[0845] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "Your body fat percentage has increased recently. Consider increasing your exercise."

[0846] Step 11:

[0847] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[0848] Example: A user decides to add a morning jog.

[0849] Example 1

[0850] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0851] In today's modern living environment, it is important to monitor individual health conditions in real time and receive appropriate feedback and advice, but the systems required to do so are complex and difficult to use.In addition, the lack of analytical tools to provide personalized health advice makes it difficult to provide accurate health management.

[0852] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0853] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, a means for automatically transmitting multiple pieces of data measured by the sensor means to the server, and a means for evaluating the user's health condition with the generation AI and generating advice based on past and current data. This enables users to easily monitor their own health condition in real time in their daily lives and receive personalized feedback and advice.

[0854] The "sensor means" is a device that measures the user's body fat percentage and skin temperature.

[0855] The "communication means" is a network device for transmitting measurement data to a server.

[0856] The "server" is a computer system that receives measurement data, sends it to the generation AI, and sends the analysis results to the terminal.

[0857] "Generative AI" is artificial intelligence that analyzes the data it receives and generates feedback and advice about the user's health status.

[0858] A "terminal" is a device that visually displays feedback and advice to a user.

[0859] A "database" is an information system for storing received measurement data.

[0860] A "prompt sentence" is an instruction sentence that the generative AI uses to analyze the user's health condition.

[0861] "Display means" refers to a display device that allows a user to visually confirm information.

[0862] "Health status assessment" means that the generative AI analyzes new and old data to determine the user's physical condition.

[0863] "Feedback" is information that the generative AI provides to the user based on the analysis results.

[0864] "Advice" is specific instructions provided by the generative AI to improve the user's health.

[0865] The above are definitions of important terms contained in the claims.

[0866] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives, and is mainly composed of sensor means, communication means, a server, generation AI, and a terminal. Specific use cases and the hardware and software used are described below.

[0867] Sensor Means

[0868] The sensor means is used to measure the user's body fat percentage and skin temperature. This system uses a general-purpose body fat scale sensor and temperature sensor. For example, a commercially available general-purpose sensor is used as the body fat scale sensor, and a high-precision temperature sensor is used. The sensors are embedded in the IoT mirror and automatically start up and collect data when the user stands in front of the mirror.

[0869] communication means

[0870] The measured data is sent to the server in real time via a communication means. This communication uses common network technologies such as Wi-Fi. Specifically, by using an ESP8266 as a Wi-Fi module, data acquired from the sensor can be sent to the server quickly and reliably.

[0871] Server and Data Analysis

[0872] The server stores the received data and sends it to the generation AI for analysis. The server uses a database management system (e.g., MySQL) to efficiently store the measurement data. The server manages the data using SQL statements such as:

[0873] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[0874] Analysis by generative AI

[0875] The server sends the data to a generator AI, which evaluates the user's health status. The generator AI uses, for example, OpenAI's GPT model. The generator AI analyzes the data using prompts like the following:

[0876] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[0877] The generative AI generates appropriate feedback and advice based on the input data, such as "Your body fat percentage has increased recently. Consider increasing your exercise."

[0878] Send feedback and advice

[0879] The feedback and advice generated by the AI ​​is then sent back to the device via the server, again via the Wi-Fi module.

[0880] Display on device

[0881] The device visually displays this feedback and advice to the user. For example, a Raspberry Pi touchscreen monitor can be used as the display device. When the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[0882] Specifically, it will appear as follows:

[0883] Body fat percentage: 20%

[0884] Skin temperature: 36.5 degrees

[0885] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[0886] In this way, the IoT mirror system provides support for users in managing their health on a daily basis, allowing them to easily monitor their own health status and receive accurate feedback and advice.

[0887] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0888] Step 1:

[0889] A user stands in front of the IoT mirror in the bathroom in the morning. The sensor means is automatically activated to measure the user's body fat percentage and skin temperature. The input is "user's body fat percentage and skin temperature" and the output is "measured data (e.g. body fat percentage 20%, skin temperature 36.5 degrees)." The sensor uses a body fat scale sensor and a temperature sensor to acquire data.

[0890] Step 2:

[0891] The device transmits the measured data to the server in real time via a communication means. The input is the "measured data (body fat percentage 20%, skin temperature 36.5°C)" and the output is the "data transmitted to the server." A Wi-Fi module (e.g., ESP8266) is used for this communication.

[0892] Step 3:

[0893] The data received by the server is saved in a database. The input is "data sent to the server" (e.g., user ID, measurement date and time, body fat percentage, skin temperature), and the output is "data saved in the database." A MySQL database is used for saving, and the following SQL statement is issued:

[0894] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[0895] Step 4:

[0896] The server formats the data to send to the generation AI. The input is "data stored in the database" (e.g., body fat percentage, skin temperature, past measurement data), and the output is "formatted data" (e.g., data sent with the prompt). The prompt is formatted as follows:

[0897] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[0898] Step 5:

[0899] The generative AI analyzes the data and generates feedback and advice. The input is "formatted data" (e.g., the latest body fat percentage and skin temperature, past data), and the output is "generated feedback and advice." For example, feedback in the form of "Your body fat percentage has increased recently. Please consider increasing your exercise" is generated.

[0900] Step 6:

[0901] The server sends the generated feedback and advice to the terminal. The input is "generated feedback and advice" and the output is "feedback and advice sent to the terminal." This communication is also done via the Wi-Fi module.

[0902] Step 7:

[0903] The terminal displays visual feedback and advice to the user. The input is "feedback and advice sent to the terminal" and the output is "visually displayed feedback and advice". Specifically, it is displayed on the Raspberry Pi's touchscreen monitor as follows:

[0904] Body fat percentage: 20%

[0905] Skin temperature: 36.5 degrees

[0906] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[0907] This allows users to easily check their health status and receive appropriate advice.

[0908] (Application example 1)

[0909] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0910] Conventional health management systems require users to voluntarily enter data, making them difficult to use on a daily basis. Furthermore, there was a lack of a way for fitness facilities to efficiently manage members' health data and provide individualized feedback in real time.

[0911] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0912] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and a means for measuring the member's health data at the fitness facility and providing feedback analyzed by the generation AI in real time. This allows users to easily obtain health data on a daily basis and receive individual health advice at the fitness facility.

[0913] "User" refers to a member of a fitness facility or an individual person who generally uses the system.

[0914] "Body fat percentage" is health data that indicates the percentage of fat in a user's total body weight.

[0915] "Skin temperature" is a measurement of the temperature of the user's skin surface, and is data that indicates part of the user's health condition.

[0916] "Sensor means" refers to an apparatus or device for measuring body fat percentage and skin temperature in real time.

[0917] "Communication means" refers to the technology, including network connections and protocols, for transmitting measurement data to a server.

[0918] "Server" refers to the central management system for receiving measurement data and transmitting it to the storage and generating AI.

[0919] "Generative AI" is an artificial intelligence tool that analyzes measurement data and generates feedback and health advice.

[0920] "Feedback" refers to the evaluations and comments that the generative AI provides to the user based on the analysis results.

[0921] "Advice" refers to specific recommendations or suggestions for action that the generative AI provides to users based on the analysis results.

[0922] "Terminal" refers to the device (smartphone, tablet, IoT mirror, etc.) through which the user visually views feedback and advice.

[0923] "Fitness facilities" refer to facilities where individuals can exercise and manage their health, such as gyms and fitness clubs.

[0924] The present invention relates to a "Gym Health Advisor" system for efficiently managing the health of members at fitness facilities. Specific embodiments for realizing this system will be described below.

[0925] System Overview

[0926] The system uses IoT mirrors installed in fitness facilities to measure members' health data and provide real-time feedback analyzed by generative AI.

[0927] Hardware and software used

[0928] IoT Mirror: Equipped with sensors to measure body fat percentage and skin temperature.

[0929] Communication method: Measurement data is sent to the server via Wi-Fi, Bluetooth, etc.

[0930] Server: Stores measurement data in a database and sends the data to the generation AI.

[0931] Generative AI: Analyzes measurement data and generates feedback and advice. For generative AI, models such as GPT-4 are used.

[0932] Device: A device, such as a smartphone or tablet, that allows members to visually view feedback and advice.

[0933] Program processing overview

[0934] The server receives the user's body fat percentage and skin temperature measurement data sent from the IoT mirror and stores it in a database.The server then sends this measurement data to the generation AI for analysis.

[0935] The Generative AI uses the user's past health data and newly received data to assess their health status and generate personalized feedback and advice, which are then sent to the device via a server and displayed visually on the device.

[0936] Specific examples

[0937] When a user stands in front of the IoT mirror at a fitness facility, the mirror measures their body fat percentage and skin temperature and sends the data to a server in real time. The server receives this data and sends it to the generating AI. The generating AI analyzes the data and generates advice such as, "Your body fat percentage is high. Please review your diet." This feedback and advice is then sent back to the user's smartphone via the server.

[0938] Example prompts for generative AI models

[0939] Analyze the user's body fat percentage (20%) and skin temperature (36.5°C) and generate appropriate feedback and health management advice.

[0940] In this way, the present invention is designed to efficiently manage the health of members at fitness facilities, allowing users to easily check their daily health status and receive appropriate advice in real time.

[0941] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0942] Step 1:

[0943] When a user stands in front of an IoT mirror at a fitness facility, the sensor built into the IoT mirror measures body fat percentage and skin temperature. The input is the user's physical data (body fat percentage and skin temperature), and the output is the measured health data. Specifically, the sensor measures the user's body fat percentage as, for example, 20% and records the skin temperature as 36.5 degrees.

[0944] Step 2:

[0945] The measurement data is sent to a server in real time via a communication method. The input is the body fat percentage and skin temperature data obtained from the sensor, and the output is the data sent to the server. Specifically, the IoT mirror uses Wi-Fi and Bluetooth to send data such as a body fat percentage of 20% and a skin temperature of 36.5 degrees to the server.

[0946] Step 3:

[0947] The server stores the received measurement data in a database. The input is the transmitted data (body fat percentage and skin temperature), and the output is the record stored in the database. Specifically, the server uses database software (e.g., MySQL) to store the received data in a specific table.

[0948] Step 4:

[0949] The server sends the saved measurement data to the generation AI for analysis. The input is the saved measurement data and past health data, and the output is the analysis results. Specifically, the server sends the data to a generation AI model (e.g., GPT-4) and performs the analysis using the prompt, "Analyze the user's body fat percentage of 20% and skin temperature of 36.5 degrees, and generate appropriate feedback and health management advice."

[0950] Step 5:

[0951] The server then sends the feedback and advice generated by the generating AI back to the device. The input is the feedback and advice as the analysis results, and the output is the data sent to the user's device. Specifically, the server organizes the information received from the generating AI and sends advice such as, "Your body fat percentage is high. Please review your diet" to the user's smartphone or tablet.

[0952] Step 6:

[0953] The feedback and advice received by the user's device is visually displayed. The input is the feedback and advice sent from the server, and the output is the visual display content. Specifically, the smartphone application displays on the screen "Body fat percentage 20%, skin temperature 36.5°C" along with the advice "Your body fat percentage is high. Please review your diet."

[0954] This process allows users to understand their own health status in real time and receive appropriate advice.

[0955] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0956] This invention combines an IoT mirror system designed to allow users to easily manage their health in their daily lives with an emotion engine that recognizes the user's emotions. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means in the server for transmitting this data to a generation AI for analysis, a means for transmitting feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and an emotion engine that recognizes the user's emotions.

[0957] System Components and Functions

[0958] Sensor Means

[0959] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[0960] communication means

[0961] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[0962] Server and Generating AI

[0963] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[0964] Emotion Engine

[0965] The emotion engine recognizes emotions by analyzing the user's facial expressions and tone of voice. For example, when a user stands in front of a mirror checking their body fat percentage or skin temperature, it can determine whether the user is happy or stressed.

[0966] Generate and send feedback and advice

[0967] The generative AI generates feedback and advice appropriate for the user based on the analysis results. Meanwhile, the emotion engine appropriately modifies the feedback and advice provided by the generative AI based on the user's emotions as recognized. For example, if the user is feeling stressed, the advice may be adjusted to include relaxation techniques.

[0968] Terminals and Visual Displays

[0969] The device will visually display this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with tailored advice provided by the generative AI and emotion engine.

[0970] Specific examples

[0971] Below are some concrete examples of how this system works in a user's daily life:

[0972] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[0973] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[0974] 3. The measurement data is sent to the server via a communication means.

[0975] 4. The server stores this data and sends it to the generating AI.

[0976] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[0977] 6. The emotion engine analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[0978] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion engine, generating adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[0979] 8. Feedback and advice is sent to the device through the server.

[0980] 9. When the user stands in front of the mirror again, the mirror will display "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[0981] In this way, the present invention helps users manage their health on a daily basis, and by combining it with the emotion engine's recognition of the user's emotions, it provides more personalized feedback and supports users in improving their lifestyle habits.

[0982] The processing flow will be explained below.

[0983] Step 1:

[0984] A user stands in front of the IoT mirror in the bathroom in the morning.

[0985] Step 2:

[0986] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[0987] Step 3:

[0988] The device sends the measured data (e.g., body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[0989] Step 4:

[0990] The server stores the received measurement data in a database and also sends the measurement data to the generation AI.

[0991] Step 5:

[0992] The generating AI analyzes the received data and the user's past health data to assess the user's health condition.

[0993] Step 6:

[0994] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[0995] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[0996] Step 7:

[0997] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

[0998] Step 8:

[0999] The server sends the analysis results of the emotion engine to the generation AI.

[1000] Step 9:

[1001] The generative AI adjusts feedback and advice based on the analysis results of the emotion engine.

[1002] For example: If the user is feeling stressed, "I recommend increasing your exercise and taking time to relax."

[1003] Step 10:

[1004] The generated AI sends the generated feedback and advice to the server.

[1005] Step 11:

[1006] The server sends the feedback and advice received from the generated AI to the terminal.

[1007] Step 12:

[1008] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[1009] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax."

[1010] Step 13:

[1011] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[1012] Example: A user decides to incorporate meditation time into their morning jog.

[1013] Example 2

[1014] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1015] Conventional health management systems have difficulty providing personalized feedback and advice because they cannot consider how data such as a user's body fat percentage and skin temperature correlates with the user's emotions and stress levels. Furthermore, even if a user is feeling stressed or frustrated, they cannot generate advice that takes this into account, preventing effective health management.

[1016] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI and analyzing it, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion recognition means for analyzing the user's facial expression and tone of voice to recognize their emotions, and a means for appropriately correcting the feedback and advice provided by the generation AI based on the results of the emotion recognition means. This enables personalized health management that takes the user's emotions into consideration.

[1017] "Sensor means" refers to a device that measures the user's body fat percentage and skin temperature.

[1018] "Communication means" refers to an interface for transmitting measurement data to a server.

[1019] "Generative AI" refers to artificial intelligence that analyzes measurement data on the server and generates feedback and advice.

[1020] "Emotion recognition means" refers to technology that recognizes emotions by analyzing a user's facial expressions and tone of voice.

[1021] "Means for sending feedback and advice back to the terminal" refers to a communication means for sending the feedback and advice generated by the generating AI to the terminal.

[1022] "Means for visually displaying feedback and advice at a terminal" refers to a display function on a terminal for visually displaying feedback and advice to a user.

[1023] "Means for appropriately modifying the feedback and advice provided by the generating AI based on the results of the emotion recognition means" refers to an algorithm that enables the generating AI to modify its feedback and advice in consideration of the user's emotional data obtained by the emotion recognition means.

[1024] The present invention relates to a system that allows users to easily manage their health on a daily basis, and is characterized in that it provides personalized feedback and advice taking into account the user's emotions. Specific implementation methods for this system are described below.

[1025] Sensor Means

[1026] When a user stands in front of the mirror, the sensors built into the IoT mirror automatically measure the user's body fat percentage and skin temperature. The sensors include a bioimpedance sensor to measure body fat percentage and a thermistor to measure skin temperature.

[1027] communication means

[1028] The measurement data is sent to the server via a communication means such as a Wi-Fi module or Bluetooth module, eliminating the need for users to manually input the measurement data.

[1029] Server and Generating AI

[1030] The server stores the received data in a database. The stored data is then sent to the Generator AI for analysis. This Generator AI uses advanced artificial intelligence, such as GPT-4. This Generator AI analyzes past health data and newly received data to assess the user's health status and generate specific feedback and advice.

[1031] emotion recognition means

[1032] The mirror uses deep learning technology to recognize emotions by analyzing the user's facial expressions and tone of voice. A camera and microphone attached to the mirror capture the user's facial expressions and voice for analysis.

[1033] Generate feedback and advice

[1034] The AI ​​then generates feedback and advice for the user based on the analysis results. Furthermore, the emotion recognition system takes the user's emotions into account and modifies the AI's feedback and advice accordingly. For example, if the user is feeling stressed, the AI ​​may add relaxation techniques.

[1035] Viewing feedback and advice

[1036] The generated feedback and advice is sent to the device via the server, and when the user stands in front of the mirror again, the adjusted feedback and advice is displayed on the mirror along with the current values ​​of body fat percentage and skin temperature.

[1037] Specific examples

[1038] Below are some concrete examples of how this system works in a user's daily life:

[1039] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[1040] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[1041] 3. The measurement data is sent to the server via the communication means.

[1042] 4. The server stores this data and sends it to the generating AI.

[1043] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[1044] 6. The emotion recognition means analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[1045] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion recognition means, and generates adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[1046] 8. Feedback and advice is sent to the device through the server.

[1047] 9. When the user stands in front of the mirror again, the mirror displays "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[1048] Prompt Sentence Examples

[1049] Below are some example prompts to be input to the generative AI model:

[1050] The user's body fat percentage has increased recently, so you would like to advise the user to increase their exercise. Additionally, the user is feeling stressed, so you would like to encourage them to take time to relax.

[1051] As a result, the present invention can provide personalized feedback and advice that takes into account the user's emotions, allowing the user to manage their health more effectively.

[1052] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1053] Step 1:

[1054] A user stands in front of the IoT mirror in the bathroom in the morning. Specifically, the user simply stands in front of the mirror to wash their face, and the system automatically activates. The input is the user's presence, and the output is the trigger that activates the sensor.

[1055] Step 2:

[1056] The sensor means measures body fat percentage and skin temperature. Specifically, a bioimpedance sensor mounted on the mirror measures body fat percentage, and a thermistor measures skin temperature. The input is the user's body fat percentage and skin temperature, and the output is these measurement data.

[1057] Step 3:

[1058] The device sends the measurement data to the server. Specifically, the communication means (Wi-Fi module) sends the measurement data (e.g., body fat percentage 20%, skin temperature 36.5°C) to the server in real time. The input is the measurement data, and the output is the data sent to the server.

[1059] Step 4:

[1060] The server stores the received data in a database. Specifically, the server executes a procedure to store the data in a database such as MySQL. The input is the transmitted measurement data, and the output is the stored data.

[1061] Step 5:

[1062] The server sends data to the generation AI. Specifically, the server sends data to the generation AI (e.g., GPT-4) via an API. The input is the stored measurement data, and the output is the data sent to the generation AI.

[1063] Step 6:

[1064] The generating AI analyzes the received data and generates feedback and advice. Specifically, the generating AI analyzes the user's past health data and newly received data. The input is measurement data and past health data, and the output is feedback and advice such as, "Your body fat percentage has been increasing recently. Please consider increasing your exercise."

[1065] Step 7:

[1066] The emotion recognition means recognizes the user's emotions. Specifically, a camera and microphone attached to the mirror capture the user's facial expressions and tone of voice, which are then analyzed by a deep learning model. The input is the user's facial expression and voice data, and the output is the user's emotional data.

[1067] Step 8:

[1068] The generative AI modifies the feedback and advice based on the results of the emotion recognition means. Specifically, the generative AI takes the emotion data into account and modifies the advice to "Increase your exercise and take time to relax." The inputs are the feedback, advice, and emotion data, and the output is the modified feedback and advice.

[1069] Step 9:

[1070] The server sends the feedback and advice to the terminal. As a specific operation, the feedback and advice are sent to the terminal again through the communication means. The input is the modified feedback and advice, and the output is the data sent to the terminal.

[1071] Step 10:

[1072] The device visually displays feedback and advice. Specifically, when the user stands in front of the mirror again, the mirror displays the message "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax." The input is the data sent to the device, and the output is the feedback and advice displayed on the mirror.

[1073] (Application example 2)

[1074] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1075] In autonomous vehicles, safety and comfort are not sufficiently ensured because the driver's health and emotional state are not monitored in real time. In addition, it is difficult to provide appropriate advice based on the driver's health and emotional state, which may result in the driver's fatigue and stress being overlooked.

[1076] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1077] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion analysis means for analyzing the driver's facial expression and tone of voice to recognize the emotional state, and a means for appropriately correcting the feedback and advice by the generation AI based on the emotion analysis results. This makes it possible to manage the driver's health and emotional state in real time in an autonomous vehicle and provide appropriate feedback and advice.

[1078] A "user" is a person who utilizes the system to receive feedback on their health and emotional state.

[1079] "Body fat percentage" is a numerical value that indicates the percentage of fat present in the user's body.

[1080] "Skin temperature" is a numerical value measuring the surface temperature of the user's skin.

[1081] "Sensor means" refers to devices and techniques for measuring body fat percentage and skin temperature.

[1082] "Communication means" refers to the communication protocol and device for transmitting the measured data to the server.

[1083] "Generative AI" refers to artificial intelligence technology that analyzes past health data and newly received data to generate appropriate feedback and advice.

[1084] "Emotion analysis means" refers to technology and devices that analyze facial expressions and tone of voice to recognize the user's emotional state.

[1085] "Terminal" refers to a device for visually displaying feedback and advice to a user.

[1086] "Feedback and advice" refers to specific instructions and advice provided to the user based on the results of the generative AI's analysis.

[1087] "Emotion analysis result" refers to the analysis result of the user's emotional state obtained by the emotion analysis means.

[1088] "Cloud server" refers to the computer system that stores the measured data and on which the generative AI and emotion analysis means run.

[1089] The present invention relates to a system for providing real-time monitoring of the driver's well-being and emotional state in an autonomous vehicle. The system includes the following main components:

[1090] Sensor Means

[1091] Health status data collection

[1092] The sensor means is installed in the autonomous vehicle and includes a device for measuring body fat percentage and skin temperature. The body fat percentage sensor and skin temperature sensor measure the driver's health data in real time and acquire the values.

[1093] communication means

[1094] Data transmission

[1095] The acquired body fat percentage and skin temperature data is sent in real time to a cloud server via a communication module in the vehicle (e.g., Bluetooth, Wi-Fi), eliminating the need for the driver to check the measurement data directly.

[1096] Server and Generating AI

[1097] Data analysis

[1098] The cloud server stores the received body fat percentage and skin temperature data and sends it to the generative AI model (e.g., GPT-4), which analyzes this data, evaluates the driver's health status, and generates specific feedback and advice.

[1099] Emotion analysis means

[1100] emotion recognition

[1101] Emotion analysis refers to technology for analyzing the driver's facial expressions and tone of voice. Driver information is collected using the smartphone's camera and microphone, and analyzed by an emotion analysis engine (e.g., Emotion AI SDK). This allows the driver's emotional state (e.g., stress, satisfaction) to be recognized.

[1102] Generate and send feedback and advice

[1103] Feedback Adjustment

[1104] The generative AI will then adjust the feedback and advice appropriately based on the emotional data obtained from the emotion analysis means. For example, if the emotion analysis determines that the driver is feeling stressed, the AI ​​will adjust the advice by adding relaxation techniques.

[1105] Terminals and Visual Displays

[1106] Information presentation

[1107] Final feedback and advice is provided to the driver via a smartphone application, which displays the driver's current body fat percentage and skin temperature, as well as tailored feedback and advice provided by the AI.

[1108] Specific examples

[1109] Below is a concrete example of how this system works within a self-driving vehicle.

[1110] 1. The driver enters the vehicle and the sensors measure his body fat percentage at 19% and his skin temperature at 36.0°C.

[1111] 2. Measurement data is sent in real time to a cloud server via a smartphone.

[1112] 3. The AI ​​analyzes the data stored on the cloud server and generates feedback such as, "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break."

[1113] 4. The emotion analysis engine recognizes fatigue from the driver's facial expression, and the generative AI adjusts the feedback to, "It would be a good idea to exercise and drink a warm drink."

[1114] 5. Calibrated feedback will be displayed on your smartphone.

[1115] Prompt Sentence Examples

[1116] User Health Data:

[1117] Body fat percentage: 19%

[1118] Skin temperature: 36.0 degrees

[1119] User's emotional state: Fatigue

[1120] Generate feedback and advice:

[1121] Your body fat percentage is normal, but your skin temperature may be low. We recommend taking a break.

[1122] Exercise and drink warm drinks.

[1123] As described above, this invention enables drivers to manage their health and monitor their emotional state while in an autonomous vehicle, and provides appropriate feedback and advice, which is expected to significantly improve driving safety and comfort.

[1124] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1125] Step 1:

[1126] Health data collection

[1127] When a user sits in the autonomous vehicle, the sensor measures their body fat percentage and skin temperature, and the measured data is sent to a smartphone.

[1128] Input: User's body fat percentage and skin temperature

[1129] Output: Measurement data (e.g., body fat percentage 19%, skin temperature 36.0°C)

[1130] Step 2:

[1131] Data transmission

[1132] The device (smartphone) transmits the collected body fat percentage and skin temperature data to a cloud server in real time, using Bluetooth and Wi-Fi as the communication method.

[1133] Input: Measurement data (body fat percentage 19%, skin temperature 36.0°C)

[1134] Output: Data sent to the cloud server

[1135] Step 3:

[1136] Data storage and analysis preparation

[1137] The server stores the received data in a database and prepares to send the data to the generation AI.

[1138] Input: Data sent to the cloud server

[1139] Output: Data stored in the database, input data to the generative AI

[1140] Step 4:

[1141] Analysis by generative AI

[1142] The AI ​​analyzes the body fat percentage and skin temperature data sent from the server, as well as past health data, to assess the user's current health condition. Based on the analysis results, it generates specific feedback and advice.

[1143] Input: Data input to the AI ​​(body fat percentage 19%, skin temperature 36.0°C)

[1144] Output: Generated feedback and advice (e.g., "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break.")

[1145] Step 5:

[1146] Emotion recognition using emotion analysis methods

[1147] The smartphone's camera and microphone are used to analyze the user's facial expressions and tone of voice, and an emotion analysis engine (e.g., Emotion AI SDK) recognizes the user's emotional state.

[1148] Input: facial expression data and tone of voice

[1149] Output: Sentiment analysis result (e.g., fatigue)

[1150] Step 6:

[1151] Feedback Adjustment

[1152] The generative AI then adjusts the feedback and advice appropriately based on the emotional data obtained from the emotion analysis tool. For example, if the user is feeling stressed, it will add relaxation techniques.

[1153] Input: Generated feedback and advice, sentiment analysis results

[1154] Output: Tailored feedback and advice (e.g., "You should exercise and drink warm drinks.")

[1155] Step 7:

[1156] Sending feedback and suggestions

[1157] The server sends tailored feedback and advice to the terminal.

[1158] Input: Calibrated feedback and advice

[1159] Output: Feedback and advice sent to the device

[1160] Step 8:

[1161] Information presentation

[1162] The device (smartphone) visually displays tailored feedback and advice to the driver.

[1163] Input: Feedback and advice sent to the device

[1164] Output: Feedback and advice displayed on the smartphone (e.g., "It would be good for you to exercise and drink warm drinks.")

[1165] The above are the specific processing steps of the system that realizes the application example.

[1166] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1167] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1168] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1169] [Fourth embodiment]

[1170] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1171] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1172] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1173] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1174] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1175] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1176] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1177] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1178] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1179] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1180] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1181] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1182] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1183] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means on the server for having the generated AI analyze the data, a means for transmitting feedback and advice generated by the generated AI back to the terminal, and a means for visually displaying the feedback and advice on the terminal.

[1184] System Components and Functions

[1185] Sensor Means

[1186] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[1187] communication means

[1188] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[1189] Server and Generating AI

[1190] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[1191] Send feedback and advice

[1192] The feedback and advice generated by the AI ​​is then sent back to the device via the server. For example, if it determines that the body fat percentage is on the rise, the AI ​​might generate the advice, "Your body fat percentage has been increasing recently. Consider increasing your exercise."

[1193] Terminals and Visual Displays

[1194] The device visually displays this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[1195] Specific examples

[1196] Below are some concrete examples of how this system works in a user's daily life:

[1197] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[1198] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[1199] 3. The measurement data is sent to the server via a communication means.

[1200] 4. The server stores this data and sends it to the generating AI.

[1201] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[1202] 6. Feedback from the generated AI is sent to the device via the server.

[1203] 7. When the user stands in front of the mirror again, the mirror will display the AI's advice, along with "Body fat percentage 20%, skin temperature 36.5 degrees."

[1204] In this way, the present invention helps users manage their health on a daily basis, allowing users to easily check their own health status and receive advice tailored to their needs.

[1205] The processing flow will be explained below.

[1206] Step 1:

[1207] A user stands in front of the IoT mirror in the bathroom in the morning.

[1208] Step 2:

[1209] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[1210] Step 3:

[1211] The device sends the measured data (body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[1212] Step 4:

[1213] The server stores the received measurement data in a database.

[1214] Step 5:

[1215] The server prepares the saved data to be sent to the generation AI.

[1216] Step 6:

[1217] The generating AI analyzes the received data and past data to assess the user's health status.

[1218] Step 7:

[1219] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[1220] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[1221] Step 8:

[1222] The generated AI sends the generated feedback and advice to the server.

[1223] Step 9:

[1224] The server sends the feedback and advice received from the generated AI to the terminal.

[1225] Step 10:

[1226] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[1227] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "Your body fat percentage has increased recently. Consider increasing your exercise."

[1228] Step 11:

[1229] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[1230] Example: A user decides to add a morning jog.

[1231] Example 1

[1232] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1233] In today's modern living environment, it is important to monitor individual health conditions in real time and receive appropriate feedback and advice, but the systems required to do so are complex and difficult to use.In addition, the lack of analytical tools to provide personalized health advice makes it difficult to provide accurate health management.

[1234] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1235] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, a means for automatically transmitting multiple pieces of data measured by the sensor means to the server, and a means for evaluating the user's health condition with the generation AI and generating advice based on past and current data. This enables users to easily monitor their own health condition in real time in their daily lives and receive personalized feedback and advice.

[1236] The "sensor means" is a device that measures the user's body fat percentage and skin temperature.

[1237] The "communication means" is a network device for transmitting measurement data to a server.

[1238] The "server" is a computer system that receives measurement data, sends it to the generation AI, and sends the analysis results to the terminal.

[1239] "Generative AI" is artificial intelligence that analyzes the data it receives and generates feedback and advice about the user's health status.

[1240] A "terminal" is a device that visually displays feedback and advice to a user.

[1241] A "database" is an information system for storing received measurement data.

[1242] A "prompt sentence" is an instruction sentence that the generative AI uses to analyze the user's health condition.

[1243] "Display means" refers to a display device that allows a user to visually confirm information.

[1244] "Health status assessment" means that the generative AI analyzes new and old data to determine the user's physical condition.

[1245] "Feedback" is information that the generative AI provides to the user based on the analysis results.

[1246] "Advice" is specific instructions provided by the generative AI to improve the user's health.

[1247] The above are definitions of important terms contained in the claims.

[1248] This invention is an IoT mirror system designed to allow users to easily manage their health in their daily lives, and is mainly composed of sensor means, communication means, a server, generation AI, and a terminal. Specific use cases and the hardware and software used are described below.

[1249] Sensor Means

[1250] The sensor means is used to measure the user's body fat percentage and skin temperature. This system uses a general-purpose body fat scale sensor and temperature sensor. For example, a commercially available general-purpose sensor is used as the body fat scale sensor, and a high-precision temperature sensor is used. The sensors are embedded in the IoT mirror and automatically start up and collect data when the user stands in front of the mirror.

[1251] communication means

[1252] The measured data is sent to the server in real time via a communication means. This communication uses common network technologies such as Wi-Fi. Specifically, by using an ESP8266 as a Wi-Fi module, data acquired from the sensor can be sent to the server quickly and reliably.

[1253] Server and Data Analysis

[1254] The server stores the received data and sends it to the generation AI for analysis. The server uses a database management system (e.g., MySQL) to efficiently store the measurement data. The server manages the data using SQL statements such as:

[1255] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[1256] Analysis by generative AI

[1257] The server sends the data to a generator AI, which evaluates the user's health status. The generator AI uses, for example, OpenAI's GPT model. The generator AI analyzes the data using prompts like the following:

[1258] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[1259] The generative AI generates appropriate feedback and advice based on the input data, such as "Your body fat percentage has increased recently. Consider increasing your exercise."

[1260] Send feedback and advice

[1261] The feedback and advice generated by the AI ​​is then sent back to the device via the server, again via the Wi-Fi module.

[1262] Display on device

[1263] The device visually displays this feedback and advice to the user. For example, a Raspberry Pi touchscreen monitor can be used as the display device. When the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with advice from the generated AI.

[1264] Specifically, it will appear as follows:

[1265] Body fat percentage: 20%

[1266] Skin temperature: 36.5 degrees

[1267] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[1268] In this way, the IoT mirror system provides support for users in managing their health on a daily basis, allowing them to easily monitor their own health status and receive accurate feedback and advice.

[1269] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1270] Step 1:

[1271] A user stands in front of the IoT mirror in the bathroom in the morning. The sensor means is automatically activated to measure the user's body fat percentage and skin temperature. The input is "user's body fat percentage and skin temperature" and the output is "measured data (e.g. body fat percentage 20%, skin temperature 36.5 degrees)." The sensor uses a body fat scale sensor and a temperature sensor to acquire data.

[1272] Step 2:

[1273] The device transmits the measured data to the server in real time via a communication means. The input is the "measured data (body fat percentage 20%, skin temperature 36.5°C)" and the output is the "data transmitted to the server." A Wi-Fi module (e.g., ESP8266) is used for this communication.

[1274] Step 3:

[1275] The data received by the server is saved in a database. The input is "data sent to the server" (e.g., user ID, measurement date and time, body fat percentage, skin temperature), and the output is "data saved in the database." A MySQL database is used for saving, and the following SQL statement is issued:

[1276] INSERT INTO HealthDB (userID, measured_at, body_fat, skin_temp) VALUES ('user123', '2023-10-21 07:30:00', '20%', '36.5C');

[1277] Step 4:

[1278] The server formats the data to send to the generation AI. The input is "data stored in the database" (e.g., body fat percentage, skin temperature, past measurement data), and the output is "formatted data" (e.g., data sent with the prompt). The prompt is formatted as follows:

[1279] "My body fat percentage has been increasing recently. What feedback and advice should I provide to my users based on their exercise and food logs from the past week?"

[1280] Step 5:

[1281] The generative AI analyzes the data and generates feedback and advice. The input is "formatted data" (e.g., the latest body fat percentage and skin temperature, past data), and the output is "generated feedback and advice." For example, feedback in the form of "Your body fat percentage has increased recently. Please consider increasing your exercise" is generated.

[1282] Step 6:

[1283] The server sends the generated feedback and advice to the terminal. The input is "generated feedback and advice" and the output is "feedback and advice sent to the terminal." This communication is also done via the Wi-Fi module.

[1284] Step 7:

[1285] The terminal displays visual feedback and advice to the user. The input is "feedback and advice sent to the terminal" and the output is "visually displayed feedback and advice". Specifically, it is displayed on the Raspberry Pi's touchscreen monitor as follows:

[1286] Body fat percentage: 20%

[1287] Skin temperature: 36.5 degrees

[1288] Advice: Your body fat percentage has increased recently. Consider increasing your exercise.

[1289] This allows users to easily check their health status and receive appropriate advice.

[1290] (Application example 1)

[1291] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1292] Conventional health management systems require users to voluntarily enter data, making them difficult to use on a daily basis. Furthermore, there was a lack of a way for fitness facilities to efficiently manage members' health data and provide individualized feedback in real time.

[1293] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1294] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and a means for measuring the member's health data at the fitness facility and providing feedback analyzed by the generation AI in real time. This allows users to easily obtain health data on a daily basis and receive individual health advice at the fitness facility.

[1295] "User" refers to a member of a fitness facility or an individual person who generally uses the system.

[1296] "Body fat percentage" is health data that indicates the percentage of fat in a user's total body weight.

[1297] "Skin temperature" is a measurement of the temperature of the user's skin surface, and is data that indicates part of the user's health condition.

[1298] "Sensor means" refers to an apparatus or device for measuring body fat percentage and skin temperature in real time.

[1299] "Communication means" refers to the technology, including network connections and protocols, for transmitting measurement data to a server.

[1300] "Server" refers to the central management system for receiving measurement data and transmitting it to the storage and generating AI.

[1301] "Generative AI" is an artificial intelligence tool that analyzes measurement data and generates feedback and health advice.

[1302] "Feedback" refers to the evaluations and comments that the generative AI provides to the user based on the analysis results.

[1303] "Advice" refers to specific recommendations or suggestions for action that the generative AI provides to users based on the analysis results.

[1304] "Terminal" refers to the device (smartphone, tablet, IoT mirror, etc.) through which the user visually views feedback and advice.

[1305] "Fitness facilities" refer to facilities where individuals can exercise and manage their health, such as gyms and fitness clubs.

[1306] The present invention relates to a "Gym Health Advisor" system for efficiently managing the health of members at fitness facilities. Specific embodiments for realizing this system will be described below.

[1307] System Overview

[1308] The system uses IoT mirrors installed in fitness facilities to measure members' health data and provide real-time feedback analyzed by generative AI.

[1309] Hardware and software used

[1310] IoT Mirror: Equipped with sensors to measure body fat percentage and skin temperature.

[1311] Communication method: Measurement data is sent to the server via Wi-Fi, Bluetooth, etc.

[1312] Server: Stores measurement data in a database and sends the data to the generation AI.

[1313] Generative AI: Analyzes measurement data and generates feedback and advice. For generative AI, models such as GPT-4 are used.

[1314] Device: A device, such as a smartphone or tablet, that allows members to visually view feedback and advice.

[1315] Program processing overview

[1316] The server receives the user's body fat percentage and skin temperature measurement data sent from the IoT mirror and stores it in a database.The server then sends this measurement data to the generation AI for analysis.

[1317] The Generative AI uses the user's past health data and newly received data to assess their health status and generate personalized feedback and advice, which are then sent to the device via a server and displayed visually on the device.

[1318] Specific examples

[1319] When a user stands in front of the IoT mirror at a fitness facility, the mirror measures their body fat percentage and skin temperature and sends the data to a server in real time. The server receives this data and sends it to the generating AI. The generating AI analyzes the data and generates advice such as, "Your body fat percentage is high. Please review your diet." This feedback and advice is then sent back to the user's smartphone via the server.

[1320] Example prompts for generative AI models

[1321] Analyze the user's body fat percentage (20%) and skin temperature (36.5°C) and generate appropriate feedback and health management advice.

[1322] In this way, the present invention is designed to efficiently manage the health of members at fitness facilities, allowing users to easily check their daily health status and receive appropriate advice in real time.

[1323] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1324] Step 1:

[1325] When a user stands in front of an IoT mirror at a fitness facility, the sensor built into the IoT mirror measures body fat percentage and skin temperature. The input is the user's physical data (body fat percentage and skin temperature), and the output is the measured health data. Specifically, the sensor measures the user's body fat percentage as, for example, 20% and records the skin temperature as 36.5 degrees.

[1326] Step 2:

[1327] The measurement data is sent to a server in real time via a communication method. The input is the body fat percentage and skin temperature data obtained from the sensor, and the output is the data sent to the server. Specifically, the IoT mirror uses Wi-Fi and Bluetooth to send data such as a body fat percentage of 20% and a skin temperature of 36.5 degrees to the server.

[1328] Step 3:

[1329] The server stores the received measurement data in a database. The input is the transmitted data (body fat percentage and skin temperature), and the output is the record stored in the database. Specifically, the server uses database software (e.g., MySQL) to store the received data in a specific table.

[1330] Step 4:

[1331] The server sends the saved measurement data to the generation AI for analysis. The input is the saved measurement data and past health data, and the output is the analysis results. Specifically, the server sends the data to a generation AI model (e.g., GPT-4) and performs the analysis using the prompt, "Analyze the user's body fat percentage of 20% and skin temperature of 36.5 degrees, and generate appropriate feedback and health management advice."

[1332] Step 5:

[1333] The server then sends the feedback and advice generated by the generating AI back to the device. The input is the feedback and advice as the analysis results, and the output is the data sent to the user's device. Specifically, the server organizes the information received from the generating AI and sends advice such as, "Your body fat percentage is high. Please review your diet" to the user's smartphone or tablet.

[1334] Step 6:

[1335] The feedback and advice received by the user's device is visually displayed. The input is the feedback and advice sent from the server, and the output is the visual display content. Specifically, the smartphone application displays on the screen "Body fat percentage 20%, skin temperature 36.5°C" along with the advice "Your body fat percentage is high. Please review your diet."

[1336] This process allows users to understand their own health status in real time and receive appropriate advice.

[1337] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1338] This invention combines an IoT mirror system designed to allow users to easily manage their health in their daily lives with an emotion engine that recognizes the user's emotions. The system's main components include a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to a server, a means in the server for transmitting this data to a generation AI for analysis, a means for transmitting feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, and an emotion engine that recognizes the user's emotions.

[1339] System Components and Functions

[1340] Sensor Means

[1341] The sensor means is built into the IoT mirror and measures the user's body fat percentage and skin temperature. For example, when a user stands in front of the mirror in the bathroom in the morning, the sensor automatically collects data on the user's body fat percentage and skin temperature.

[1342] communication means

[1343] The measured data is sent to the server in real time via communication means, and the data is automatically saved on the server without the user having to check the data directly.

[1344] Server and Generating AI

[1345] The server stores the received data in a database and sends it to the generation AI, which analyzes the user's past health data and newly received data to evaluate the user's health status and generate specific advice.

[1346] Emotion Engine

[1347] The emotion engine recognizes emotions by analyzing the user's facial expressions and tone of voice. For example, when a user stands in front of a mirror checking their body fat percentage or skin temperature, it can determine whether the user is happy or stressed.

[1348] Generate and send feedback and advice

[1349] The generative AI generates feedback and advice appropriate for the user based on the analysis results. Meanwhile, the emotion engine appropriately modifies the feedback and advice provided by the generative AI based on the user's emotions as recognized. For example, if the user is feeling stressed, the advice may be adjusted to include relaxation techniques.

[1350] Terminals and Visual Displays

[1351] The device will visually display this feedback and advice to the user. For example, when the user stands in front of the mirror again, the mirror will display their current body fat percentage and skin temperature, along with tailored advice provided by the generative AI and emotion engine.

[1352] Specific examples

[1353] Below are some concrete examples of how this system works in a user's daily life:

[1354] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[1355] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[1356] 3. The measurement data is sent to the server via a communication means.

[1357] 4. The server stores this data and sends it to the generating AI.

[1358] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[1359] 6. The emotion engine analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[1360] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion engine, generating adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[1361] 8. Feedback and advice is sent to the device through the server.

[1362] 9. When the user stands in front of the mirror again, the mirror will display "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[1363] In this way, the present invention helps users manage their health on a daily basis, and by combining it with the emotion engine's recognition of the user's emotions, it provides more personalized feedback and supports users in improving their lifestyle habits.

[1364] The processing flow will be explained below.

[1365] Step 1:

[1366] A user stands in front of the IoT mirror in the bathroom in the morning.

[1367] Step 2:

[1368] The sensor on the device (IoT mirror) detects the user's presence and measures their body fat percentage and skin temperature.

[1369] Step 3:

[1370] The device sends the measured data (e.g., body fat percentage 20%, skin temperature 36.5 degrees) to the server via communication means.

[1371] Step 4:

[1372] The server stores the received measurement data in a database and also sends the measurement data to the generation AI.

[1373] Step 5:

[1374] The generating AI analyzes the received data and the user's past health data to assess the user's health condition.

[1375] Step 6:

[1376] Based on the analysis results, the generative AI generates feedback and advice appropriate for the user.

[1377] For example: "Your body fat percentage has increased recently. You might consider increasing your exercise."

[1378] Step 7:

[1379] The emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

[1380] Step 8:

[1381] The server sends the analysis results of the emotion engine to the generation AI.

[1382] Step 9:

[1383] The generative AI adjusts feedback and advice based on the analysis results of the emotion engine.

[1384] For example: If the user is feeling stressed, "I recommend increasing your exercise and taking time to relax."

[1385] Step 10:

[1386] The generated AI sends the generated feedback and advice to the server.

[1387] Step 11:

[1388] The server sends the feedback and advice received from the generated AI to the terminal.

[1389] Step 12:

[1390] When the user stands in front of the mirror again, the device visually displays the feedback and advice received from the server.

[1391] Example: The mirror displays "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax."

[1392] Step 13:

[1393] Users can review the feedback and advice provided and take action to improve their lifestyle if necessary.

[1394] Example: A user decides to incorporate meditation time into their morning jog.

[1395] Example 2

[1396] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1397] Conventional health management systems have difficulty providing personalized feedback and advice because they cannot consider how data such as a user's body fat percentage and skin temperature correlates with the user's emotions and stress levels. Furthermore, even if a user is feeling stressed or frustrated, they cannot generate advice that takes this into account, preventing effective health management.

[1398] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI and analyzing it, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion recognition means for analyzing the user's facial expression and tone of voice to recognize their emotions, and a means for appropriately correcting the feedback and advice provided by the generation AI based on the results of the emotion recognition means. This enables personalized health management that takes the user's emotions into consideration.

[1399] "Sensor means" refers to a device that measures the user's body fat percentage and skin temperature.

[1400] "Communication means" refers to an interface for transmitting measurement data to a server.

[1401] "Generative AI" refers to artificial intelligence that analyzes measurement data on the server and generates feedback and advice.

[1402] "Emotion recognition means" refers to technology that recognizes emotions by analyzing a user's facial expressions and tone of voice.

[1403] "Means for sending feedback and advice back to the terminal" refers to a communication means for sending the feedback and advice generated by the generating AI to the terminal.

[1404] "Means for visually displaying feedback and advice at a terminal" refers to a display function on a terminal for visually displaying feedback and advice to a user.

[1405] "Means for appropriately modifying the feedback and advice provided by the generating AI based on the results of the emotion recognition means" refers to an algorithm that enables the generating AI to modify its feedback and advice in consideration of the user's emotional data obtained by the emotion recognition means.

[1406] The present invention relates to a system that allows users to easily manage their health on a daily basis, and is characterized in that it provides personalized feedback and advice taking into account the user's emotions. Specific implementation methods for this system are described below.

[1407] Sensor Means

[1408] When a user stands in front of the mirror, the sensors built into the IoT mirror automatically measure the user's body fat percentage and skin temperature. The sensors include a bioimpedance sensor to measure body fat percentage and a thermistor to measure skin temperature.

[1409] communication means

[1410] The measurement data is sent to the server via a communication means such as a Wi-Fi module or Bluetooth module, eliminating the need for users to manually input the measurement data.

[1411] Server and Generating AI

[1412] The server stores the received data in a database. The stored data is then sent to the Generator AI for analysis. This Generator AI uses advanced artificial intelligence, such as GPT-4. This Generator AI analyzes past health data and newly received data to assess the user's health status and generate specific feedback and advice.

[1413] emotion recognition means

[1414] The mirror uses deep learning technology to recognize emotions by analyzing the user's facial expressions and tone of voice. A camera and microphone attached to the mirror capture the user's facial expressions and voice for analysis.

[1415] Generate feedback and advice

[1416] The AI ​​then generates feedback and advice for the user based on the analysis results. Furthermore, the emotion recognition system takes the user's emotions into account and modifies the AI's feedback and advice accordingly. For example, if the user is feeling stressed, the AI ​​may add relaxation techniques.

[1417] Viewing feedback and advice

[1418] The generated feedback and advice is sent to the device via the server, and when the user stands in front of the mirror again, the adjusted feedback and advice is displayed on the mirror along with the current values ​​of body fat percentage and skin temperature.

[1419] Specific examples

[1420] Below are some concrete examples of how this system works in a user's daily life:

[1421] 1. A user stands in front of the IoT mirror in the bathroom in the morning.

[1422] 2. The sensor means measures a body fat percentage of 20% and a skin temperature of 36.5 degrees.

[1423] 3. The measurement data is sent to the server via the communication means.

[1424] 4. The server stores this data and sends it to the generating AI.

[1425] 5. The generative AI analyzes the data and generates feedback and advice such as, "Your body fat percentage has increased recently. Consider increasing your exercise."

[1426] 6. The emotion recognition means analyzes the user's facial expressions and voice and recognizes when the user is feeling stressed.

[1427] 7. The generative AI modifies the feedback and advice based on the analysis results of the emotion recognition means, and generates adjusted advice such as, "We recommend increasing your exercise but also taking time to relax."

[1428] 8. Feedback and advice is sent to the device through the server.

[1429] 9. When the user stands in front of the mirror again, the mirror displays "Body fat percentage 20%, skin temperature 36.5°C" and "We recommend increasing your exercise and taking time to relax."

[1430] Prompt Sentence Examples

[1431] Below are some example prompts to be input to the generative AI model:

[1432] The user's body fat percentage has increased recently, so you would like to advise the user to increase their exercise. Additionally, the user is feeling stressed, so you would like to encourage them to take time to relax.

[1433] As a result, the present invention can provide personalized feedback and advice that takes into account the user's emotions, allowing the user to manage their health more effectively.

[1434] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1435] Step 1:

[1436] A user stands in front of the IoT mirror in the bathroom in the morning. Specifically, the user simply stands in front of the mirror to wash their face, and the system automatically activates. The input is the user's presence, and the output is the trigger that activates the sensor.

[1437] Step 2:

[1438] The sensor means measures body fat percentage and skin temperature. Specifically, a bioimpedance sensor mounted on the mirror measures body fat percentage, and a thermistor measures skin temperature. The input is the user's body fat percentage and skin temperature, and the output is these measurement data.

[1439] Step 3:

[1440] The device sends the measurement data to the server. Specifically, the communication means (Wi-Fi module) sends the measurement data (e.g., body fat percentage 20%, skin temperature 36.5°C) to the server in real time. The input is the measurement data, and the output is the data sent to the server.

[1441] Step 4:

[1442] The server stores the received data in a database. Specifically, the server executes a procedure to store the data in a database such as MySQL. The input is the transmitted measurement data, and the output is the stored data.

[1443] Step 5:

[1444] The server sends data to the generation AI. Specifically, the server sends data to the generation AI (e.g., GPT-4) via an API. The input is the stored measurement data, and the output is the data sent to the generation AI.

[1445] Step 6:

[1446] The generating AI analyzes the received data and generates feedback and advice. Specifically, the generating AI analyzes the user's past health data and newly received data. The input is measurement data and past health data, and the output is feedback and advice such as, "Your body fat percentage has been increasing recently. Please consider increasing your exercise."

[1447] Step 7:

[1448] The emotion recognition means recognizes the user's emotions. Specifically, a camera and microphone attached to the mirror capture the user's facial expressions and tone of voice, which are then analyzed by a deep learning model. The input is the user's facial expression and voice data, and the output is the user's emotional data.

[1449] Step 8:

[1450] The generative AI modifies the feedback and advice based on the results of the emotion recognition means. Specifically, the generative AI takes the emotion data into account and modifies the advice to "Increase your exercise and take time to relax." The inputs are the feedback, advice, and emotion data, and the output is the modified feedback and advice.

[1451] Step 9:

[1452] The server sends the feedback and advice to the terminal. As a specific operation, the feedback and advice are sent to the terminal again through the communication means. The input is the modified feedback and advice, and the output is the data sent to the terminal.

[1453] Step 10:

[1454] The device visually displays feedback and advice. Specifically, when the user stands in front of the mirror again, the mirror displays the message "Body fat percentage 20%, skin temperature 36.5°C" along with "We recommend increasing your exercise but also taking time to relax." The input is the data sent to the device, and the output is the feedback and advice displayed on the mirror.

[1455] (Application example 2)

[1456] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1457] In autonomous vehicles, safety and comfort are not sufficiently ensured because the driver's health and emotional state are not monitored in real time. In addition, it is difficult to provide appropriate advice based on the driver's health and emotional state, which may result in the driver's fatigue and stress being overlooked.

[1458] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1459] In this invention, the server includes a sensor means for measuring the user's body fat percentage and skin temperature, a communication means for transmitting the measurement data to the server, a means in the server for transmitting the measurement data to the generation AI for analysis, a means for transmitting the feedback and advice generated by the generation AI back to the terminal, a means for visually displaying the feedback and advice on the terminal, an emotion analysis means for analyzing the driver's facial expression and tone of voice to recognize the emotional state, and a means for appropriately correcting the feedback and advice by the generation AI based on the emotion analysis results. This makes it possible to manage the driver's health and emotional state in real time in an autonomous vehicle and provide appropriate feedback and advice.

[1460] A "user" is a person who utilizes the system to receive feedback on their health and emotional state.

[1461] "Body fat percentage" is a numerical value that indicates the percentage of fat present in the user's body.

[1462] "Skin temperature" is a numerical value measuring the surface temperature of the user's skin.

[1463] "Sensor means" refers to devices and techniques for measuring body fat percentage and skin temperature.

[1464] "Communication means" refers to the communication protocol and device for transmitting the measured data to the server.

[1465] "Generative AI" refers to artificial intelligence technology that analyzes past health data and newly received data to generate appropriate feedback and advice.

[1466] "Emotion analysis means" refers to technology and devices that analyze facial expressions and tone of voice to recognize the user's emotional state.

[1467] "Terminal" refers to a device for visually displaying feedback and advice to a user.

[1468] "Feedback and advice" refers to specific instructions and advice provided to the user based on the results of the generative AI's analysis.

[1469] "Emotion analysis result" refers to the analysis result of the user's emotional state obtained by the emotion analysis means.

[1470] "Cloud server" refers to the computer system that stores the measured data and on which the generative AI and emotion analysis means run.

[1471] The present invention relates to a system for providing real-time monitoring of the driver's well-being and emotional state in an autonomous vehicle. The system includes the following main components:

[1472] Sensor Means

[1473] Health status data collection

[1474] The sensor means is installed in the autonomous vehicle and includes a device for measuring body fat percentage and skin temperature. The body fat percentage sensor and skin temperature sensor measure the driver's health data in real time and acquire the values.

[1475] communication means

[1476] Data transmission

[1477] The acquired body fat percentage and skin temperature data is sent in real time to a cloud server via a communication module in the vehicle (e.g., Bluetooth, Wi-Fi), eliminating the need for the driver to check the measurement data directly.

[1478] Server and Generating AI

[1479] Data analysis

[1480] The cloud server stores the received body fat percentage and skin temperature data and sends it to the generative AI model (e.g., GPT-4), which analyzes this data, evaluates the driver's health status, and generates specific feedback and advice.

[1481] Emotion analysis means

[1482] emotion recognition

[1483] Emotion analysis refers to technology for analyzing the driver's facial expressions and tone of voice. Driver information is collected using the smartphone's camera and microphone, and analyzed by an emotion analysis engine (e.g., Emotion AI SDK). This allows the driver's emotional state (e.g., stress, satisfaction) to be recognized.

[1484] Generate and send feedback and advice

[1485] Feedback Adjustment

[1486] The generative AI will then adjust the feedback and advice appropriately based on the emotional data obtained from the emotion analysis means. For example, if the emotion analysis determines that the driver is feeling stressed, the AI ​​will adjust the advice by adding relaxation techniques.

[1487] Terminals and Visual Displays

[1488] Information presentation

[1489] Final feedback and advice is provided to the driver via a smartphone application, which displays the driver's current body fat percentage and skin temperature, as well as tailored feedback and advice provided by the AI.

[1490] Specific examples

[1491] Below is a concrete example of how this system works within a self-driving vehicle.

[1492] 1. The driver enters the vehicle and the sensors measure his body fat percentage at 19% and his skin temperature at 36.0°C.

[1493] 2. Measurement data is sent in real time to a cloud server via a smartphone.

[1494] 3. The AI ​​analyzes the data stored on the cloud server and generates feedback such as, "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break."

[1495] 4. The emotion analysis engine recognizes fatigue from the driver's facial expression, and the generative AI adjusts the feedback to, "It would be a good idea to exercise and drink a warm drink."

[1496] 5. Calibrated feedback will be displayed on your smartphone.

[1497] Prompt Sentence Examples

[1498] User Health Data:

[1499] Body fat percentage: 19%

[1500] Skin temperature: 36.0 degrees

[1501] User's emotional state: Fatigue

[1502] Generate feedback and advice:

[1503] Your body fat percentage is normal, but your skin temperature may be low. We recommend taking a break.

[1504] Exercise and drink warm drinks.

[1505] As described above, this invention enables drivers to manage their health and monitor their emotional state while in an autonomous vehicle, and provides appropriate feedback and advice, which is expected to significantly improve driving safety and comfort.

[1506] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1507] Step 1:

[1508] Health data collection

[1509] When a user sits in the autonomous vehicle, the sensor measures their body fat percentage and skin temperature, and the measured data is sent to a smartphone.

[1510] Input: User's body fat percentage and skin temperature

[1511] Output: Measurement data (e.g., body fat percentage 19%, skin temperature 36.0°C)

[1512] Step 2:

[1513] Data transmission

[1514] The device (smartphone) transmits the collected body fat percentage and skin temperature data to a cloud server in real time, using Bluetooth and Wi-Fi as the communication method.

[1515] Input: Measurement data (body fat percentage 19%, skin temperature 36.0°C)

[1516] Output: Data sent to the cloud server

[1517] Step 3:

[1518] Data storage and analysis preparation

[1519] The server stores the received data in a database and prepares to send the data to the generation AI.

[1520] Input: Data sent to the cloud server

[1521] Output: Data stored in the database, input data to the generative AI

[1522] Step 4:

[1523] Analysis by generative AI

[1524] The AI ​​analyzes the body fat percentage and skin temperature data sent from the server, as well as past health data, to assess the user's current health condition. Based on the analysis results, it generates specific feedback and advice.

[1525] Input: Data input to the AI ​​(body fat percentage 19%, skin temperature 36.0°C)

[1526] Output: Generated feedback and advice (e.g., "Your body fat percentage is normal, but your skin temperature may be low. We recommend that you take a break.")

[1527] Step 5:

[1528] Emotion recognition using emotion analysis methods

[1529] The smartphone's camera and microphone are used to analyze the user's facial expressions and tone of voice, and an emotion analysis engine (e.g., Emotion AI SDK) recognizes the user's emotional state.

[1530] Input: facial expression data and tone of voice

[1531] Output: Sentiment analysis result (e.g., fatigue)

[1532] Step 6:

[1533] Feedback Adjustment

[1534] The generative AI then adjusts the feedback and advice appropriately based on the emotional data obtained from the emotion analysis tool. For example, if the user is feeling stressed, it will add relaxation techniques.

[1535] Input: Generated feedback and advice, sentiment analysis results

[1536] Output: Tailored feedback and advice (e.g., "You should exercise and drink warm drinks.")

[1537] Step 7:

[1538] Sending feedback and suggestions

[1539] The server sends tailored feedback and advice to the terminal.

[1540] Input: Calibrated feedback and advice

[1541] Output: Feedback and advice sent to the device

[1542] Step 8:

[1543] Information presentation

[1544] The device (smartphone) visually displays tailored feedback and advice to the driver.

[1545] Input: Feedback and advice sent to the device

[1546] Output: Feedback and advice displayed on the smartphone (e.g., "It would be good for you to exercise and drink warm drinks.")

[1547] The above are the specific processing steps of the system that realizes the application example.

[1548] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1549] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1550] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1551] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1552] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1553] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1554] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1555] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1556] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1557] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1558] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1559] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1560] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1561] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1562] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1563] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1564] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1565] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1566] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1567] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1568] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1569] The following is further disclosed regarding the above embodiment.

[1570] (Claim 1)

[1571] a sensor means for measuring a user's body fat percentage and skin temperature;

[1572] a communication means for transmitting the measurement data to a server;

[1573] A means for transmitting the measurement data to the generation AI in the server and analyzing the data;

[1574] A means for transmitting the feedback and advice generated by the generating AI back to the terminal;

[1575] means for visually displaying feedback and advice at the terminal;

[1576] A system including:

[1577] (Claim 2)

[1578] 2. The system of claim 1, further comprising sensor means for measuring body fat percentage and skin temperature in real time.

[1579] (Claim 3)

[1580] The system of claim 1, wherein the generating AI has means for analyzing the user's current health condition based on past health data and generating personalized health advice.

[1581] "Example 1"

[1582] (Claim 1)

[1583] a sensor means for measuring a user's body fat percentage and skin temperature;

[1584] a communication means for transmitting the measurement data to a server;

[1585] A means for transmitting the measurement data to the generation AI in the server and analyzing the data;

[1586] A means for transmitting the feedback and advice generated by the generating AI back to the terminal;

[1587] means for visually displaying feedback and advice at the terminal;

[1588] means for automatically transmitting a plurality of pieces of data measured by the sensor means to a server;

[1589] A means to use generative AI to assess the user's health status and generate advice based on past and current data;

[1590] A system including:

[1591] (Claim 2)

[1592] 2. The system of claim 1, further comprising sensor means for measuring body fat percentage and skin temperature in real time.

[1593] (Claim 3)

[1594] The system of claim 1, wherein the generating AI has means for analyzing the user's current health condition based on past health data and generating personalized health advice.

[1595] (Claim 4)

[1596] 10. The system of claim 1, wherein the server has means for storing the received data in a database and formatting it for analysis.

[1597] (Claim 5)

[1598] 10. The system of claim 1, further comprising means for generating advice using a specific prompt sentence when sending it to the generative AI model.

[1599] (Claim 6)

[1600] 2. The system of claim 1, wherein the terminal comprises display means for visually displaying feedback and advice when the user stands in front of the mirror again.

[1601] "Application Example 1"

[1602] (Claim 1)

[1603] a sensor means for measuring a user's body fat percentage and skin temperature;

[1604] a communication means for transmitting the measurement data to a server;

[1605] A means for transmitting the measurement data to the generation AI in the server and analyzing the data;

[1606] A means for transmitting the feedback and advice generated by the generating AI back to the terminal;

[1607] means for visually displaying feedback and advice at the terminal;

[1608] A means to measure members' health data at fitness facilities and provide feedback generated by AI in real time.

[1609] A system including:

[1610] (Claim 2)

[1611] 2. The system of claim 1, further comprising sensor means for measuring body fat percentage and skin temperature in real time.

[1612] (Claim 3)

[1613] The system of claim 1, wherein the generating AI has means for analyzing the user's current health condition based on past health data and generating personalized health advice.

[1614] "Example 2: Combining Emotion Engines"

[1615] (Claim 1)

[1616] a sensor means for measuring a user's body fat percentage and skin temperature;

[1617] a communication means for transmitting the measurement data to a server;

[1618] A means for transmitting the measurement data to the generation AI in the server and analyzing the data;

[1619] A means for transmitting the feedback and advice generated by the generating AI back to the terminal;

[1620] means for visually displaying feedback and advice at the terminal;

[1621] An emotion recognition means for recognizing emotions by analyzing the user's facial expressions and tone of voice;

[1622] A means for appropriately modifying the feedback and advice provided by the generation AI based on the results of the emotion recognition means;

[1623] A system including:

[1624] (Claim 2)

[1625] 2. The system of claim 1, further comprising sensor means for measuring body fat percentage and skin temperature in real time.

[1626] (Claim 3)

[1627] The system of claim 1, wherein the generating AI has a means for analyzing the user's current health condition based on past health data and the user's emotional data and generating personalized health advice.

[1628] "Application example 2 when combining emotion engines"

[1629] (Claim 1)

[1630] a sensor means for measuring a user's body fat percentage and skin temperature;

[1631] a communication means for transmitting the measurement data to a server;

[1632] A means for transmitting the measurement data to the generation AI in the server and analyzing the data;

[1633] A means for transmitting the feedback and advice generated by the generating AI back to the terminal;

[1634] means for visually displaying feedback and advice at the terminal;

[1635] An emotion analysis means for analyzing the driver's facial expression and tone of voice to recognize the driver's emotional state;

[1636] A means for appropriately modifying the feedback and advice provided by the generative AI based on the results of the sentiment analysis; and

[1637] A system including:

[1638] (Claim 2)

[1639] 2. The system of claim 1, further comprising sensor means for measuring body fat percentage and skin temperature in real time.

[1640] (Claim 3)

[1641] The system of claim 1, wherein the generating AI has means for analyzing the user's current health condition based on past health data and generating personalized health advice.

[1642] (Claim 4)

[1643] 2. The system according to claim 1, wherein the emotion analysis means analyzes the driver's facial expression and tone of voice in real time.

[1644] (Claim 5)

[1645] The system of claim 1, wherein the generating AI has means for appropriately modifying the feedback and advice taking into account the analysis results of the emotion analysis means. [Explanation of symbols]

[1646] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a sensor means for measuring a user's body fat percentage and skin temperature; a communication means for transmitting the measurement data to a server; A means for transmitting the measurement data to the generation AI in the server and analyzing the data; A means for transmitting the feedback and advice generated by the generating AI back to the terminal; means for visually displaying feedback and advice at the terminal; A system including:

2. 2. The system of claim 1, further comprising sensor means for measuring body fat percentage and skin temperature in real time.

3. The system of claim 1, wherein the generating AI has means for analyzing the user's current health condition based on past health data and generating personalized health advice.

Images