System

Abstract

A system is provided.SOLUTION: A system includes a means for receiving health data from a user, a means for answering a health consultation from the user using a generated AI model, a database means for storing the health data and a history of the health consultation, a means for making a detailed consultation or an online medical examination with an expert, and a means for automatically collecting health data in cooperation with an IoT device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current health management applications only allow users to access limited functionality for daily health and medical consultations, and certain features require subscriptions. This makes it difficult to attract new users, and they do not adequately meet the needs of users who are constantly concerned about their health or who are considering improving their physical condition through health management. Furthermore, many users rely on free online information to manage their health on their own judgment. It is necessary to resolve these issues with the current system, increase the number of new app users, and encourage them to convert to paid membership. [Means for solving the problem]

[0005] The present invention proposes a system that receives health data from users and provides answers to health consultations using a generative AI model. Specifically, the following means are provided to solve the problems.

[0006] 1. Provide a means to receive health data from users, allowing them to easily input their daily health information into the app.

[0007] 2. The system uses a generative AI model to generate appropriate answers to health inquiries from users, allowing users to receive simple health consultations free of charge on a daily basis.

[0008] 3. A database will be established to store health data and health consultation history, allowing users' health information to be managed centrally and utilized for ongoing health management.

[0009] 4. We will provide a means to schedule and conduct detailed consultations with specialists and online medical consultations, thereby creating an environment where people can receive specialized medical support when needed.

[0010] 5. Equipping the app with a means to automatically collect health data in conjunction with IoT devices, thereby enabling users to obtain accurate health information in real time and improving the usability of the app.

[0011] These measures allow users to receive everything from daily health management to specialized consultations and medical treatment in one place, which can encourage app usage and ultimately lead to subscriptions.

[0012] "Health data" refers to information about a user's physical condition and lifestyle habits, including measurements and records of weight, number of steps, diet, sleep time, blood pressure, etc.

[0013] A "generative AI model" refers to an artificial intelligence technology that automatically generates appropriate answers to health inquiries from users, and is primarily achieved using natural language processing technology.

[0014] "Database" refers to an electronic data storage device or system for comprehensively storing and managing health data and health consultation history.

[0015] "Expert" refers to a person who is qualified in a medical or health-related field, such as a doctor, nutritionist, or trainer, and can provide professional advice to users.

[0016] "Online medical consultation" refers to medical treatment conducted between a specialist and a user via the internet using communication methods such as video calls.

[0017] "IoT devices" refer to devices that are connected to the Internet and automatically collect and transmit health data, such as smartwatches and blood pressure monitors.

[0018] "Subscription" refers to a contractual arrangement in which a user can continue to use a particular service by paying a periodic fee.

[0019] "Free Health Consultation" refers to a feature that uses generative AI models to allow users to get answers to everyday health questions free of charge. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention is a health management system that provides free health consultations using generative AI to assist users in managing their health. The system collects input data from users, uses generative AI models to answer health questions, and even provides in-depth consultations with experts and online medical consultations.

[0042] System configuration

[0043] Server Features

[0044] 1. Database Management

[0045] The server receives health data entered by the user (e.g., weight, number of steps, dietary content, sleep time, etc.) and stores this data in a centrally managed database.

[0046] 2. Operation of generative AI

[0047] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[0048] 3. Collaboration with experts

[0049] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[0050] 4. IoT device data collection

[0051] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[0052] Device Features

[0053] 1. Data input interface

[0054] The device provides an interface where users can input their daily health data, which is then sent to a server and stored in a database.

[0055] 2. Health consultation chat function

[0056] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[0057] 3. IoT device integration

[0058] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[0059] User operations

[0060] 1. Initial Setup

[0061] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.).

[0062] 2. Data entry and consultation

[0063] Users can input their daily health data and ask the AI ​​for health advice, such as, "How balanced is your diet today?" or "What kind of exercise should I do?"

[0064] 3. Consult with a specialist and book an online consultation

[0065] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[0066] 4. Subscription

[0067] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[0068] Specific examples

[0069] 1. Example of initial settings

[0070] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[0071] 2. Examples of daily data entry and consultation

[0072] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[0073] 3. Example of collaboration with IoT devices

[0074] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[0075] 4. Consultations with specialists and examples of treatment

[0076] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[0077] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[0078] The processing flow will be explained below.

[0079] Step 1: Initial Setup

[0080] 1. The server monitors the installation of the application and accepts a connection request to the server when the application is first launched.

[0081] 2. The device displays an account creation screen to the user, prompting them to enter basic information such as their name, email address, and password.

[0082] 3. The device sends the entered basic information to the server.

[0083] 4. The server stores the received basic information in a database and creates an account.

[0084] 5. The server sends a notification to the device that the account has been created.

[0085] 6. The device will display a notification to the user that the account has been created.

[0086] Step 2: Daily entry of health data

[0087] 1. The user launches the app and enters daily health data such as weight, number of steps, diet, and sleep time.

[0088] 2. The device sends the entered health data to the server.

[0089] 3. The server stores the received health data in a database.

[0090] 4. The device will notify the user that the data has been entered and allow them to visually check the data in a graph.

[0091] Step 3: IoT device integration

[0092] 1. The user connects an IoT device such as a smartwatch or blood pressure monitor to the app.

[0093] 2. The device automatically collects health data from connected IoT devices.

[0094] 3. The device sends the collected data to the server.

[0095] 4. The server stores the received data in a database.

[0096] 5. The device notifies the user that the latest health data has been updated.

[0097] Step 4: Free health consultation

[0098] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[0099] 2. The device sends the user's question to the server.

[0100] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[0101] 4. The server sends the generated response to the terminal.

[0102] 5. The device displays the answer from the generating AI to the user.

[0103] Step 5: Consult with a specialist and online consultation

[0104] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[0105] 2. The device sends the user's request to the server.

[0106] 3. The server queries the expert's schedule to check available dates and times.

[0107] 4. The server sends the confirmed schedule information to the terminal.

[0108] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[0109] 6. The user selects the desired date and time, and the device sends the selection to the server.

[0110] 7. The server finalizes the booking and notifies the user and the expert.

[0111] 8. The user launches the app at the scheduled time and begins the online consultation.

[0112] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[0113] 10. The server records the medical treatment details and advice in a database.

[0114] Step 6: Subscribe

[0115] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[0116] 2. The device will display instructions to the user via push notification or email.

[0117] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[0118] 4. The terminal sends the entered payment information to the server.

[0119] 5. The server processes the payment information and updates the subscription status to paid.

[0120] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[0121] Through these steps, the system efficiently supports users in managing their health, providing a comprehensive range of services from free consultations to professional advice and medical treatment.

[0122] Example 1

[0123] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0124] In modern society, personal health management is an important topic, and many users need to understand their own health status and receive appropriate advice. However, collecting health data on a daily basis and utilizing it to receive effective advice requires specialized knowledge and expensive equipment, placing a heavy burden on the average user. In addition, there is a lack of systems that allow seamless in-depth consultations with experts or online medical consultations. This creates a challenge for users, preventing them from optimally managing their health.

[0125] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0126] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and conducting detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, and means for the health data to serve as the basis for analysis and recommendations based on the generative AI model, thereby enabling users to understand their own health status and receive appropriate advice without the need for specialized knowledge or expensive equipment.

[0127] "Health data from the user" refers to health-related data such as weight, number of steps, dietary details, and sleep time entered by the user.

[0128] A "generative AI model" is an artificial intelligence model that generates appropriate answers to health inquiries from users.

[0129] "Health consultation history" is a record of past health-related questions and answers asked by the user.

[0130] "Database" means an information management system for storing health data and health consultation history.

[0131] "Detailed consultations and online medical treatment with experts" refers to specific health consultations and remote medical treatments that users receive from experts.

[0132] An "IoT device" is a device, such as a smartwatch or blood pressure monitor, that is connected to the Internet and collects health data.

[0133] "Analysis and recommendation" is the process of analyzing collected health data and providing appropriate health advice to the user.

[0134] "Subscription status" is information that indicates the user's subscription service usage status and expiration date.

[0135] "Real-time communication means" refers to communication technologies that allow users to ask questions using generative AI models and receive instant answers.

[0136] The present invention is a health consultation system that utilizes a generative AI model to support users' health management. This system functions in cooperation with users, terminals, and a server.

[0137] Hardware and software used

[0138] 1. Hardware

[0139] Device: Smartphone or tablet

[0140] Server: High-performance cloud server

[0141] IoT devices: smartwatches, blood pressure monitors, etc.

[0142] 2. Software

[0143] Application: Health management app

[0144] Database: MySQL (registered trademark)

[0145] Generative AI model: GPT-4(registered trademark) by OpenAI(registered trademark)

[0146] Scheduling Software: Professional scheduling system

[0147] Communication protocol: REST API

[0148] Program processing

[0149] Server Features

[0150] The server receives health data sent by users and stores it in a database. Specifically, it uses a REST API to collect data such as the user's weight, number of steps, dietary habits, and sleep time. This data is stored in a MySQL database and managed centrally.

[0151] Furthermore, the server runs the generative AI model GPT-4, which generates appropriate answers to health inquiries from users. For example, if a user asks, "How is your diet balanced today?", GPT-4 will generate an answer such as, "Your diet today is high in carbohydrates. Eating a few more vegetables would improve the balance."

[0152] The server also accepts requests for in-depth consultations with specialists and online consultations, and uses dedicated scheduling software to manage the specialists' schedules and confirm appropriate appointments.

[0153] Device Features

[0154] The device provides an interface that allows users to input their daily health data. The input data is sent to a server in real time and stored in a database. It also provides an interface for chatting with the generating AI for health consultations. When a user accesses the chat screen and asks, "What kind of exercise should I do?", the answer from the generating AI is displayed instantly.

[0155] The device also connects to IoT devices such as smartwatches and blood pressure monitors, automatically collecting data from these devices and sending it to a server. For example, heart rate data obtained from a smartwatch is automatically sent to the server.

[0156] User operations

[0157] First, users download the app and create an account. They enter basic health information (age, weight, medical history, allergies, health goals, etc.). Next, they can enter their daily health data and ask the AI ​​for health advice.

[0158] Users can also make appointments for detailed consultations with specialists or online medical consultations, and receive specific health advice via video call at the scheduled time. After the free trial period ends, users can continue to receive detailed advice and online medical consultations by subscribing.

[0159] Specific examples

[0160] 1. Example of initial settings

[0161] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[0162] 2. Examples of daily data entry and consultation

[0163] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[0164] 3. Example of collaboration with IoT devices

[0165] When a user is wearing a smartwatch, the heart rate data acquired from the smartwatch is automatically sent to the device and stored on the server.

[0166] 4. Consultations with specialists and examples of treatment

[0167] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[0168] keyword

[0169] Generative AI model, prompt sentence

[0170] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0171] Step 1:

[0172] Initial Setup

[0173] The user installs and launches the health management app. Next, they select "New Registration" and enter basic health information such as their name, age, sex, weight, height, allergy information, and health goals. The entered information is sent from the device to the server, which receives it and stores it in a database.

[0174] Input: Name, age, gender, weight, height, allergy information, health goals

[0175] Output: User information is saved in the database

[0176] Step 2:

[0177] Routine data entry

[0178] Users enter their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the app. The data is sent in real time from the device to the server, which receives it and stores it in a database.

[0179] Input: weight, diet, steps, sleep time

[0180] Output: Health data is stored in a database

[0181] Step 3:

[0182] Using generative AI models for health consultations

[0183] A user uses the app's consultation function to ask a health question in chat format. For example, the user might ask, "How balanced is my diet today?" This question is sent from the device to the server. The server then invokes a generative AI model (GPT-4) to generate an answer to the question. The generated answer is sent to the user's device and displayed on the chat screen.

[0184] Input: User question (e.g., "How balanced is your diet today?")

[0185] Output: Answer from the generative AI model (e.g., "Your meal today is high in carbohydrates. Adding a few more vegetables would improve the balance.")

[0186] Step 4:

[0187] Book an in-depth consultation with a specialist or online consultation

[0188] A user uses the app's "Consultation with a Specialist" feature to book a consultation or consultation. For example, they request, "I'd like to book a consultation with a nutritionist this weekend at 3:00 PM." This request is sent to the server, which uses dedicated scheduling software to manage the specialist's schedule and confirm the appointment. The appointment details are sent to the user's device and notified.

[0189] Input: Appointment request (e.g., "I'd like to schedule a consultation with a nutritionist this weekend at 3pm")

[0190] Output: Confirmation of booking

[0191] Step 5:

[0192] Data linkage with IoT devices

[0193] When a user uses IoT devices such as a smartwatch or blood pressure monitor, health data is collected in real time from these devices. The data is transmitted to the device via Bluetooth or Wi-Fi. The device then sends the data to a server, which stores it in a database. For example, heart rate data from a smartwatch is automatically collected and recorded.

[0194] Input: Health data from IoT devices (e.g., heart rate, blood pressure)

[0195] Output: Health data is stored in a database

[0196] Step 6:

[0197] Manage your subscription

[0198] The server manages the user's subscription status. Before the free period ends, the server notifies the user and prompts them to subscribe. The user selects the desired plan within the app and enters their credit card information to complete the payment. The server verifies the payment information and updates the subscription status.

[0199] Input: Subscription request, credit card information

[0200] Output: Updated subscription status, payment confirmation notification

[0201] (Application example 1)

[0202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] In modern society, many health problems, such as lifestyle-related diseases and obesity, are on the rise. In particular, busy lifestyles make it difficult to choose healthy meals. Other issues include the inability to easily access health consultations and medical treatment from experts, and a lack of appropriate advice based on individual health data. In response to these issues, there is a demand for fast and accurate health management support.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0205] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, means for proposing health menus based on user data, and means for conducting health consultations with the user in a chat format. This allows users to receive dietary suggestions based on their own health information and easily consult with experts, thereby enabling them to achieve a healthy lifestyle.

[0206] The "means for receiving health data from the user" is an interface for sending and receiving health-related information (for example, weight, dietary content, amount of exercise, etc.) input by the user to the server.

[0207] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate answers to health-related questions asked by users.

[0208] The "database means for storing health data and health consultation history" is a database system for long-term storage and management of health data entered by users and the health consultation response history provided by the generation AI.

[0209] "Means for booking and conducting detailed consultations and online medical consultations with specialists" refers to a system that allows users to book and conduct detailed health consultations and online medical consultations with specialists.

[0210] "Means for automatically collecting health data in conjunction with IoT devices" refers to a function that works in conjunction with smartwatches and everyday health measurement devices to collect health data automatically obtained from them and send it to a server.

[0211] "Means for suggesting healthy menus based on user data" refers to a function in which AI uses health data provided by the user to generate and suggest healthy meal menus based on that data.

[0212] The "means for providing health consultations to users in chat format" is a system that allows users and generative AI models to exchange chat-style messages in real time and respond to health-related questions.

[0213] This invention is a system that supports users' health management, and aims to collect health data of individual users through collaboration with smartphones and IoT devices, and provide health consultations and dietary suggestions using a generative AI model. A detailed description of the system is provided below.

[0214] System configuration

[0215] This invention is mainly composed of three main elements: a server, a terminal, and a user.

[0216] Server Features

[0217] The server has the following main functions:

[0218] 1. Means of receiving health data from users:

[0219] The server receives basic health information (age, weight, health goals, allergy information, etc.) entered by the user through a smartphone application and stores it in a database.

[0220] 2. How to use generative AI models to answer health questions:

[0221] Leveraging OpenAI's ChatCompletion API, the system automatically generates answers to health-related questions from users. For example, if a user asks, "How balanced is your diet today?", the generative AI model instantly provides an answer.

[0222] 3. Database means for storing health data and health consultation history:

[0223] The server uses SQLite or another database management system to store the health data provided by the user and the history of answers provided by the generative AI model.

[0224] 4. How to book and conduct in-depth consultations with specialists and online consultations:

[0225] We provide a reservation system that has a function for linking with experts and allows users to consult with experts online, allowing users to easily obtain professional advice.

[0226] 5. Means of automatically collecting health data in conjunction with IoT devices:

[0227] It has the ability to automatically collect data from IoT devices such as smartwatches and blood pressure monitors, and send it to a server for storage.

[0228] 6. How to suggest healthy menus based on user data:

[0229] Based on the data provided by the user, the generative AI model will suggest healthy meals, for example, if the user's goal is to lose weight, it will provide a suitable meal plan.

[0230] 7. Means of providing health consultations to users through chat:

[0231] Users can chat with the generated AI model in real time via a smartphone app and ask for health advice.

[0232] Device Features

[0233] The device (smartphone, etc.) has the following functions:

[0234] 1. Data input interface:

[0235] It provides an interface that allows users to manually input their health data, making it easy to record daily weight, diet, exercise, etc.

[0236] 2. Health consultation chat function:

[0237] It provides an interface that allows users to chat with the generated AI and receive health advice.

[0238] 3. IoT device integration:

[0239] It works in conjunction with IoT devices such as smartwatches and blood pressure monitors, and has the ability to automatically collect data and send it to a server.

[0240] User operations

[0241] The user uses this system by following the steps below:

[0242] 1. Initial Setup:

[0243] Install the app and create an account, which starts by entering basic health information.

[0244] 2. Data entry and consultation:

[0245] Enter your daily health data and ask the AI ​​for health advice. For example, you can ask, "Could you please give me some advice about my recent diet?"

[0246] 3. Specialist consultations and online consultation appointments:

[0247] You can book consultations and appointments with specialists and receive professional advice via video call.

[0248] 4. Subscription:

[0249] After the free trial period ends, you can subscribe to continue using the service for detailed advice and online consultations.

[0250] Specific examples

[0251] Here is a concrete example of its use:

[0252] 1. Example of suggesting healthy menus based on user data:

[0253] After the user enters their health goals, they can ask, "How is your diet balanced today?" and the generative AI will suggest, "You're eating too many carbohydrates today. Try to eat more vegetables at your next meal."

[0254] 2. Example prompt:

[0255] User data: Age: 30, Weight: 70kg, Goal: Lose weight, Allergy: Peanuts

[0256] User Question: How balanced is your diet today?

[0257] Please provide your answer:

[0258] This allows users to understand their health status in real time and receive advice on appropriate diet and exercise.

[0259] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0260] Step 1: Initial Setup

[0261] Users download the smartphone app and create an account. The app prompts users to enter basic health information (age, weight, health goals, allergy information, etc.), which is then sent to a server and stored in a database.

[0262] Input: User's health information (age, weight, health goals, allergy information)

[0263] Data processing: Send input data to the server in JSON format

[0264] Output: User information is saved in the database

[0265] Step 2: Data entry

[0266] Users enter their daily health data (weight, diet, exercise, etc.) into the app, which then sends this data to a server and stores it in a database.

[0267] Input: User's daily health data (weight, diet, exercise)

[0268] Data processing: Send input data to the server in JSON format

[0269] Output: Daily health data is stored in a database

[0270] Step 3: IoT device integration

[0271] The app connects with users' IoT devices such as smartwatches and blood pressure monitors, allowing data collected from the devices to be automatically sent to the server.

[0272] Input: Health data from IoT devices (heart rate, steps, etc.)

[0273] Data processing: Automatically send data to the server

[0274] Output: Data from IoT devices is stored in a database

[0275] Step 4: Conducting health consultations

[0276] Users use the app's chat function to ask questions about their health. When a user enters a question, the question is sent to the server, which uses a generative AI model to generate an answer and sends it back to the user.

[0277] Input: User's health consultation question

[0278] Data computation: Generative AI models generate answers based on prompts

[0279] Output: The answer is returned to the user

[0280] Step 5: Healthy menu suggestions

[0281] When a user requests a healthy meal plan, they input their request into the app, and the server uses a generative AI model based on the user data to suggest an appropriate meal plan.

[0282] Input: User's health menu request

[0283] Data arithmetic: Generative AI models generate menus based on prompts

[0284] Output: A suggested meal menu is displayed to the user

[0285] Step 6: Book a consultation with a specialist

[0286] When a user wants to consult with a specialist, they make a reservation through the app. The server checks the specialist's schedule and confirms the reservation.

[0287] Input: User's consultation appointment request

[0288] Data processing: Matching booking requests with specialist schedules

[0289] Output: A confirmation of the booking is sent to the user.

[0290] Step 7: Manage your subscriptions

[0291] After the free trial period ends, users can subscribe to continue receiving detailed advice and treatment features. The app will manage users' subscription status and notify them when the trial period ends.

[0292] Input: Free period end information

[0293] Data Processing: Subscription Status Update

[0294] Output: A prompt to subscribe is sent to the user

[0295] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0296] This invention is a health management system that provides free health consultations utilizing generative AI and an emotion engine to assist users in managing their health. The system collects input data from users, uses a generative AI model and an emotion engine to answer health questions, and even provides in-depth consultations with experts and online medical treatments.

[0297] System configuration

[0298] Server Features

[0299] 1. Database Management

[0300] The server receives health data (e.g., weight, number of steps, dietary content, sleep time, etc.) and emotional data entered by the user and stores this data in a centrally managed database.

[0301] 2. Operation of generative AI

[0302] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[0303] 3. Operating the Emotion Engine

[0304] The server runs an emotion engine, analyzes the data input by the user and the content of the health consultation, and recognizes the user's emotional state. This recognized emotional data is also stored in the database.

[0305] 4. Collaboration with experts

[0306] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[0307] 5. IoT device data collection

[0308] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[0309] Device Features

[0310] 1. Data input interface

[0311] The device provides an interface through which users can input their daily health and emotional data, which is then sent to a server and stored in a database.

[0312] 2. Health consultation chat function

[0313] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[0314] 3. Emotion data input assistance

[0315] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends this data to a server.

[0316] 4. IoT device integration

[0317] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[0318] User operations

[0319] 1. Initial Setup

[0320] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.). The app also configures facial recognition and voice analysis settings, and sets up how to input emotional data.

[0321] 2. Data entry and consultation

[0322] Users input their daily health data and ask the AI ​​for health advice. For example, they can ask questions such as, "How balanced is my diet today?" or "What kind of exercise should I do?" The emotion engine analyzes the user's emotions and can suggest, for example, "How to relax if you're feeling stressed."

[0323] 3. Consult with a specialist and book an online consultation

[0324] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[0325] 4. Subscription

[0326] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[0327] Specific examples

[0328] 1. Example of initial settings

[0329] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. They then configure facial recognition and voice analysis. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg" to enable the facial recognition feature.

[0330] 2. Examples of daily data entry and consultation

[0331] The user enters their daily weight and dietary information and asks in chat, "How is your diet balance today?" The generative AI responds, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance." The emotion engine recognizes signs of stress from the user's facial expressions and voice and provides additional advice, such as "Try taking deep breaths to relax."

[0332] 3. Example of collaboration with IoT devices

[0333] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[0334] 4. Consultations with specialists and examples of treatment

[0335] A user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist contact each other via video call, and receive specific dietary advice. The user's emotional state is also provided to the nutritionist, enabling more personalized advice.

[0336] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[0337] The processing flow will be explained below.

[0338] Step 1: Initial Setup

[0339] 1. The user downloads and installs the app.

[0340] 2. The device will display the account creation screen when you first start it up.

[0341] 3. The user creates an account by entering their name, email address, password, age, weight, gender, and allergy information.

[0342] 4. The terminal sends the entered information to the server.

[0343] 5. The server stores the received information in its database and completes the account creation.

[0344] 6. The server sends a notification to the device that account creation is complete.

[0345] 7. The device displays a notification to the user that the account has been created.

[0346] Step 2: Configure facial recognition and voice analysis

[0347] 1. The device will display the initial setting screen for the emotion engine.

[0348] 2. The user configures the facial recognition and voice analysis settings (for example, by moving their face towards the camera or reading a specific phrase aloud).

[0349] 3. The device sends the settings to the server.

[0350] 4. The server saves the emotion engine configuration data in the database.

[0351] Step 3: Daily entry of health and emotional data

[0352] 1. The user launches the app and enters their daily health data, such as weight, number of steps, diet, and sleep time.

[0353] 2. The terminal sends the entered data to the server.

[0354] 3. The server stores the received data in a database.

[0355] 4. The device collects emotional data from the user's facial recognition and voice, and also sends this to the server.

[0356] 5. The server stores the emotion data in a database.

[0357] Step 4: Free health consultation

[0358] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[0359] 2. The device sends the user's question to the server.

[0360] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[0361] 4. The server sends the generated response to the terminal.

[0362] 5. The server starts the emotion engine and analyzes the user's emotion data.

[0363] 6. The server customizes the answer based on the sentiment data and generates appropriate follow-up advice.

[0364] 7. The server sends a customized answer and additional advice to the device.

[0365] 8. The device displays the user the answer from the generative AI and additional advice based on the emotional data.

[0366] Step 5: Consult with a specialist and online consultation

[0367] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[0368] 2. The device sends the user's request to the server.

[0369] 3. The server queries the expert's schedule to check available dates and times.

[0370] 4. The server sends the confirmed schedule information to the terminal.

[0371] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[0372] 6. The user selects the desired date and time, and the device sends the selection to the server.

[0373] 7. The server confirms the booking and notifies the user and the expert.

[0374] 8. The user launches the app at the scheduled time and begins the online consultation.

[0375] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[0376] 10. The server records the medical treatment details and advice in a database.

[0377] Step 6: Subscribe

[0378] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[0379] 2. The device will display instructions to the user via push notification or email.

[0380] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[0381] 4. The terminal sends the entered payment information to the server.

[0382] 5. The server processes the payment information and updates the subscription status to paid.

[0383] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[0384] Through these steps, the system efficiently supports users in managing their health and provides a comprehensive approach that includes emotional data. The emotion engine analyzes the user's emotional state and customizes the generative AI model's answers, resulting in more personalized healthcare services.

[0385] Example 2

[0386] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0387] Conventional health management systems have difficulty collecting not only users' health data but also their emotional data and providing appropriate advice based on that data. Furthermore, the procedures for making appointments for consultations with specialists or online medical consultations are often complicated, placing a burden on users. Furthermore, insufficient integration with IoT devices results in inefficient data collection. This makes it difficult for users to effectively manage their health on a daily basis.

[0388] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0389] In this invention, the server includes means for receiving health data and emotional data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing health data and health consultation history, means for analyzing the user's emotional state using an emotion engine, means for booking and implementing detailed consultations with experts and online medical treatments, and means for automatically collecting health data in cooperation with IoT devices. This enables comprehensive health management based on health data and emotional data, and smooth cooperation with experts reduces the burden on the user and enables effective health management.

[0390] "Health data from the user" refers to data relating to the user's daily health condition, such as weight, number of steps, dietary content, and sleep time, which the user inputs himself or herself.

[0391] "Emotional data" is data about a user's emotional state collected through facial recognition and voice analysis of the user.

[0392] A "generative AI model" is an artificial intelligence model that generates answers in natural language in response to health inquiries from users. An example of this would be a chatbot.

[0393] A "means for answering health inquiries" is a method or system that enables a generative AI model to provide appropriate answers to health inquiries from users.

[0394] The "database means" is a data management system for storing the user's health data and health consultation history.

[0395] An "emotion engine" is a technology that analyzes input data and dialogue content from users to recognize their emotional state.

[0396] "Means for booking and conducting in-depth consultations and online medical consultations with specialists" refers to systems and functions that allow users to book a date and time for a consultation or medical consultation with a specialist and to conduct that appointment online.

[0397] "Means for automatically collecting health data in cooperation with IoT devices" refers to a method or system for automatically acquiring health data from IoT devices such as smartwatches and blood pressure monitors and transmitting that data to a server.

[0398] The present invention is a health management system that provides a free health consultation function utilizing a generative AI model and an emotion engine to assist users in managing their health. The system collects health and emotion data from users, uses the generative AI model and the emotion engine to answer health questions, and also provides detailed consultations with experts and online medical treatments. The following describes in detail the embodiments of the present invention.

[0399] server

[0400] The server performs the following functions:

[0401] 1. Data Collection and Management

[0402] The server receives the health and emotion data sent from the device and stores them in a centralized database, which is managed using MySQL or PostgreSQL, for example.

[0403] 2. Operation of generative AI models

[0404] When a health consultation from a user arrives at the server, the server generates a request to a generative AI model (e.g., GPT-4) and creates and sends an appropriate prompt. For example, in response to the question, "How is your diet balance today?", the server generates a prompt such as, "Please rate the user's diet balance today."

[0405] 3. Operating the Emotion Engine

[0406] The server runs an emotion engine (e.g., Affectiva or Azure® Emotion Recognition API) that analyzes the user's input data and dialogue to recognize their emotional state. The recognized emotion data is also stored in a database.

[0407] 4. Scheduling with an expert

[0408] The server manages the schedules of the experts and confirms the reservation based on the reservation request from the user. The scheduling is done using the Google (registered trademark) Calendar API.

[0409] 5. IoT device data collection

[0410] The server connects to IoT devices (e.g., smartwatches and blood pressure monitors) and automatically collects health data from these devices and stores it in a database. In this case, Firebase and AWS (registered trademark) IoT are used.

[0411] Terminal

[0412] The terminal provides the following user interface:

[0413] 1. Data input interface

[0414] The device provides an interface where users can input their daily health and emotional data. The input data is immediately sent to a server and stored in a database. This is implemented using a mobile app developed using Xamarin and React Native.

[0415] 2. Health consultation chat function

[0416] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[0417] 3. Auxiliary input of emotion data

[0418] The device performs facial recognition and voice analysis of the user, collects emotion data, and sends it to a server using OpenCV and Google Cloud Vision API.

[0419] 4. IoT device integration

[0420] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[0421] User operations

[0422] 1. Initial settings of the app

[0423] Users download the app and create an account. They enter basic health information and configure facial recognition and voice analysis. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, and want to lose 5 kg," and then activate the facial recognition feature.

[0424] 2. Daily data entry and consultation

[0425] Users input their daily health data and provide health advice to the generative AI model. For example, they can ask a question via chat, such as, "How is the balance of your diet today?" The generative AI will respond, "Your diet today is high in carbohydrates. If you eat a few more vegetables, it will be more balanced."

[0426] 3. Consultation and appointment with a specialist

[0427] The user can make an appointment for a detailed consultation or online medical consultation with a specialist and then complete the appointment. For example, the user can request, "I would like to make an appointment with a nutritionist at 3:00 PM this weekend," and the server will check the specialist's schedule and confirm the appointment.

[0428] 4. Subscription

[0429] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[0430] In this way, the present invention utilizes a generative AI model and an emotion engine to provide a system that comprehensively supports users' health management. As a specific example, if a user inputs their daily weight and dietary details and asks in chat, "How is the balance of your diet today?", the generative AI can respond, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance," and the emotion engine can recognize signs of stress from the user's facial expressions and voice and provide additional advice, such as, "Try taking deep breaths to relax."

[0431] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0432] Step 1: User creates account

[0433] Users download the app and enter basic health information such as name, age, weight, gender, and allergies on the account creation screen. The data is then saved on the device and sent to the server, which then records the received data in a database.

[0434] Step 2: User performs initial setup

[0435] The user configures facial recognition and voice analysis settings. The device collects the user's facial images and voice samples and sends them to the server, where they are analyzed using an emotion engine and the results are stored in a database.

[0436] Step 3: Enter your daily health data

[0437] Users input their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the device's input interface. The device then sends the data to the server, which then stores the health data in a database.

[0438] Step 4: Enter your health consultation information

[0439] The user inputs a question using the health consultation chat function on the device. For example, they input a question such as, "How balanced is your diet today?" The device then sends this question to the server. The server receives the question and generates an appropriate prompt based on the user's input data.

[0440] Step 5: Request to the generative AI model

[0441] The server sends a prompt to the generative AI model. For example, it generates a prompt such as "Please rate the balance of the user's diet today" and sends it to the generative AI model. The generative AI model generates an answer based on this.

[0442] Step 6: Get the answer from the generative AI model

[0443] The generative AI model generates an answer based on the prompt and sends it back to the server, which then converts the answer into an appropriate format, such as "Your meal today is high in carbohydrates. Adding a few more vegetables would make it more balanced."

[0444] Step 7: Send the answer to the user

[0445] The server sends the answer received from the generative AI model to the device, which displays the answer on the user's chat screen. The user can view the answer and use it to manage their daily health.

[0446] Step 8: Analyze the sentiment data

[0447] The device constantly recognizes the user's face and analyzes their voice to collect emotional data. This data is sent to a server and analyzed by an emotion engine. The analysis results are stored in a database and used to provide advice to the user as needed.

[0448] Step 9: Book an appointment online with a specialist

[0449] The user inputs a request into the device to schedule a consultation with a specialist or an online medical consultation. For example, the user might request, "I'd like to schedule a consultation with a nutritionist this weekend at 3:00 PM." The device then sends this request to the server.

[0450] Step 10: Scheduling and Confirmation

[0451] The server checks the specialist's scheduling system and confirms the appointment based on the user's request. If the appointment is confirmed, the server notifies the user. The user can then have an online consultation with the specialist at the scheduled date and time.

[0452] Step 11: Conducting online consultations

[0453] Based on the scheduled date and time, users can receive online medical treatment and detailed health consultations through video calls with specialists. The server records the session information in a database for future reference.

[0454] Step 12: Manage your subscriptions

[0455] After the free trial period ends, users can choose whether or not to subscribe within the app. The device sends the user's subscription status to the server, which stores the information in a database. If users wish to continue using the app, a feature is provided to prompt them to subscribe.

[0456] (Application example 2)

[0457] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0458] Conventional health management systems collect and manage users' health data and can answer health questions using generative AI, but there were almost no systems designed for use in physical stores. As a result, there was a lack of a way for customers to receive health consultations easily and quickly in physical stores. It was also difficult to provide appropriate health advice in real time in physical stores.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0460] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with specialists and online medical examinations, means for automatically collecting health data in cooperation with IoT devices, and means for providing health consultations and advice to users via smart devices installed in physical stores. This enables customers to easily and quickly receive health consultations in physical stores and appropriate health advice in real time.

[0461] The "means for receiving health data from the user" is an interface for transmitting health data such as weight, number of steps, dietary details, and sleep time input by the user to the server.

[0462] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate appropriate answers to users' health-related questions.

[0463] The "database means for storing health data and health consultation history" is a database system for centrally managing and storing information about health data and health consultations entered by users.

[0464] "Means for reserving and conducting detailed consultations and online medical consultations with specialists" is a function that enables users to reserve and conduct detailed health consultations and online medical consultations with specialists.

[0465] "Means for automatically collecting health data in cooperation with IoT devices" refers to a function that automatically collects health data from IoT devices such as smartwatches and blood pressure monitors and sends it to a server.

[0466] "Means for providing health consultations and advice to users via smart devices installed in physical stores" refers to an interface that allows users to conduct health consultations and receive advice from the generating AI using smart glasses, head-mounted displays, tablets, or smartphones installed in physical stores.

[0467] This invention is a system that realizes health management support for customers in physical stores. Using smart devices installed in physical stores, customers can receive health consultations easily and quickly.

[0468] System configuration

[0469] This system is mainly composed of three entities: a server, a terminal, and a user.

[0470] Server Features

[0471] 1. Means of receiving health data from users

[0472] It receives health data such as weight, number of steps, dietary intake, and sleep time entered by the user and stores it in a database. This requires an internet connection and stable communication.

[0473] 2. A method for answering health questions from users using a generative AI model

[0474] The server runs a generative AI model (e.g., GPT) to automatically generate prompt and appropriate answers to health inquiries from users. Responses to questions are generated using prompt sentences.

[0475] 3. Database means for storing health data and health consultation history

[0476] The received health data and health consultation information are stored in a centralized database system, using database solutions such as MySQL or MongoDB.

[0477] 4. A way to schedule and conduct detailed consultations with specialists and online medical consultations

[0478] The server manages the schedules of specialists, allows users to book detailed health consultations and online consultations, and integrates video calling functionality to provide consultations and advice.

[0479] 5. Means of automatically collecting health data in conjunction with IoT devices

[0480] Health data is automatically collected from IoT devices such as smartwatches and blood pressure monitors and stored in a database.

[0481] Device Features

[0482] 1. Data input interface

[0483] It provides an interface where users can input their health data using smart devices (smart glasses, head-mounted displays, tablets, smartphones) in physical stores, and the input data is sent to a server.

[0484] 2. Health consultation chat function

[0485] The device provides an interface for users to chat with the AI ​​to discuss health issues. Questions are sent to the server in real time, and the AI ​​responds immediately.

[0486] Examples of prompts:

[0487] "What healthy foods should I eat after a workout?"

[0488] 3. Emotion data input assistance

[0489] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends it to a server, allowing the AI ​​to reflect the user's emotional state in its responses.

[0490] User operations

[0491] 1. Initial Setup

[0492] Users download the app, create an account, enter basic health information, configure facial recognition and voice analysis settings, and set up how to input emotional data.

[0493] 2. Data entry and consultation

[0494] Users can use smart devices in physical stores to input their daily health data and also consult with the AI ​​to receive appropriate health advice.

[0495] Examples:

[0496] When a user types in, "What healthy foods should I eat after exercise?", the generative AI will answer, "I recommend a protein bar that is effective after exercise."

[0497] 3. Consult with a specialist and book an online consultation

[0498] Users can easily book and schedule detailed consultations with specialists or online medical consultations using the server's reservation management system.

[0499] Examples of technology implementation

[0500] Hardware Usage

[0501] Smart glasses, head-mounted displays, tablets, and smartphones will be installed in the physical store, and these devices will act as interfaces with users.

[0502] Software Usage

[0503] It uses Python's Flask as the server backend, MySQL as the database, and the open source GPT as the generative AI model.

[0504] These features allow users to seamlessly manage and consult about their health within a physical store, and they can also receive expert advice in real time, contributing to improving their health.

[0505] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0506] Step 1:

[0507] Users access smart devices installed in physical stores and input health data (weight, number of steps, dietary content, sleep time, etc.).

[0508] Input: Health data entered by the user through their smart device.

[0509] Output: The entered data is sent to the server.

[0510] How it works: An input interface is displayed on the screen of the smart device, and the user enters their health information item by item. After completing the input, the data is sent to the server in real time.

[0511] Step 2:

[0512] The server stores the received health data in a database.

[0513] Input: Health data submitted in step 1.

[0514] Output: Health data is stored in a database.

[0515] What happens: The server executes a query to properly structure the received data and store it in a MySQL database.

[0516] Step 3:

[0517] The user uses a smart device to provide health consultation to the generated AI model.

[0518] Input: A question that the user types into their smart device.

[0519] Output: The answer generated by the generative AI model.

[0520] How it works: The user types a prompt into their smart device, such as "What healthy foods should I eat after exercise?", and it is sent to the server, which then passes the question to the generative AI model.

[0521] Step 4:

[0522] The server uses the generative AI model to generate answers to the user's questions.

[0523] Input: The question entered by the user in step 3.

[0524] Output: The correct answer to the question.

[0525] How it works: The server inputs a question as a prompt to a generative AI model (e.g., GPT), and the model generates the optimal answer to that question. The generated answer is returned to the server.

[0526] Step 5:

[0527] The server sends the generated answer back to the smart device and displays it to the user.

[0528] Input: The answer generated in step 4.

[0529] Output: The answer is displayed on the user's smart device.

[0530] How it works: The server sends the answer obtained from the generative AI model to the smart device, which immediately displays the answer to the user for confirmation.

[0531] Step 6:

[0532] If necessary, the user books an online consultation with a specialist.

[0533] Input: Booking request entered by user into smart device.

[0534] Output: Booking confirmation and schedule confirmation.

[0535] Specific operation: The user inputs the desired date and time for consultation with a specialist through their smart device and sends it to the server. The server checks the specialist's schedule and, if the appointment is confirmed, notifies the user's smart device of the information.

[0536] Step 7:

[0537] Online consultations with specialists are conducted, providing detailed advice on the user's health condition.

[0538] Input: User's health condition and consultation details at the time of appointment.

[0539] Output: Expert advice.

[0540] Specific operation: At the scheduled time, the expert and the user will be connected via video call, where the expert will check the user's health condition and provide advice.

[0541] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0542] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0543] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0544] [Second embodiment]

[0545] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0546] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0547] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0548] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0549] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0550] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0551] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0552] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0553] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0554] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0555] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0556] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0557] This invention is a health management system that provides free health consultations using generative AI to assist users in managing their health. The system collects input data from users, uses generative AI models to answer health questions, and even provides in-depth consultations with experts and online medical consultations.

[0558] System configuration

[0559] Server Features

[0560] 1. Database Management

[0561] The server receives health data entered by the user (e.g., weight, number of steps, dietary content, sleep time, etc.) and stores this data in a centrally managed database.

[0562] 2. Operation of generative AI

[0563] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[0564] 3. Collaboration with experts

[0565] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[0566] 4. IoT device data collection

[0567] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[0568] Device Features

[0569] 1. Data input interface

[0570] The device provides an interface where users can input their daily health data, which is then sent to a server and stored in a database.

[0571] 2. Health consultation chat function

[0572] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[0573] 3. IoT device integration

[0574] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[0575] User operations

[0576] 1. Initial Setup

[0577] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.).

[0578] 2. Data entry and consultation

[0579] Users can input their daily health data and ask the AI ​​for health advice, such as, "How balanced is your diet today?" or "What kind of exercise should I do?"

[0580] 3. Consult with a specialist and book an online consultation

[0581] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[0582] 4. Subscription

[0583] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[0584] Specific examples

[0585] 1. Example of initial settings

[0586] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[0587] 2. Examples of daily data entry and consultation

[0588] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[0589] 3. Example of collaboration with IoT devices

[0590] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[0591] 4. Consultations with specialists and examples of treatment

[0592] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[0593] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[0594] The processing flow will be explained below.

[0595] Step 1: Initial Setup

[0596] 1. The server monitors the installation of the application and accepts a connection request to the server when the application is first launched.

[0597] 2. The device displays an account creation screen to the user, prompting them to enter basic information such as their name, email address, and password.

[0598] 3. The device sends the entered basic information to the server.

[0599] 4. The server stores the received basic information in a database and creates an account.

[0600] 5. The server sends a notification to the device that the account has been created.

[0601] 6. The device will display a notification to the user that the account has been created.

[0602] Step 2: Daily entry of health data

[0603] 1. The user launches the app and enters daily health data such as weight, number of steps, diet, and sleep time.

[0604] 2. The device sends the entered health data to the server.

[0605] 3. The server stores the received health data in a database.

[0606] 4. The device will notify the user that the data has been entered and allow them to visually check the data in a graph.

[0607] Step 3: IoT device integration

[0608] 1. The user connects an IoT device such as a smartwatch or blood pressure monitor to the app.

[0609] 2. The device automatically collects health data from connected IoT devices.

[0610] 3. The device sends the collected data to the server.

[0611] 4. The server stores the received data in a database.

[0612] 5. The device notifies the user that the latest health data has been updated.

[0613] Step 4: Free health consultation

[0614] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[0615] 2. The device sends the user's question to the server.

[0616] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[0617] 4. The server sends the generated response to the terminal.

[0618] 5. The device displays the answer from the generating AI to the user.

[0619] Step 5: Consult with a specialist and online consultation

[0620] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[0621] 2. The device sends the user's request to the server.

[0622] 3. The server queries the expert's schedule to check available dates and times.

[0623] 4. The server sends the confirmed schedule information to the terminal.

[0624] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[0625] 6. The user selects the desired date and time, and the device sends the selection to the server.

[0626] 7. The server finalizes the booking and notifies the user and the expert.

[0627] 8. The user launches the app at the scheduled time and begins the online consultation.

[0628] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[0629] 10. The server records the medical treatment details and advice in a database.

[0630] Step 6: Subscribe

[0631] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[0632] 2. The device will display instructions to the user via push notification or email.

[0633] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[0634] 4. The terminal sends the entered payment information to the server.

[0635] 5. The server processes the payment information and updates the subscription status to paid.

[0636] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[0637] Through these steps, the system efficiently supports users in managing their health, providing a comprehensive range of services from free consultations to professional advice and medical treatment.

[0638] Example 1

[0639] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0640] In modern society, personal health management is an important topic, and many users need to understand their own health status and receive appropriate advice. However, collecting health data on a daily basis and utilizing it to receive effective advice requires specialized knowledge and expensive equipment, placing a heavy burden on the average user. In addition, there is a lack of systems that allow seamless in-depth consultations with experts or online medical consultations. This creates a challenge for users, preventing them from optimally managing their health.

[0641] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0642] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and conducting detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, and means for the health data to serve as the basis for analysis and recommendations based on the generative AI model, thereby enabling users to understand their own health status and receive appropriate advice without the need for specialized knowledge or expensive equipment.

[0643] "Health data from the user" refers to health-related data such as weight, number of steps, dietary details, and sleep time entered by the user.

[0644] A "generative AI model" is an artificial intelligence model that generates appropriate answers to health inquiries from users.

[0645] "Health consultation history" is a record of past health-related questions and answers asked by the user.

[0646] "Database" means an information management system for storing health data and health consultation history.

[0647] "Detailed consultations and online medical treatment with experts" refers to specific health consultations and remote medical treatments that users receive from experts.

[0648] An "IoT device" is a device, such as a smartwatch or blood pressure monitor, that is connected to the Internet and collects health data.

[0649] "Analysis and recommendation" is the process of analyzing collected health data and providing appropriate health advice to the user.

[0650] "Subscription status" is information that indicates the user's subscription service usage status and expiration date.

[0651] "Real-time communication means" refers to communication technologies that allow users to ask questions using generative AI models and receive instant answers.

[0652] The present invention is a health consultation system that utilizes a generative AI model to support users' health management. This system functions in cooperation with users, terminals, and a server.

[0653] Hardware and software used

[0654] 1. Hardware

[0655] Device: Smartphone or tablet

[0656] Server: High-performance cloud server

[0657] IoT devices: smartwatches, blood pressure monitors, etc.

[0658] 2. Software

[0659] Application: Health management app

[0660] Database: MySQL

[0661] Generative AI model: OpenAI's GPT-4

[0662] Scheduling Software: Professional scheduling system

[0663] Communication protocol: REST API

[0664] Program processing

[0665] Server Features

[0666] The server receives health data sent by users and stores it in a database. Specifically, it uses a REST API to collect data such as the user's weight, number of steps, dietary habits, and sleep time. This data is stored in a MySQL database and managed centrally.

[0667] Furthermore, the server runs the generative AI model GPT-4, which generates appropriate answers to health inquiries from users. For example, if a user asks, "How is your diet balanced today?", GPT-4 will generate an answer such as, "Your diet today is high in carbohydrates. Eating a few more vegetables would improve the balance."

[0668] The server also accepts requests for in-depth consultations with specialists and online consultations, and uses dedicated scheduling software to manage the specialists' schedules and confirm appropriate appointments.

[0669] Device Features

[0670] The device provides an interface that allows users to input their daily health data. The input data is sent to a server in real time and stored in a database. It also provides an interface for chatting with the generating AI for health consultations. When a user accesses the chat screen and asks, "What kind of exercise should I do?", the answer from the generating AI is displayed instantly.

[0671] The device also connects to IoT devices such as smartwatches and blood pressure monitors, automatically collecting data from these devices and sending it to a server. For example, heart rate data obtained from a smartwatch is automatically sent to the server.

[0672] User operations

[0673] First, users download the app and create an account. They enter basic health information (age, weight, medical history, allergies, health goals, etc.). Next, they can enter their daily health data and ask the AI ​​for health advice.

[0674] Users can also make appointments for detailed consultations with specialists or online medical consultations, and receive specific health advice via video call at the scheduled time. After the free trial period ends, users can continue to receive detailed advice and online medical consultations by subscribing.

[0675] Specific examples

[0676] 1. Example of initial settings

[0677] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[0678] 2. Examples of daily data entry and consultation

[0679] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[0680] 3. Example of collaboration with IoT devices

[0681] When a user is wearing a smartwatch, the heart rate data acquired from the smartwatch is automatically sent to the device and stored on the server.

[0682] 4. Consultations with specialists and examples of treatment

[0683] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[0684] keyword

[0685] Generative AI model, prompt sentence

[0686] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0687] Step 1:

[0688] Initial Setup

[0689] The user installs and launches the health management app. Next, they select "New Registration" and enter basic health information such as their name, age, sex, weight, height, allergy information, and health goals. The entered information is sent from the device to the server, which receives it and stores it in a database.

[0690] Input: Name, age, gender, weight, height, allergy information, health goals

[0691] Output: User information is saved in the database

[0692] Step 2:

[0693] Routine data entry

[0694] Users enter their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the app. The data is sent in real time from the device to the server, which receives it and stores it in a database.

[0695] Input: weight, diet, steps, sleep time

[0696] Output: Health data is stored in a database

[0697] Step 3:

[0698] Using generative AI models for health consultations

[0699] A user uses the app's consultation function to ask a health question in chat format. For example, the user might ask, "How balanced is my diet today?" This question is sent from the device to the server. The server then invokes a generative AI model (GPT-4) to generate an answer to the question. The generated answer is sent to the user's device and displayed on the chat screen.

[0700] Input: User question (e.g., "How balanced is your diet today?")

[0701] Output: Answer from the generative AI model (e.g., "Your meal today is high in carbohydrates. Adding a few more vegetables would improve the balance.")

[0702] Step 4:

[0703] Book an in-depth consultation with a specialist or online consultation

[0704] A user uses the app's "Consultation with a Specialist" feature to book a consultation or consultation. For example, they request, "I'd like to book a consultation with a nutritionist this weekend at 3:00 PM." This request is sent to the server, which uses dedicated scheduling software to manage the specialist's schedule and confirm the appointment. The appointment details are sent to the user's device and notified.

[0705] Input: Appointment request (e.g., "I'd like to schedule a consultation with a nutritionist this weekend at 3pm")

[0706] Output: Confirmation of booking

[0707] Step 5:

[0708] Data linkage with IoT devices

[0709] When a user uses IoT devices such as a smartwatch or blood pressure monitor, health data is collected in real time from these devices. The data is transmitted to the device via Bluetooth or Wi-Fi. The device then sends the data to a server, which stores it in a database. For example, heart rate data from a smartwatch is automatically collected and recorded.

[0710] Input: Health data from IoT devices (e.g., heart rate, blood pressure)

[0711] Output: Health data is stored in a database

[0712] Step 6:

[0713] Manage your subscription

[0714] The server manages the user's subscription status. Before the free period ends, the server notifies the user and prompts them to subscribe. The user selects the desired plan within the app and enters their credit card information to complete the payment. The server verifies the payment information and updates the subscription status.

[0715] Input: Subscription request, credit card information

[0716] Output: Updated subscription status, payment confirmation notification

[0717] (Application example 1)

[0718] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] In modern society, many health problems, such as lifestyle-related diseases and obesity, are on the rise. In particular, busy lifestyles make it difficult to choose healthy meals. Other issues include the inability to easily access health consultations and medical treatment from experts, and a lack of appropriate advice based on individual health data. In response to these issues, there is a demand for fast and accurate health management support.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0721] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, means for proposing health menus based on user data, and means for conducting health consultations with the user in a chat format. This allows users to receive dietary suggestions based on their own health information and easily consult with experts, thereby enabling them to achieve a healthy lifestyle.

[0722] The "means for receiving health data from the user" is an interface for sending and receiving health-related information (for example, weight, dietary content, amount of exercise, etc.) input by the user to the server.

[0723] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate answers to health-related questions asked by users.

[0724] The "database means for storing health data and health consultation history" is a database system for long-term storage and management of health data entered by users and the health consultation response history provided by the generation AI.

[0725] "Means for booking and conducting detailed consultations and online medical consultations with specialists" refers to a system that allows users to book and conduct detailed health consultations and online medical consultations with specialists.

[0726] "Means for automatically collecting health data in conjunction with IoT devices" refers to a function that works in conjunction with smartwatches and everyday health measurement devices to collect health data automatically obtained from them and send it to a server.

[0727] "Means for suggesting healthy menus based on user data" refers to a function in which AI uses health data provided by the user to generate and suggest healthy meal menus based on that data.

[0728] The "means for providing health consultations to users in chat format" is a system that allows users and generative AI models to exchange chat-style messages in real time and respond to health-related questions.

[0729] This invention is a system that supports users' health management, and aims to collect health data of individual users through collaboration with smartphones and IoT devices, and provide health consultations and dietary suggestions using a generative AI model. A detailed description of the system is provided below.

[0730] System configuration

[0731] This invention is mainly composed of three main elements: a server, a terminal, and a user.

[0732] Server Features

[0733] The server has the following main functions:

[0734] 1. Means of receiving health data from users:

[0735] The server receives basic health information (age, weight, health goals, allergy information, etc.) entered by the user through a smartphone application and stores it in a database.

[0736] 2. How to use generative AI models to answer health questions:

[0737] Leveraging OpenAI's ChatCompletion API, the system automatically generates answers to health-related questions from users. For example, if a user asks, "How balanced is your diet today?", the generative AI model instantly provides an answer.

[0738] 3. Database means for storing health data and health consultation history:

[0739] The server uses SQLite or another database management system to store the health data provided by the user and the history of answers provided by the generative AI model.

[0740] 4. How to book and conduct in-depth consultations with specialists and online consultations:

[0741] We provide a reservation system that has a function for linking with experts and allows users to consult with experts online, allowing users to easily obtain professional advice.

[0742] 5. Means of automatically collecting health data in conjunction with IoT devices:

[0743] It has the ability to automatically collect data from IoT devices such as smartwatches and blood pressure monitors, and send it to a server for storage.

[0744] 6. How to suggest healthy menus based on user data:

[0745] Based on the data provided by the user, the generative AI model will suggest healthy meals, for example, if the user's goal is to lose weight, it will provide a suitable meal plan.

[0746] 7. Means of providing health consultations to users through chat:

[0747] Users can chat with the generated AI model in real time via a smartphone app and ask for health advice.

[0748] Device Features

[0749] The device (smartphone, etc.) has the following functions:

[0750] 1. Data input interface:

[0751] It provides an interface that allows users to manually input their health data, making it easy to record daily weight, diet, exercise, etc.

[0752] 2. Health consultation chat function:

[0753] It provides an interface that allows users to chat with the generated AI and receive health advice.

[0754] 3. IoT device integration:

[0755] It works in conjunction with IoT devices such as smartwatches and blood pressure monitors, and has the ability to automatically collect data and send it to a server.

[0756] User operations

[0757] The user uses this system by following the steps below:

[0758] 1. Initial Setup:

[0759] Install the app and create an account, which starts by entering basic health information.

[0760] 2. Data entry and consultation:

[0761] Enter your daily health data and ask the AI ​​for health advice. For example, you can ask, "Could you please give me some advice about my recent diet?"

[0762] 3. Specialist consultations and online consultation appointments:

[0763] You can book consultations and appointments with specialists and receive professional advice via video call.

[0764] 4. Subscription:

[0765] After the free trial period ends, you can subscribe to continue using the service for detailed advice and online consultations.

[0766] Specific examples

[0767] Here is a concrete example of its use:

[0768] 1. Example of suggesting healthy menus based on user data:

[0769] After the user enters their health goals, they can ask, "How is your diet balanced today?" and the generative AI will suggest, "You're eating too many carbohydrates today. Try to eat more vegetables at your next meal."

[0770] 2. Example prompt:

[0771] User data: Age: 30, Weight: 70kg, Goal: Lose weight, Allergy: Peanuts

[0772] User Question: How balanced is your diet today?

[0773] Please provide your answer:

[0774] This allows users to understand their health status in real time and receive advice on appropriate diet and exercise.

[0775] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0776] Step 1: Initial Setup

[0777] Users download the smartphone app and create an account. The app prompts users to enter basic health information (age, weight, health goals, allergy information, etc.), which is then sent to a server and stored in a database.

[0778] Input: User's health information (age, weight, health goals, allergy information)

[0779] Data processing: Send input data to the server in JSON format

[0780] Output: User information is saved in the database

[0781] Step 2: Data entry

[0782] Users enter their daily health data (weight, diet, exercise, etc.) into the app, which then sends this data to a server and stores it in a database.

[0783] Input: User's daily health data (weight, diet, exercise)

[0784] Data processing: Send input data to the server in JSON format

[0785] Output: Daily health data is stored in a database

[0786] Step 3: IoT device integration

[0787] The app connects with users' IoT devices such as smartwatches and blood pressure monitors, allowing data collected from the devices to be automatically sent to the server.

[0788] Input: Health data from IoT devices (heart rate, steps, etc.)

[0789] Data processing: Automatically send data to the server

[0790] Output: Data from IoT devices is stored in a database

[0791] Step 4: Conducting health consultations

[0792] Users use the app's chat function to ask questions about their health. When a user enters a question, the question is sent to the server, which uses a generative AI model to generate an answer and sends it back to the user.

[0793] Input: User's health consultation question

[0794] Data computation: Generative AI models generate answers based on prompts

[0795] Output: The answer is returned to the user

[0796] Step 5: Healthy menu suggestions

[0797] When a user requests a healthy meal plan, they input their request into the app, and the server uses a generative AI model based on the user data to suggest an appropriate meal plan.

[0798] Input: User's health menu request

[0799] Data arithmetic: Generative AI models generate menus based on prompts

[0800] Output: A suggested meal menu is displayed to the user

[0801] Step 6: Book a consultation with a specialist

[0802] When a user wants to consult with a specialist, they make a reservation through the app. The server checks the specialist's schedule and confirms the reservation.

[0803] Input: User's consultation appointment request

[0804] Data processing: Matching booking requests with specialist schedules

[0805] Output: A confirmation of the booking is sent to the user.

[0806] Step 7: Manage your subscriptions

[0807] After the free trial period ends, users can subscribe to continue receiving detailed advice and treatment features. The app will manage users' subscription status and notify them when the trial period ends.

[0808] Input: Free period end information

[0809] Data Processing: Subscription Status Update

[0810] Output: A prompt to subscribe is sent to the user

[0811] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0812] This invention is a health management system that provides free health consultations utilizing generative AI and an emotion engine to assist users in managing their health. The system collects input data from users, uses a generative AI model and an emotion engine to answer health questions, and even provides in-depth consultations with experts and online medical treatments.

[0813] System configuration

[0814] Server Features

[0815] 1. Database Management

[0816] The server receives health data (e.g., weight, number of steps, dietary content, sleep time, etc.) and emotional data entered by the user and stores this data in a centrally managed database.

[0817] 2. Operation of generative AI

[0818] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[0819] 3. Operating the Emotion Engine

[0820] The server runs an emotion engine, analyzes the data input by the user and the content of the health consultation, and recognizes the user's emotional state. This recognized emotional data is also stored in the database.

[0821] 4. Collaboration with experts

[0822] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[0823] 5. IoT device data collection

[0824] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[0825] Device Features

[0826] 1. Data input interface

[0827] The device provides an interface through which users can input their daily health and emotional data, which is then sent to a server and stored in a database.

[0828] 2. Health consultation chat function

[0829] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[0830] 3. Emotion data input assistance

[0831] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends this data to a server.

[0832] 4. IoT device integration

[0833] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[0834] User operations

[0835] 1. Initial Setup

[0836] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.). The app also configures facial recognition and voice analysis settings, and sets up how to input emotional data.

[0837] 2. Data entry and consultation

[0838] Users input their daily health data and ask the AI ​​for health advice. For example, they can ask questions such as, "How balanced is my diet today?" or "What kind of exercise should I do?" The emotion engine analyzes the user's emotions and can suggest, for example, "How to relax if you're feeling stressed."

[0839] 3. Consult with a specialist and book an online consultation

[0840] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[0841] 4. Subscription

[0842] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[0843] Specific examples

[0844] 1. Example of initial settings

[0845] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. They then configure facial recognition and voice analysis. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg" to enable the facial recognition feature.

[0846] 2. Examples of daily data entry and consultation

[0847] The user enters their daily weight and dietary information and asks in chat, "How is your diet balance today?" The generative AI responds, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance." The emotion engine recognizes signs of stress from the user's facial expressions and voice and provides additional advice, such as "Try taking deep breaths to relax."

[0848] 3. Example of collaboration with IoT devices

[0849] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[0850] 4. Consultations with specialists and examples of treatment

[0851] A user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist contact each other via video call, and receive specific dietary advice. The user's emotional state is also provided to the nutritionist, enabling more personalized advice.

[0852] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[0853] The processing flow will be explained below.

[0854] Step 1: Initial Setup

[0855] 1. The user downloads and installs the app.

[0856] 2. The device will display the account creation screen when you first start it up.

[0857] 3. The user creates an account by entering their name, email address, password, age, weight, gender, and allergy information.

[0858] 4. The terminal sends the entered information to the server.

[0859] 5. The server stores the received information in its database and completes the account creation.

[0860] 6. The server sends a notification to the device that account creation is complete.

[0861] 7. The device displays a notification to the user that the account has been created.

[0862] Step 2: Configure facial recognition and voice analysis

[0863] 1. The device will display the initial setting screen for the emotion engine.

[0864] 2. The user configures the facial recognition and voice analysis settings (for example, by moving their face towards the camera or reading a specific phrase aloud).

[0865] 3. The device sends the settings to the server.

[0866] 4. The server saves the emotion engine configuration data in the database.

[0867] Step 3: Daily entry of health and emotional data

[0868] 1. The user launches the app and enters their daily health data, such as weight, number of steps, diet, and sleep time.

[0869] 2. The terminal sends the entered data to the server.

[0870] 3. The server stores the received data in a database.

[0871] 4. The device collects emotional data from the user's facial recognition and voice, and also sends this to the server.

[0872] 5. The server stores the emotion data in a database.

[0873] Step 4: Free health consultation

[0874] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[0875] 2. The device sends the user's question to the server.

[0876] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[0877] 4. The server sends the generated response to the terminal.

[0878] 5. The server starts the emotion engine and analyzes the user's emotion data.

[0879] 6. The server customizes the answer based on the sentiment data and generates appropriate follow-up advice.

[0880] 7. The server sends a customized answer and additional advice to the device.

[0881] 8. The device displays the user the answer from the generative AI and additional advice based on the emotional data.

[0882] Step 5: Consult with a specialist and online consultation

[0883] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[0884] 2. The device sends the user's request to the server.

[0885] 3. The server queries the expert's schedule to check available dates and times.

[0886] 4. The server sends the confirmed schedule information to the terminal.

[0887] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[0888] 6. The user selects the desired date and time, and the device sends the selection to the server.

[0889] 7. The server confirms the booking and notifies the user and the expert.

[0890] 8. The user launches the app at the scheduled time and begins the online consultation.

[0891] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[0892] 10. The server records the medical treatment details and advice in a database.

[0893] Step 6: Subscribe

[0894] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[0895] 2. The device will display instructions to the user via push notification or email.

[0896] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[0897] 4. The terminal sends the entered payment information to the server.

[0898] 5. The server processes the payment information and updates the subscription status to paid.

[0899] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[0900] Through these steps, the system efficiently supports users in managing their health and provides a comprehensive approach that includes emotional data. The emotion engine analyzes the user's emotional state and customizes the generative AI model's answers, resulting in more personalized healthcare services.

[0901] Example 2

[0902] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0903] Conventional health management systems have difficulty collecting not only users' health data but also their emotional data and providing appropriate advice based on that data. Furthermore, the procedures for making appointments for consultations with specialists or online medical consultations are often complicated, placing a burden on users. Furthermore, insufficient integration with IoT devices results in inefficient data collection. This makes it difficult for users to effectively manage their health on a daily basis.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0905] In this invention, the server includes means for receiving health data and emotional data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing health data and health consultation history, means for analyzing the user's emotional state using an emotion engine, means for booking and implementing detailed consultations with experts and online medical treatments, and means for automatically collecting health data in cooperation with IoT devices. This enables comprehensive health management based on health data and emotional data, and smooth cooperation with experts reduces the burden on the user and enables effective health management.

[0906] "Health data from the user" refers to data relating to the user's daily health condition, such as weight, number of steps, dietary content, and sleep time, which the user inputs himself or herself.

[0907] "Emotional data" is data about a user's emotional state collected through facial recognition and voice analysis of the user.

[0908] A "generative AI model" is an artificial intelligence model that generates answers in natural language in response to health inquiries from users. An example of this would be a chatbot.

[0909] A "means for answering health inquiries" is a method or system that enables a generative AI model to provide appropriate answers to health inquiries from users.

[0910] The "database means" is a data management system for storing the user's health data and health consultation history.

[0911] An "emotion engine" is a technology that analyzes input data and dialogue content from users to recognize their emotional state.

[0912] "Means for booking and conducting in-depth consultations and online medical consultations with specialists" refers to systems and functions that allow users to book a date and time for a consultation or medical consultation with a specialist and to conduct that appointment online.

[0913] "Means for automatically collecting health data in cooperation with IoT devices" refers to a method or system for automatically acquiring health data from IoT devices such as smartwatches and blood pressure monitors and transmitting that data to a server.

[0914] The present invention is a health management system that provides a free health consultation function utilizing a generative AI model and an emotion engine to assist users in managing their health. The system collects health and emotion data from users, uses the generative AI model and the emotion engine to answer health questions, and also provides detailed consultations with experts and online medical treatments. The following describes in detail the embodiments of the present invention.

[0915] server

[0916] The server performs the following functions:

[0917] 1. Data Collection and Management

[0918] The server receives the health and emotion data sent from the device and stores them in a centralized database, which is managed using MySQL or PostgreSQL, for example.

[0919] 2. Operation of generative AI models

[0920] When a health consultation from a user arrives at the server, the server generates a request to a generative AI model (e.g., GPT-4) and creates and sends an appropriate prompt. For example, in response to the question, "How is your diet balance today?", the server generates a prompt such as, "Please rate the user's diet balance today."

[0921] 3. Operating the Emotion Engine

[0922] The server runs an emotion engine (e.g., Affectiva or Azure Emotion Recognition API) that analyzes the user's input data and dialogue to recognize their emotional state. The recognized emotion data is also stored in a database.

[0923] 4. Scheduling with an expert

[0924] The server manages the schedules of the experts and confirms the reservation based on the reservation request from the user. The scheduling is done using the Google Calendar API.

[0925] 5. IoT device data collection

[0926] The server connects to IoT devices (e.g., smartwatches and blood pressure monitors) and automatically collects health data from these devices and stores it in a database. Firebase and AWS IoT are used here.

[0927] Terminal

[0928] The terminal provides the following user interface:

[0929] 1. Data input interface

[0930] The device provides an interface where users can input their daily health and emotional data. The input data is immediately sent to a server and stored in a database. This is implemented using a mobile app developed using Xamarin and React Native.

[0931] 2. Health consultation chat function

[0932] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[0933] 3. Auxiliary input of emotion data

[0934] The device performs facial recognition and voice analysis of the user, collects emotion data, and sends it to a server using OpenCV and Google Cloud Vision API.

[0935] 4. IoT device integration

[0936] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[0937] User operations

[0938] 1. Initial settings of the app

[0939] Users download the app and create an account. They enter basic health information and configure facial recognition and voice analysis. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, and want to lose 5 kg," and then activate the facial recognition feature.

[0940] 2. Daily data entry and consultation

[0941] Users input their daily health data and provide health advice to the generative AI model. For example, they can ask a question via chat, such as, "How is the balance of your diet today?" The generative AI will respond, "Your diet today is high in carbohydrates. If you eat a few more vegetables, it will be more balanced."

[0942] 3. Consultation and appointment with a specialist

[0943] The user can make an appointment for a detailed consultation or online medical consultation with a specialist and then complete the appointment. For example, the user can request, "I would like to make an appointment with a nutritionist at 3:00 PM this weekend," and the server will check the specialist's schedule and confirm the appointment.

[0944] 4. Subscription

[0945] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[0946] In this way, the present invention utilizes a generative AI model and an emotion engine to provide a system that comprehensively supports users' health management. As a specific example, if a user inputs their daily weight and dietary details and asks in chat, "How is the balance of your diet today?", the generative AI can respond, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance," and the emotion engine can recognize signs of stress from the user's facial expressions and voice and provide additional advice, such as, "Try taking deep breaths to relax."

[0947] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0948] Step 1: User creates account

[0949] Users download the app and enter basic health information such as name, age, weight, gender, and allergies on the account creation screen. The data is then saved on the device and sent to the server, which then records the received data in a database.

[0950] Step 2: User performs initial setup

[0951] The user configures facial recognition and voice analysis settings. The device collects the user's facial images and voice samples and sends them to the server, where they are analyzed using an emotion engine and the results are stored in a database.

[0952] Step 3: Enter your daily health data

[0953] Users input their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the device's input interface. The device then sends the data to the server, which then stores the health data in a database.

[0954] Step 4: Enter your health consultation information

[0955] The user inputs a question using the health consultation chat function on the device. For example, they input a question such as, "How balanced is your diet today?" The device then sends this question to the server. The server receives the question and generates an appropriate prompt based on the user's input data.

[0956] Step 5: Request to the generative AI model

[0957] The server sends a prompt to the generative AI model. For example, it generates a prompt such as "Please rate the balance of the user's diet today" and sends it to the generative AI model. The generative AI model generates an answer based on this.

[0958] Step 6: Get the answer from the generative AI model

[0959] The generative AI model generates an answer based on the prompt and sends it back to the server, which then converts the answer into an appropriate format, such as "Your meal today is high in carbohydrates. Adding a few more vegetables would make it more balanced."

[0960] Step 7: Send the answer to the user

[0961] The server sends the answer received from the generative AI model to the device, which displays the answer on the user's chat screen. The user can view the answer and use it to manage their daily health.

[0962] Step 8: Analyze the sentiment data

[0963] The device constantly recognizes the user's face and analyzes their voice to collect emotional data. This data is sent to a server and analyzed by an emotion engine. The analysis results are stored in a database and used to provide advice to the user as needed.

[0964] Step 9: Book an appointment online with a specialist

[0965] The user inputs a request into the device to schedule a consultation with a specialist or an online medical consultation. For example, the user might request, "I'd like to schedule a consultation with a nutritionist this weekend at 3:00 PM." The device then sends this request to the server.

[0966] Step 10: Scheduling and Confirmation

[0967] The server checks the specialist's scheduling system and confirms the appointment based on the user's request. If the appointment is confirmed, the server notifies the user. The user can then have an online consultation with the specialist at the scheduled date and time.

[0968] Step 11: Conducting online consultations

[0969] Based on the scheduled date and time, users can receive online medical treatment and detailed health consultations through video calls with specialists. The server records the session information in a database for future reference.

[0970] Step 12: Manage your subscriptions

[0971] After the free trial period ends, users can choose whether or not to subscribe within the app. The device sends the user's subscription status to the server, which stores the information in a database. If users wish to continue using the app, a feature is provided to prompt them to subscribe.

[0972] (Application example 2)

[0973] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0974] Conventional health management systems collect and manage users' health data and can answer health questions using generative AI, but there were almost no systems designed for use in physical stores. As a result, there was a lack of a way for customers to receive health consultations easily and quickly in physical stores. It was also difficult to provide appropriate health advice in real time in physical stores.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0976] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with specialists and online medical examinations, means for automatically collecting health data in cooperation with IoT devices, and means for providing health consultations and advice to users via smart devices installed in physical stores. This enables customers to easily and quickly receive health consultations in physical stores and appropriate health advice in real time.

[0977] The "means for receiving health data from the user" is an interface for transmitting health data such as weight, number of steps, dietary details, and sleep time input by the user to the server.

[0978] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate appropriate answers to users' health-related questions.

[0979] The "database means for storing health data and health consultation history" is a database system for centrally managing and storing information about health data and health consultations entered by users.

[0980] "Means for reserving and conducting detailed consultations and online medical consultations with specialists" is a function that enables users to reserve and conduct detailed health consultations and online medical consultations with specialists.

[0981] "Means for automatically collecting health data in cooperation with IoT devices" refers to a function that automatically collects health data from IoT devices such as smartwatches and blood pressure monitors and sends it to a server.

[0982] "Means for providing health consultations and advice to users via smart devices installed in physical stores" refers to an interface that allows users to conduct health consultations and receive advice from the generating AI using smart glasses, head-mounted displays, tablets, or smartphones installed in physical stores.

[0983] This invention is a system that realizes health management support for customers in physical stores. Using smart devices installed in physical stores, customers can receive health consultations easily and quickly.

[0984] System configuration

[0985] This system is mainly composed of three entities: a server, a terminal, and a user.

[0986] Server Features

[0987] 1. Means of receiving health data from users

[0988] It receives health data such as weight, number of steps, dietary intake, and sleep time entered by the user and stores it in a database. This requires an internet connection and stable communication.

[0989] 2. A method for answering health questions from users using a generative AI model

[0990] The server runs a generative AI model (e.g., GPT) to automatically generate prompt and appropriate answers to health inquiries from users. Responses to questions are generated using prompt sentences.

[0991] 3. Database means for storing health data and health consultation history

[0992] The received health data and health consultation information are stored in a centralized database system, using database solutions such as MySQL or MongoDB.

[0993] 4. A way to schedule and conduct detailed consultations with specialists and online medical consultations

[0994] The server manages the schedules of specialists, allows users to book detailed health consultations and online consultations, and integrates video calling functionality to provide consultations and advice.

[0995] 5. Means of automatically collecting health data in conjunction with IoT devices

[0996] Health data is automatically collected from IoT devices such as smartwatches and blood pressure monitors and stored in a database.

[0997] Device Features

[0998] 1. Data input interface

[0999] It provides an interface where users can input their health data using smart devices (smart glasses, head-mounted displays, tablets, smartphones) in physical stores, and the input data is sent to a server.

[1000] 2. Health consultation chat function

[1001] The device provides an interface for users to chat with the AI ​​to discuss health issues. Questions are sent to the server in real time, and the AI ​​responds immediately.

[1002] Examples of prompts:

[1003] "What healthy foods should I eat after a workout?"

[1004] 3. Emotion data input assistance

[1005] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends it to a server, allowing the AI ​​to reflect the user's emotional state in its responses.

[1006] User operations

[1007] 1. Initial Setup

[1008] Users download the app, create an account, enter basic health information, configure facial recognition and voice analysis settings, and set up how to input emotional data.

[1009] 2. Data entry and consultation

[1010] Users can use smart devices in physical stores to input their daily health data and also consult with the AI ​​to receive appropriate health advice.

[1011] Examples:

[1012] When a user types in, "What healthy foods should I eat after exercise?", the generative AI will answer, "I recommend a protein bar that is effective after exercise."

[1013] 3. Consult with a specialist and book an online consultation

[1014] Users can easily book and schedule detailed consultations with specialists or online medical consultations using the server's reservation management system.

[1015] Examples of technology implementation

[1016] Hardware Usage

[1017] Smart glasses, head-mounted displays, tablets, and smartphones will be installed in the physical store, and these devices will act as interfaces with users.

[1018] Software Usage

[1019] It uses Python's Flask as the server backend, MySQL as the database, and the open source GPT as the generative AI model.

[1020] These features allow users to seamlessly manage and consult about their health within a physical store, and they can also receive expert advice in real time, contributing to improving their health.

[1021] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1022] Step 1:

[1023] Users access smart devices installed in physical stores and input health data (weight, number of steps, dietary content, sleep time, etc.).

[1024] Input: Health data entered by the user through their smart device.

[1025] Output: The entered data is sent to the server.

[1026] How it works: An input interface is displayed on the screen of the smart device, and the user enters their health information item by item. After completing the input, the data is sent to the server in real time.

[1027] Step 2:

[1028] The server stores the received health data in a database.

[1029] Input: Health data submitted in step 1.

[1030] Output: Health data is stored in a database.

[1031] What happens: The server executes a query to properly structure the received data and store it in a MySQL database.

[1032] Step 3:

[1033] The user uses a smart device to provide health consultation to the generated AI model.

[1034] Input: A question that the user types into their smart device.

[1035] Output: The answer generated by the generative AI model.

[1036] How it works: The user types a prompt into their smart device, such as "What healthy foods should I eat after exercise?", and it is sent to the server, which then passes the question to the generative AI model.

[1037] Step 4:

[1038] The server uses the generative AI model to generate answers to the user's questions.

[1039] Input: The question entered by the user in step 3.

[1040] Output: The correct answer to the question.

[1041] How it works: The server inputs a question as a prompt to a generative AI model (e.g., GPT), and the model generates the optimal answer to that question. The generated answer is returned to the server.

[1042] Step 5:

[1043] The server sends the generated answer back to the smart device and displays it to the user.

[1044] Input: The answer generated in step 4.

[1045] Output: The answer is displayed on the user's smart device.

[1046] How it works: The server sends the answer obtained from the generative AI model to the smart device, which immediately displays the answer to the user for confirmation.

[1047] Step 6:

[1048] If necessary, the user books an online consultation with a specialist.

[1049] Input: Booking request entered by user into smart device.

[1050] Output: Booking confirmation and schedule confirmation.

[1051] Specific operation: The user inputs the desired date and time for consultation with a specialist through their smart device and sends it to the server. The server checks the specialist's schedule and, if the appointment is confirmed, notifies the user's smart device of the information.

[1052] Step 7:

[1053] Online consultations with specialists are conducted, providing detailed advice on the user's health condition.

[1054] Input: User's health condition and consultation details at the time of appointment.

[1055] Output: Expert advice.

[1056] Specific operation: At the scheduled time, the expert and the user will be connected via video call, where the expert will check the user's health condition and provide advice.

[1057] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1058] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1059] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1060] [Third embodiment]

[1061] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1062] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1063] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1064] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1065] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1066] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1067] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1068] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1069] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1070] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1071] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1072] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1073] This invention is a health management system that provides free health consultations using generative AI to assist users in managing their health. The system collects input data from users, uses generative AI models to answer health questions, and even provides in-depth consultations with experts and online medical consultations.

[1074] System configuration

[1075] Server Features

[1076] 1. Database Management

[1077] The server receives health data entered by the user (e.g., weight, number of steps, dietary content, sleep time, etc.) and stores this data in a centrally managed database.

[1078] 2. Operation of generative AI

[1079] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[1080] 3. Collaboration with experts

[1081] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[1082] 4. IoT device data collection

[1083] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[1084] Device Features

[1085] 1. Data input interface

[1086] The device provides an interface where users can input their daily health data, which is then sent to a server and stored in a database.

[1087] 2. Health consultation chat function

[1088] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[1089] 3. IoT device integration

[1090] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[1091] User operations

[1092] 1. Initial Setup

[1093] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.).

[1094] 2. Data entry and consultation

[1095] Users can input their daily health data and ask the AI ​​for health advice, such as, "How balanced is your diet today?" or "What kind of exercise should I do?"

[1096] 3. Consult with a specialist and book an online consultation

[1097] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[1098] 4. Subscription

[1099] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[1100] Specific examples

[1101] 1. Example of initial settings

[1102] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[1103] 2. Examples of daily data entry and consultation

[1104] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[1105] 3. Example of collaboration with IoT devices

[1106] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[1107] 4. Consultations with specialists and examples of treatment

[1108] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[1109] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[1110] The processing flow will be explained below.

[1111] Step 1: Initial Setup

[1112] 1. The server monitors the installation of the application and accepts a connection request to the server when the application is first launched.

[1113] 2. The device displays an account creation screen to the user, prompting them to enter basic information such as their name, email address, and password.

[1114] 3. The device sends the entered basic information to the server.

[1115] 4. The server stores the received basic information in a database and creates an account.

[1116] 5. The server sends a notification to the device that the account has been created.

[1117] 6. The device will display a notification to the user that the account has been created.

[1118] Step 2: Daily entry of health data

[1119] 1. The user launches the app and enters daily health data such as weight, number of steps, diet, and sleep time.

[1120] 2. The device sends the entered health data to the server.

[1121] 3. The server stores the received health data in a database.

[1122] 4. The device will notify the user that the data has been entered and allow them to visually check the data in a graph.

[1123] Step 3: IoT device integration

[1124] 1. The user connects an IoT device such as a smartwatch or blood pressure monitor to the app.

[1125] 2. The device automatically collects health data from connected IoT devices.

[1126] 3. The device sends the collected data to the server.

[1127] 4. The server stores the received data in a database.

[1128] 5. The device notifies the user that the latest health data has been updated.

[1129] Step 4: Free health consultation

[1130] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[1131] 2. The device sends the user's question to the server.

[1132] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[1133] 4. The server sends the generated response to the terminal.

[1134] 5. The device displays the answer from the generating AI to the user.

[1135] Step 5: Consult with a specialist and online consultation

[1136] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[1137] 2. The device sends the user's request to the server.

[1138] 3. The server queries the expert's schedule to check available dates and times.

[1139] 4. The server sends the confirmed schedule information to the terminal.

[1140] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[1141] 6. The user selects the desired date and time, and the device sends the selection to the server.

[1142] 7. The server finalizes the booking and notifies the user and the expert.

[1143] 8. The user launches the app at the scheduled time and begins the online consultation.

[1144] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[1145] 10. The server records the medical treatment details and advice in a database.

[1146] Step 6: Subscribe

[1147] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[1148] 2. The device will display instructions to the user via push notification or email.

[1149] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[1150] 4. The terminal sends the entered payment information to the server.

[1151] 5. The server processes the payment information and updates the subscription status to paid.

[1152] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[1153] Through these steps, the system efficiently supports users in managing their health, providing a comprehensive range of services from free consultations to professional advice and medical treatment.

[1154] Example 1

[1155] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1156] In modern society, personal health management is an important topic, and many users need to understand their own health status and receive appropriate advice. However, collecting health data on a daily basis and utilizing it to receive effective advice requires specialized knowledge and expensive equipment, placing a heavy burden on the average user. In addition, there is a lack of systems that allow seamless in-depth consultations with experts or online medical consultations. This creates a challenge for users, preventing them from optimally managing their health.

[1157] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1158] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and conducting detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, and means for the health data to serve as the basis for analysis and recommendations based on the generative AI model, thereby enabling users to understand their own health status and receive appropriate advice without the need for specialized knowledge or expensive equipment.

[1159] "Health data from the user" refers to health-related data such as weight, number of steps, dietary details, and sleep time entered by the user.

[1160] A "generative AI model" is an artificial intelligence model that generates appropriate answers to health inquiries from users.

[1161] "Health consultation history" is a record of past health-related questions and answers asked by the user.

[1162] "Database" means an information management system for storing health data and health consultation history.

[1163] "Detailed consultations and online medical treatment with experts" refers to specific health consultations and remote medical treatments that users receive from experts.

[1164] An "IoT device" is a device, such as a smartwatch or blood pressure monitor, that is connected to the Internet and collects health data.

[1165] "Analysis and recommendation" is the process of analyzing collected health data and providing appropriate health advice to the user.

[1166] "Subscription status" is information that indicates the user's subscription service usage status and expiration date.

[1167] "Real-time communication means" refers to communication technologies that allow users to ask questions using generative AI models and receive instant answers.

[1168] The present invention is a health consultation system that utilizes a generative AI model to support users' health management. This system functions in cooperation with users, terminals, and a server.

[1169] Hardware and software used

[1170] 1. Hardware

[1171] Device: Smartphone or tablet

[1172] Server: High-performance cloud server

[1173] IoT devices: smartwatches, blood pressure monitors, etc.

[1174] 2. Software

[1175] Application: Health management app

[1176] Database: MySQL

[1177] Generative AI model: OpenAI's GPT-4

[1178] Scheduling Software: Professional scheduling system

[1179] Communication protocol: REST API

[1180] Program processing

[1181] Server Features

[1182] The server receives health data sent by users and stores it in a database. Specifically, it uses a REST API to collect data such as the user's weight, number of steps, dietary habits, and sleep time. This data is stored in a MySQL database and managed centrally.

[1183] Furthermore, the server runs the generative AI model GPT-4, which generates appropriate answers to health inquiries from users. For example, if a user asks, "How is your diet balanced today?", GPT-4 will generate an answer such as, "Your diet today is high in carbohydrates. Eating a few more vegetables would improve the balance."

[1184] The server also accepts requests for in-depth consultations with specialists and online consultations, and uses dedicated scheduling software to manage the specialists' schedules and confirm appropriate appointments.

[1185] Device Features

[1186] The device provides an interface that allows users to input their daily health data. The input data is sent to a server in real time and stored in a database. It also provides an interface for chatting with the generating AI for health consultations. When a user accesses the chat screen and asks, "What kind of exercise should I do?", the answer from the generating AI is displayed instantly.

[1187] The device also connects to IoT devices such as smartwatches and blood pressure monitors, automatically collecting data from these devices and sending it to a server. For example, heart rate data obtained from a smartwatch is automatically sent to the server.

[1188] User operations

[1189] First, users download the app and create an account. They enter basic health information (age, weight, medical history, allergies, health goals, etc.). Next, they can enter their daily health data and ask the AI ​​for health advice.

[1190] Users can also make appointments for detailed consultations with specialists or online medical consultations, and receive specific health advice via video call at the scheduled time. After the free trial period ends, users can continue to receive detailed advice and online medical consultations by subscribing.

[1191] Specific examples

[1192] 1. Example of initial settings

[1193] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[1194] 2. Examples of daily data entry and consultation

[1195] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[1196] 3. Example of collaboration with IoT devices

[1197] When a user is wearing a smartwatch, the heart rate data acquired from the smartwatch is automatically sent to the device and stored on the server.

[1198] 4. Consultations with specialists and examples of treatment

[1199] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[1200] keyword

[1201] Generative AI model, prompt sentence

[1202] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1203] Step 1:

[1204] Initial Setup

[1205] The user installs and launches the health management app. Next, they select "New Registration" and enter basic health information such as their name, age, sex, weight, height, allergy information, and health goals. The entered information is sent from the device to the server, which receives it and stores it in a database.

[1206] Input: Name, age, gender, weight, height, allergy information, health goals

[1207] Output: User information is saved in the database

[1208] Step 2:

[1209] Routine data entry

[1210] Users enter their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the app. The data is sent in real time from the device to the server, which receives it and stores it in a database.

[1211] Input: weight, diet, steps, sleep time

[1212] Output: Health data is stored in a database

[1213] Step 3:

[1214] Using generative AI models for health consultations

[1215] A user uses the app's consultation function to ask a health question in chat format. For example, the user might ask, "How balanced is my diet today?" This question is sent from the device to the server. The server then invokes a generative AI model (GPT-4) to generate an answer to the question. The generated answer is sent to the user's device and displayed on the chat screen.

[1216] Input: User question (e.g., "How balanced is your diet today?")

[1217] Output: Answer from the generative AI model (e.g., "Your meal today is high in carbohydrates. Adding a few more vegetables would improve the balance.")

[1218] Step 4:

[1219] Book an in-depth consultation with a specialist or online consultation

[1220] A user uses the app's "Consultation with a Specialist" feature to book a consultation or consultation. For example, they request, "I'd like to book a consultation with a nutritionist this weekend at 3:00 PM." This request is sent to the server, which uses dedicated scheduling software to manage the specialist's schedule and confirm the appointment. The appointment details are sent to the user's device and notified.

[1221] Input: Appointment request (e.g., "I'd like to schedule a consultation with a nutritionist this weekend at 3pm")

[1222] Output: Confirmation of booking

[1223] Step 5:

[1224] Data linkage with IoT devices

[1225] When a user uses IoT devices such as a smartwatch or blood pressure monitor, health data is collected in real time from these devices. The data is transmitted to the device via Bluetooth or Wi-Fi. The device then sends the data to a server, which stores it in a database. For example, heart rate data from a smartwatch is automatically collected and recorded.

[1226] Input: Health data from IoT devices (e.g., heart rate, blood pressure)

[1227] Output: Health data is stored in a database

[1228] Step 6:

[1229] Manage your subscription

[1230] The server manages the user's subscription status. Before the free period ends, the server notifies the user and prompts them to subscribe. The user selects the desired plan within the app and enters their credit card information to complete the payment. The server verifies the payment information and updates the subscription status.

[1231] Input: Subscription request, credit card information

[1232] Output: Updated subscription status, payment confirmation notification

[1233] (Application example 1)

[1234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1235] In modern society, many health problems, such as lifestyle-related diseases and obesity, are on the rise. In particular, busy lifestyles make it difficult to choose healthy meals. Other issues include the inability to easily access health consultations and medical treatment from experts, and a lack of appropriate advice based on individual health data. In response to these issues, there is a demand for fast and accurate health management support.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1237] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, means for proposing health menus based on user data, and means for conducting health consultations with the user in a chat format. This allows users to receive dietary suggestions based on their own health information and easily consult with experts, thereby enabling them to achieve a healthy lifestyle.

[1238] The "means for receiving health data from the user" is an interface for sending and receiving health-related information (for example, weight, dietary content, amount of exercise, etc.) input by the user to the server.

[1239] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate answers to health-related questions asked by users.

[1240] The "database means for storing health data and health consultation history" is a database system for long-term storage and management of health data entered by users and the health consultation response history provided by the generation AI.

[1241] "Means for booking and conducting detailed consultations and online medical consultations with specialists" refers to a system that allows users to book and conduct detailed health consultations and online medical consultations with specialists.

[1242] "Means for automatically collecting health data in conjunction with IoT devices" refers to a function that works in conjunction with smartwatches and everyday health measurement devices to collect health data automatically obtained from them and send it to a server.

[1243] "Means for suggesting healthy menus based on user data" refers to a function in which AI uses health data provided by the user to generate and suggest healthy meal menus based on that data.

[1244] The "means for providing health consultations to users in chat format" is a system that allows users and generative AI models to exchange chat-style messages in real time and respond to health-related questions.

[1245] This invention is a system that supports users' health management, and aims to collect health data of individual users through collaboration with smartphones and IoT devices, and provide health consultations and dietary suggestions using a generative AI model. A detailed description of the system is provided below.

[1246] System configuration

[1247] This invention is mainly composed of three main elements: a server, a terminal, and a user.

[1248] Server Features

[1249] The server has the following main functions:

[1250] 1. Means of receiving health data from users:

[1251] The server receives basic health information (age, weight, health goals, allergy information, etc.) entered by the user through a smartphone application and stores it in a database.

[1252] 2. How to use generative AI models to answer health questions:

[1253] Leveraging OpenAI's ChatCompletion API, the system automatically generates answers to health-related questions from users. For example, if a user asks, "How balanced is your diet today?", the generative AI model instantly provides an answer.

[1254] 3. Database means for storing health data and health consultation history:

[1255] The server uses SQLite or another database management system to store the health data provided by the user and the history of answers provided by the generative AI model.

[1256] 4. How to book and conduct in-depth consultations with specialists and online consultations:

[1257] We provide a reservation system that has a function for linking with experts and allows users to consult with experts online, allowing users to easily obtain professional advice.

[1258] 5. Means of automatically collecting health data in conjunction with IoT devices:

[1259] It has the ability to automatically collect data from IoT devices such as smartwatches and blood pressure monitors, and send it to a server for storage.

[1260] 6. How to suggest healthy menus based on user data:

[1261] Based on the data provided by the user, the generative AI model will suggest healthy meals, for example, if the user's goal is to lose weight, it will provide a suitable meal plan.

[1262] 7. Means of providing health consultations to users through chat:

[1263] Users can chat with the generated AI model in real time via a smartphone app and ask for health advice.

[1264] Device Features

[1265] The device (smartphone, etc.) has the following functions:

[1266] 1. Data input interface:

[1267] It provides an interface that allows users to manually input their health data, making it easy to record daily weight, diet, exercise, etc.

[1268] 2. Health consultation chat function:

[1269] It provides an interface that allows users to chat with the generated AI and receive health advice.

[1270] 3. IoT device integration:

[1271] It works in conjunction with IoT devices such as smartwatches and blood pressure monitors, and has the ability to automatically collect data and send it to a server.

[1272] User operations

[1273] The user uses this system by following the steps below:

[1274] 1. Initial Setup:

[1275] Install the app and create an account, which starts by entering basic health information.

[1276] 2. Data entry and consultation:

[1277] Enter your daily health data and ask the AI ​​for health advice. For example, you can ask, "Could you please give me some advice about my recent diet?"

[1278] 3. Specialist consultations and online consultation appointments:

[1279] You can book consultations and appointments with specialists and receive professional advice via video call.

[1280] 4. Subscription:

[1281] After the free trial period ends, you can subscribe to continue using the service for detailed advice and online consultations.

[1282] Specific examples

[1283] Here is a concrete example of its use:

[1284] 1. Example of suggesting healthy menus based on user data:

[1285] After the user enters their health goals, they can ask, "How is your diet balanced today?" and the generative AI will suggest, "You're eating too many carbohydrates today. Try to eat more vegetables at your next meal."

[1286] 2. Example prompt:

[1287] User data: Age: 30, Weight: 70kg, Goal: Lose weight, Allergy: Peanuts

[1288] User Question: How balanced is your diet today?

[1289] Please provide your answer:

[1290] This allows users to understand their health status in real time and receive advice on appropriate diet and exercise.

[1291] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1292] Step 1: Initial Setup

[1293] Users download the smartphone app and create an account. The app prompts users to enter basic health information (age, weight, health goals, allergy information, etc.), which is then sent to a server and stored in a database.

[1294] Input: User's health information (age, weight, health goals, allergy information)

[1295] Data processing: Send input data to the server in JSON format

[1296] Output: User information is saved in the database

[1297] Step 2: Data entry

[1298] Users enter their daily health data (weight, diet, exercise, etc.) into the app, which then sends this data to a server and stores it in a database.

[1299] Input: User's daily health data (weight, diet, exercise)

[1300] Data processing: Send input data to the server in JSON format

[1301] Output: Daily health data is stored in a database

[1302] Step 3: IoT device integration

[1303] The app connects with users' IoT devices such as smartwatches and blood pressure monitors, allowing data collected from the devices to be automatically sent to the server.

[1304] Input: Health data from IoT devices (heart rate, steps, etc.)

[1305] Data processing: Automatically send data to the server

[1306] Output: Data from IoT devices is stored in a database

[1307] Step 4: Conducting health consultations

[1308] Users use the app's chat function to ask questions about their health. When a user enters a question, the question is sent to the server, which uses a generative AI model to generate an answer and sends it back to the user.

[1309] Input: User's health consultation question

[1310] Data computation: Generative AI models generate answers based on prompts

[1311] Output: The answer is returned to the user

[1312] Step 5: Healthy menu suggestions

[1313] When a user requests a healthy meal plan, they input their request into the app, and the server uses a generative AI model based on the user data to suggest an appropriate meal plan.

[1314] Input: User's health menu request

[1315] Data arithmetic: Generative AI models generate menus based on prompts

[1316] Output: A suggested meal menu is displayed to the user

[1317] Step 6: Book a consultation with a specialist

[1318] When a user wants to consult with a specialist, they make a reservation through the app. The server checks the specialist's schedule and confirms the reservation.

[1319] Input: User's consultation appointment request

[1320] Data processing: Matching booking requests with specialist schedules

[1321] Output: A confirmation of the booking is sent to the user.

[1322] Step 7: Manage your subscriptions

[1323] After the free trial period ends, users can subscribe to continue receiving detailed advice and treatment features. The app will manage users' subscription status and notify them when the trial period ends.

[1324] Input: Free period end information

[1325] Data Processing: Subscription Status Update

[1326] Output: A prompt to subscribe is sent to the user

[1327] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1328] This invention is a health management system that provides free health consultations utilizing generative AI and an emotion engine to assist users in managing their health. The system collects input data from users, uses a generative AI model and an emotion engine to answer health questions, and even provides in-depth consultations with experts and online medical treatments.

[1329] System configuration

[1330] Server Features

[1331] 1. Database Management

[1332] The server receives health data (e.g., weight, number of steps, dietary content, sleep time, etc.) and emotional data entered by the user and stores this data in a centrally managed database.

[1333] 2. Operation of generative AI

[1334] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[1335] 3. Operating the Emotion Engine

[1336] The server runs an emotion engine, analyzes the data input by the user and the content of the health consultation, and recognizes the user's emotional state. This recognized emotional data is also stored in the database.

[1337] 4. Collaboration with experts

[1338] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[1339] 5. IoT device data collection

[1340] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[1341] Device Features

[1342] 1. Data input interface

[1343] The device provides an interface through which users can input their daily health and emotional data, which is then sent to a server and stored in a database.

[1344] 2. Health consultation chat function

[1345] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[1346] 3. Emotion data input assistance

[1347] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends this data to a server.

[1348] 4. IoT device integration

[1349] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[1350] User operations

[1351] 1. Initial Setup

[1352] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.). The app also configures facial recognition and voice analysis settings, and sets up how to input emotional data.

[1353] 2. Data entry and consultation

[1354] Users input their daily health data and ask the AI ​​for health advice. For example, they can ask questions such as, "How balanced is my diet today?" or "What kind of exercise should I do?" The emotion engine analyzes the user's emotions and can suggest, for example, "How to relax if you're feeling stressed."

[1355] 3. Consult with a specialist and book an online consultation

[1356] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[1357] 4. Subscription

[1358] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[1359] Specific examples

[1360] 1. Example of initial settings

[1361] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. They then configure facial recognition and voice analysis. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg" to enable the facial recognition feature.

[1362] 2. Examples of daily data entry and consultation

[1363] The user enters their daily weight and dietary information and asks in chat, "How is your diet balance today?" The generative AI responds, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance." The emotion engine recognizes signs of stress from the user's facial expressions and voice and provides additional advice, such as "Try taking deep breaths to relax."

[1364] 3. Example of collaboration with IoT devices

[1365] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[1366] 4. Consultations with specialists and examples of treatment

[1367] A user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist contact each other via video call, and receive specific dietary advice. The user's emotional state is also provided to the nutritionist, enabling more personalized advice.

[1368] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[1369] The processing flow will be explained below.

[1370] Step 1: Initial Setup

[1371] 1. The user downloads and installs the app.

[1372] 2. The device will display the account creation screen when you first start it up.

[1373] 3. The user creates an account by entering their name, email address, password, age, weight, gender, and allergy information.

[1374] 4. The terminal sends the entered information to the server.

[1375] 5. The server stores the received information in its database and completes the account creation.

[1376] 6. The server sends a notification to the device that account creation is complete.

[1377] 7. The device displays a notification to the user that the account has been created.

[1378] Step 2: Configure facial recognition and voice analysis

[1379] 1. The device will display the initial setting screen for the emotion engine.

[1380] 2. The user configures the facial recognition and voice analysis settings (for example, by moving their face towards the camera or reading a specific phrase aloud).

[1381] 3. The device sends the settings to the server.

[1382] 4. The server saves the emotion engine configuration data in the database.

[1383] Step 3: Daily entry of health and emotional data

[1384] 1. The user launches the app and enters their daily health data, such as weight, number of steps, diet, and sleep time.

[1385] 2. The terminal sends the entered data to the server.

[1386] 3. The server stores the received data in a database.

[1387] 4. The device collects emotional data from the user's facial recognition and voice, and also sends this to the server.

[1388] 5. The server stores the emotion data in a database.

[1389] Step 4: Free health consultation

[1390] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[1391] 2. The device sends the user's question to the server.

[1392] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[1393] 4. The server sends the generated response to the terminal.

[1394] 5. The server starts the emotion engine and analyzes the user's emotion data.

[1395] 6. The server customizes the answer based on the sentiment data and generates appropriate follow-up advice.

[1396] 7. The server sends a customized answer and additional advice to the device.

[1397] 8. The device displays the user the answer from the generative AI and additional advice based on the emotional data.

[1398] Step 5: Consult with a specialist and online consultation

[1399] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[1400] 2. The device sends the user's request to the server.

[1401] 3. The server queries the expert's schedule to check available dates and times.

[1402] 4. The server sends the confirmed schedule information to the terminal.

[1403] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[1404] 6. The user selects the desired date and time, and the device sends the selection to the server.

[1405] 7. The server confirms the booking and notifies the user and the expert.

[1406] 8. The user launches the app at the scheduled time and begins the online consultation.

[1407] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[1408] 10. The server records the medical treatment details and advice in a database.

[1409] Step 6: Subscribe

[1410] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[1411] 2. The device will display instructions to the user via push notification or email.

[1412] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[1413] 4. The terminal sends the entered payment information to the server.

[1414] 5. The server processes the payment information and updates the subscription status to paid.

[1415] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[1416] Through these steps, the system efficiently supports users in managing their health and provides a comprehensive approach that includes emotional data. The emotion engine analyzes the user's emotional state and customizes the generative AI model's answers, resulting in more personalized healthcare services.

[1417] Example 2

[1418] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1419] Conventional health management systems have difficulty collecting not only users' health data but also their emotional data and providing appropriate advice based on that data. Furthermore, the procedures for making appointments for consultations with specialists or online medical consultations are often complicated, placing a burden on users. Furthermore, insufficient integration with IoT devices results in inefficient data collection. This makes it difficult for users to effectively manage their health on a daily basis.

[1420] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1421] In this invention, the server includes means for receiving health data and emotional data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing health data and health consultation history, means for analyzing the user's emotional state using an emotion engine, means for booking and implementing detailed consultations with experts and online medical treatments, and means for automatically collecting health data in cooperation with IoT devices. This enables comprehensive health management based on health data and emotional data, and smooth cooperation with experts reduces the burden on the user and enables effective health management.

[1422] "Health data from the user" refers to data relating to the user's daily health condition, such as weight, number of steps, dietary content, and sleep time, which the user inputs himself or herself.

[1423] "Emotional data" is data about a user's emotional state collected through facial recognition and voice analysis of the user.

[1424] A "generative AI model" is an artificial intelligence model that generates answers in natural language in response to health inquiries from users. An example of this would be a chatbot.

[1425] A "means for answering health inquiries" is a method or system that enables a generative AI model to provide appropriate answers to health inquiries from users.

[1426] The "database means" is a data management system for storing the user's health data and health consultation history.

[1427] An "emotion engine" is a technology that analyzes input data and dialogue content from users to recognize their emotional state.

[1428] "Means for booking and conducting in-depth consultations and online medical consultations with specialists" refers to systems and functions that allow users to book a date and time for a consultation or medical consultation with a specialist and to conduct that appointment online.

[1429] "Means for automatically collecting health data in cooperation with IoT devices" refers to a method or system for automatically acquiring health data from IoT devices such as smartwatches and blood pressure monitors and transmitting that data to a server.

[1430] The present invention is a health management system that provides a free health consultation function utilizing a generative AI model and an emotion engine to assist users in managing their health. The system collects health and emotion data from users, uses the generative AI model and the emotion engine to answer health questions, and also provides detailed consultations with experts and online medical treatments. The following describes in detail the embodiments of the present invention.

[1431] server

[1432] The server performs the following functions:

[1433] 1. Data Collection and Management

[1434] The server receives the health and emotion data sent from the device and stores them in a centralized database, which is managed using MySQL or PostgreSQL, for example.

[1435] 2. Operation of generative AI models

[1436] When a health consultation from a user arrives at the server, the server generates a request to a generative AI model (e.g., GPT-4) and creates and sends an appropriate prompt. For example, in response to the question, "How is your diet balance today?", the server generates a prompt such as, "Please rate the user's diet balance today."

[1437] 3. Operating the Emotion Engine

[1438] The server runs an emotion engine (e.g., Affectiva or Azure Emotion Recognition API) that analyzes the user's input data and dialogue to recognize their emotional state. The recognized emotion data is also stored in a database.

[1439] 4. Scheduling with an expert

[1440] The server manages the schedules of the experts and confirms the reservation based on the reservation request from the user. The scheduling is done using the Google Calendar API.

[1441] 5. IoT device data collection

[1442] The server connects to IoT devices (e.g., smartwatches and blood pressure monitors) and automatically collects health data from these devices and stores it in a database. Firebase and AWS IoT are used here.

[1443] Terminal

[1444] The terminal provides the following user interface:

[1445] 1. Data input interface

[1446] The device provides an interface where users can input their daily health and emotional data. The input data is immediately sent to a server and stored in a database. This is implemented using a mobile app developed using Xamarin and React Native.

[1447] 2. Health consultation chat function

[1448] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[1449] 3. Auxiliary input of emotion data

[1450] The device performs facial recognition and voice analysis of the user, collects emotion data, and sends it to a server using OpenCV and Google Cloud Vision API.

[1451] 4. IoT device integration

[1452] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[1453] User operations

[1454] 1. Initial settings of the app

[1455] Users download the app and create an account. They enter basic health information and configure facial recognition and voice analysis. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, and want to lose 5 kg," and then activate the facial recognition feature.

[1456] 2. Daily data entry and consultation

[1457] Users input their daily health data and provide health advice to the generative AI model. For example, they can ask a question via chat, such as, "How is the balance of your diet today?" The generative AI will respond, "Your diet today is high in carbohydrates. If you eat a few more vegetables, it will be more balanced."

[1458] 3. Consultation and appointment with a specialist

[1459] The user can make an appointment for a detailed consultation or online medical consultation with a specialist and then complete the appointment. For example, the user can request, "I would like to make an appointment with a nutritionist at 3:00 PM this weekend," and the server will check the specialist's schedule and confirm the appointment.

[1460] 4. Subscription

[1461] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[1462] In this way, the present invention utilizes a generative AI model and an emotion engine to provide a system that comprehensively supports users' health management. As a specific example, if a user inputs their daily weight and dietary details and asks in chat, "How is the balance of your diet today?", the generative AI can respond, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance," and the emotion engine can recognize signs of stress from the user's facial expressions and voice and provide additional advice, such as, "Try taking deep breaths to relax."

[1463] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1464] Step 1: User creates account

[1465] Users download the app and enter basic health information such as name, age, weight, gender, and allergies on the account creation screen. The data is then saved on the device and sent to the server, which then records the received data in a database.

[1466] Step 2: User performs initial setup

[1467] The user configures facial recognition and voice analysis settings. The device collects the user's facial images and voice samples and sends them to the server, where they are analyzed using an emotion engine and the results are stored in a database.

[1468] Step 3: Enter your daily health data

[1469] Users input their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the device's input interface. The device then sends the data to the server, which then stores the health data in a database.

[1470] Step 4: Enter your health consultation information

[1471] The user inputs a question using the health consultation chat function on the device. For example, they input a question such as, "How balanced is your diet today?" The device then sends this question to the server. The server receives the question and generates an appropriate prompt based on the user's input data.

[1472] Step 5: Request to the generative AI model

[1473] The server sends a prompt to the generative AI model. For example, it generates a prompt such as "Please rate the balance of the user's diet today" and sends it to the generative AI model. The generative AI model generates an answer based on this.

[1474] Step 6: Get the answer from the generative AI model

[1475] The generative AI model generates an answer based on the prompt and sends it back to the server, which then converts the answer into an appropriate format, such as "Your meal today is high in carbohydrates. Adding a few more vegetables would make it more balanced."

[1476] Step 7: Send the answer to the user

[1477] The server sends the answer received from the generative AI model to the device, which displays the answer on the user's chat screen. The user can view the answer and use it to manage their daily health.

[1478] Step 8: Analyze the sentiment data

[1479] The device constantly recognizes the user's face and analyzes their voice to collect emotional data. This data is sent to a server and analyzed by an emotion engine. The analysis results are stored in a database and used to provide advice to the user as needed.

[1480] Step 9: Book an appointment online with a specialist

[1481] The user inputs a request into the device to schedule a consultation with a specialist or an online medical consultation. For example, the user might request, "I'd like to schedule a consultation with a nutritionist this weekend at 3:00 PM." The device then sends this request to the server.

[1482] Step 10: Scheduling and Confirmation

[1483] The server checks the specialist's scheduling system and confirms the appointment based on the user's request. If the appointment is confirmed, the server notifies the user. The user can then have an online consultation with the specialist at the scheduled date and time.

[1484] Step 11: Conducting online consultations

[1485] Based on the scheduled date and time, users can receive online medical treatment and detailed health consultations through video calls with specialists. The server records the session information in a database for future reference.

[1486] Step 12: Manage your subscriptions

[1487] After the free trial period ends, users can choose whether or not to subscribe within the app. The device sends the user's subscription status to the server, which stores the information in a database. If users wish to continue using the app, a feature is provided to prompt them to subscribe.

[1488] (Application example 2)

[1489] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1490] Conventional health management systems collect and manage users' health data and can answer health questions using generative AI, but there were almost no systems designed for use in physical stores. As a result, there was a lack of a way for customers to receive health consultations easily and quickly in physical stores. It was also difficult to provide appropriate health advice in real time in physical stores.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1492] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with specialists and online medical examinations, means for automatically collecting health data in cooperation with IoT devices, and means for providing health consultations and advice to users via smart devices installed in physical stores. This enables customers to easily and quickly receive health consultations in physical stores and appropriate health advice in real time.

[1493] The "means for receiving health data from the user" is an interface for transmitting health data such as weight, number of steps, dietary details, and sleep time input by the user to the server.

[1494] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate appropriate answers to users' health-related questions.

[1495] The "database means for storing health data and health consultation history" is a database system for centrally managing and storing information about health data and health consultations entered by users.

[1496] "Means for reserving and conducting detailed consultations and online medical consultations with specialists" is a function that enables users to reserve and conduct detailed health consultations and online medical consultations with specialists.

[1497] "Means for automatically collecting health data in cooperation with IoT devices" refers to a function that automatically collects health data from IoT devices such as smartwatches and blood pressure monitors and sends it to a server.

[1498] "Means for providing health consultations and advice to users via smart devices installed in physical stores" refers to an interface that allows users to conduct health consultations and receive advice from the generating AI using smart glasses, head-mounted displays, tablets, or smartphones installed in physical stores.

[1499] This invention is a system that realizes health management support for customers in physical stores. Using smart devices installed in physical stores, customers can receive health consultations easily and quickly.

[1500] System configuration

[1501] This system is mainly composed of three entities: a server, a terminal, and a user.

[1502] Server Features

[1503] 1. Means of receiving health data from users

[1504] It receives health data such as weight, number of steps, dietary intake, and sleep time entered by the user and stores it in a database. This requires an internet connection and stable communication.

[1505] 2. A method for answering health questions from users using a generative AI model

[1506] The server runs a generative AI model (e.g., GPT) to automatically generate prompt and appropriate answers to health inquiries from users. Responses to questions are generated using prompt sentences.

[1507] 3. Database means for storing health data and health consultation history

[1508] The received health data and health consultation information are stored in a centralized database system, using database solutions such as MySQL or MongoDB.

[1509] 4. A way to schedule and conduct detailed consultations with specialists and online medical consultations

[1510] The server manages the schedules of specialists, allows users to book detailed health consultations and online consultations, and integrates video calling functionality to provide consultations and advice.

[1511] 5. Means of automatically collecting health data in conjunction with IoT devices

[1512] Health data is automatically collected from IoT devices such as smartwatches and blood pressure monitors and stored in a database.

[1513] Device Features

[1514] 1. Data input interface

[1515] It provides an interface where users can input their health data using smart devices (smart glasses, head-mounted displays, tablets, smartphones) in physical stores, and the input data is sent to a server.

[1516] 2. Health consultation chat function

[1517] The device provides an interface for users to chat with the AI ​​to discuss health issues. Questions are sent to the server in real time, and the AI ​​responds immediately.

[1518] Examples of prompts:

[1519] "What healthy foods should I eat after a workout?"

[1520] 3. Emotion data input assistance

[1521] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends it to a server, allowing the AI ​​to reflect the user's emotional state in its responses.

[1522] User operations

[1523] 1. Initial Setup

[1524] Users download the app, create an account, enter basic health information, configure facial recognition and voice analysis settings, and set up how to input emotional data.

[1525] 2. Data entry and consultation

[1526] Users can use smart devices in physical stores to input their daily health data and also consult with the AI ​​to receive appropriate health advice.

[1527] Examples:

[1528] When a user types in, "What healthy foods should I eat after exercise?", the generative AI will answer, "I recommend a protein bar that is effective after exercise."

[1529] 3. Consult with a specialist and book an online consultation

[1530] Users can easily book and schedule detailed consultations with specialists or online medical consultations using the server's reservation management system.

[1531] Examples of technology implementation

[1532] Hardware Usage

[1533] Smart glasses, head-mounted displays, tablets, and smartphones will be installed in the physical store, and these devices will act as interfaces with users.

[1534] Software Usage

[1535] It uses Python's Flask as the server backend, MySQL as the database, and the open source GPT as the generative AI model.

[1536] These features allow users to seamlessly manage and consult about their health within a physical store, and they can also receive expert advice in real time, contributing to improving their health.

[1537] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1538] Step 1:

[1539] Users access smart devices installed in physical stores and input health data (weight, number of steps, dietary content, sleep time, etc.).

[1540] Input: Health data entered by the user through their smart device.

[1541] Output: The entered data is sent to the server.

[1542] How it works: An input interface is displayed on the screen of the smart device, and the user enters their health information item by item. After completing the input, the data is sent to the server in real time.

[1543] Step 2:

[1544] The server stores the received health data in a database.

[1545] Input: Health data submitted in step 1.

[1546] Output: Health data is stored in a database.

[1547] What happens: The server executes a query to properly structure the received data and store it in a MySQL database.

[1548] Step 3:

[1549] The user uses a smart device to provide health consultation to the generated AI model.

[1550] Input: A question that the user types into their smart device.

[1551] Output: The answer generated by the generative AI model.

[1552] How it works: The user types a prompt into their smart device, such as "What healthy foods should I eat after exercise?", and it is sent to the server, which then passes the question to the generative AI model.

[1553] Step 4:

[1554] The server uses the generative AI model to generate answers to the user's questions.

[1555] Input: The question entered by the user in step 3.

[1556] Output: The correct answer to the question.

[1557] How it works: The server inputs a question as a prompt to a generative AI model (e.g., GPT), and the model generates the optimal answer to that question. The generated answer is returned to the server.

[1558] Step 5:

[1559] The server sends the generated answer back to the smart device and displays it to the user.

[1560] Input: The answer generated in step 4.

[1561] Output: The answer is displayed on the user's smart device.

[1562] How it works: The server sends the answer obtained from the generative AI model to the smart device, which immediately displays the answer to the user for confirmation.

[1563] Step 6:

[1564] If necessary, the user books an online consultation with a specialist.

[1565] Input: Booking request entered by user into smart device.

[1566] Output: Booking confirmation and schedule confirmation.

[1567] Specific operation: The user inputs the desired date and time for consultation with a specialist through their smart device and sends it to the server. The server checks the specialist's schedule and, if the appointment is confirmed, notifies the user's smart device of the information.

[1568] Step 7:

[1569] Online consultations with specialists are conducted, providing detailed advice on the user's health condition.

[1570] Input: User's health condition and consultation details at the time of appointment.

[1571] Output: Expert advice.

[1572] Specific operation: At the scheduled time, the expert and the user will be connected via video call, where the expert will check the user's health condition and provide advice.

[1573] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1574] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1575] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1576] [Fourth embodiment]

[1577] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1578] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1579] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1580] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1581] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1583] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1584] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1585] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1586] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1587] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1588] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1589] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1590] This invention is a health management system that provides free health consultations using generative AI to assist users in managing their health. The system collects input data from users, uses generative AI models to answer health questions, and even provides in-depth consultations with experts and online medical consultations.

[1591] System configuration

[1592] Server Features

[1593] 1. Database Management

[1594] The server receives health data entered by the user (e.g., weight, number of steps, dietary content, sleep time, etc.) and stores this data in a centrally managed database.

[1595] 2. Operation of generative AI

[1596] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[1597] 3. Collaboration with experts

[1598] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[1599] 4. IoT device data collection

[1600] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[1601] Device Features

[1602] 1. Data input interface

[1603] The device provides an interface where users can input their daily health data, which is then sent to a server and stored in a database.

[1604] 2. Health consultation chat function

[1605] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[1606] 3. IoT device integration

[1607] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[1608] User operations

[1609] 1. Initial Setup

[1610] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.).

[1611] 2. Data entry and consultation

[1612] Users can input their daily health data and ask the AI ​​for health advice, such as, "How balanced is your diet today?" or "What kind of exercise should I do?"

[1613] 3. Consult with a specialist and book an online consultation

[1614] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[1615] 4. Subscription

[1616] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[1617] Specific examples

[1618] 1. Example of initial settings

[1619] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[1620] 2. Examples of daily data entry and consultation

[1621] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[1622] 3. Example of collaboration with IoT devices

[1623] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[1624] 4. Consultations with specialists and examples of treatment

[1625] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[1626] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[1627] The processing flow will be explained below.

[1628] Step 1: Initial Setup

[1629] 1. The server monitors the installation of the application and accepts a connection request to the server when the application is first launched.

[1630] 2. The device displays an account creation screen to the user, prompting them to enter basic information such as their name, email address, and password.

[1631] 3. The device sends the entered basic information to the server.

[1632] 4. The server stores the received basic information in a database and creates an account.

[1633] 5. The server sends a notification to the device that the account has been created.

[1634] 6. The device will display a notification to the user that the account has been created.

[1635] Step 2: Daily entry of health data

[1636] 1. The user launches the app and enters daily health data such as weight, number of steps, diet, and sleep time.

[1637] 2. The device sends the entered health data to the server.

[1638] 3. The server stores the received health data in a database.

[1639] 4. The device will notify the user that the data has been entered and allow them to visually check the data in a graph.

[1640] Step 3: IoT device integration

[1641] 1. The user connects an IoT device such as a smartwatch or blood pressure monitor to the app.

[1642] 2. The device automatically collects health data from connected IoT devices.

[1643] 3. The device sends the collected data to the server.

[1644] 4. The server stores the received data in a database.

[1645] 5. The device notifies the user that the latest health data has been updated.

[1646] Step 4: Free health consultation

[1647] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[1648] 2. The device sends the user's question to the server.

[1649] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[1650] 4. The server sends the generated response to the terminal.

[1651] 5. The device displays the answer from the generating AI to the user.

[1652] Step 5: Consult with a specialist and online consultation

[1653] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[1654] 2. The device sends the user's request to the server.

[1655] 3. The server queries the expert's schedule to check available dates and times.

[1656] 4. The server sends the confirmed schedule information to the terminal.

[1657] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[1658] 6. The user selects the desired date and time, and the device sends the selection to the server.

[1659] 7. The server finalizes the booking and notifies the user and the expert.

[1660] 8. The user launches the app at the scheduled time and begins the online consultation.

[1661] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[1662] 10. The server records the medical treatment details and advice in a database.

[1663] Step 6: Subscribe

[1664] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[1665] 2. The device will display instructions to the user via push notification or email.

[1666] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[1667] 4. The terminal sends the entered payment information to the server.

[1668] 5. The server processes the payment information and updates the subscription status to paid.

[1669] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[1670] Through these steps, the system efficiently supports users in managing their health, providing a comprehensive range of services from free consultations to professional advice and medical treatment.

[1671] Example 1

[1672] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1673] In modern society, personal health management is an important topic, and many users need to understand their own health status and receive appropriate advice. However, collecting health data on a daily basis and utilizing it to receive effective advice requires specialized knowledge and expensive equipment, placing a heavy burden on the average user. In addition, there is a lack of systems that allow seamless in-depth consultations with experts or online medical consultations. This creates a challenge for users, preventing them from optimally managing their health.

[1674] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1675] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and conducting detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, and means for the health data to serve as the basis for analysis and recommendations based on the generative AI model, thereby enabling users to understand their own health status and receive appropriate advice without the need for specialized knowledge or expensive equipment.

[1676] "Health data from the user" refers to health-related data such as weight, number of steps, dietary details, and sleep time entered by the user.

[1677] A "generative AI model" is an artificial intelligence model that generates appropriate answers to health inquiries from users.

[1678] "Health consultation history" is a record of past health-related questions and answers asked by the user.

[1679] "Database" means an information management system for storing health data and health consultation history.

[1680] "Detailed consultations and online medical treatment with experts" refers to specific health consultations and remote medical treatments that users receive from experts.

[1681] An "IoT device" is a device, such as a smartwatch or blood pressure monitor, that is connected to the Internet and collects health data.

[1682] "Analysis and recommendation" is the process of analyzing collected health data and providing appropriate health advice to the user.

[1683] "Subscription status" is information that indicates the user's subscription service usage status and expiration date.

[1684] "Real-time communication means" refers to communication technologies that allow users to ask questions using generative AI models and receive instant answers.

[1685] The present invention is a health consultation system that utilizes a generative AI model to support users' health management. This system functions in cooperation with users, terminals, and a server.

[1686] Hardware and software used

[1687] 1. Hardware

[1688] Device: Smartphone or tablet

[1689] Server: High-performance cloud server

[1690] IoT devices: smartwatches, blood pressure monitors, etc.

[1691] 2. Software

[1692] Application: Health management app

[1693] Database: MySQL

[1694] Generative AI model: OpenAI's GPT-4

[1695] Scheduling Software: Professional scheduling system

[1696] Communication protocol: REST API

[1697] Program processing

[1698] Server Features

[1699] The server receives health data sent by users and stores it in a database. Specifically, it uses a REST API to collect data such as the user's weight, number of steps, dietary habits, and sleep time. This data is stored in a MySQL database and managed centrally.

[1700] Furthermore, the server runs the generative AI model GPT-4, which generates appropriate answers to health inquiries from users. For example, if a user asks, "How is your diet balanced today?", GPT-4 will generate an answer such as, "Your diet today is high in carbohydrates. Eating a few more vegetables would improve the balance."

[1701] The server also accepts requests for in-depth consultations with specialists and online consultations, and uses dedicated scheduling software to manage the specialists' schedules and confirm appropriate appointments.

[1702] Device Features

[1703] The device provides an interface that allows users to input their daily health data. The input data is sent to a server in real time and stored in a database. It also provides an interface for chatting with the generating AI for health consultations. When a user accesses the chat screen and asks, "What kind of exercise should I do?", the answer from the generating AI is displayed instantly.

[1704] The device also connects to IoT devices such as smartwatches and blood pressure monitors, automatically collecting data from these devices and sending it to a server. For example, heart rate data obtained from a smartwatch is automatically sent to the server.

[1705] User operations

[1706] First, users download the app and create an account. They enter basic health information (age, weight, medical history, allergies, health goals, etc.). Next, they can enter their daily health data and ask the AI ​​for health advice.

[1707] Users can also make appointments for detailed consultations with specialists or online medical consultations, and receive specific health advice via video call at the scheduled time. After the free trial period ends, users can continue to receive detailed advice and online medical consultations by subscribing.

[1708] Specific examples

[1709] 1. Example of initial settings

[1710] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg."

[1711] 2. Examples of daily data entry and consultation

[1712] The user inputs their daily weight and dietary information and asks in the chat, "How is the balance of your diet today?" The AI ​​responds, "Your diet today is high in carbohydrates. If you add a few more vegetables, it will be more balanced."

[1713] 3. Example of collaboration with IoT devices

[1714] When a user is wearing a smartwatch, the heart rate data acquired from the smartwatch is automatically sent to the device and stored on the server.

[1715] 4. Consultations with specialists and examples of treatment

[1716] The user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist will contact each other via video call and receive specific dietary advice.

[1717] keyword

[1718] Generative AI model, prompt sentence

[1719] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1720] Step 1:

[1721] Initial Setup

[1722] The user installs and launches the health management app. Next, they select "New Registration" and enter basic health information such as their name, age, sex, weight, height, allergy information, and health goals. The entered information is sent from the device to the server, which receives it and stores it in a database.

[1723] Input: Name, age, gender, weight, height, allergy information, health goals

[1724] Output: User information is saved in the database

[1725] Step 2:

[1726] Routine data entry

[1727] Users enter their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the app. The data is sent in real time from the device to the server, which receives it and stores it in a database.

[1728] Input: weight, diet, steps, sleep time

[1729] Output: Health data is stored in a database

[1730] Step 3:

[1731] Using generative AI models for health consultations

[1732] A user uses the app's consultation function to ask a health question in chat format. For example, the user might ask, "How balanced is my diet today?" This question is sent from the device to the server. The server then invokes a generative AI model (GPT-4) to generate an answer to the question. The generated answer is sent to the user's device and displayed on the chat screen.

[1733] Input: User question (e.g., "How balanced is your diet today?")

[1734] Output: Answer from the generative AI model (e.g., "Your meal today is high in carbohydrates. Adding a few more vegetables would improve the balance.")

[1735] Step 4:

[1736] Book an in-depth consultation with a specialist or online consultation

[1737] A user uses the app's "Consultation with a Specialist" feature to book a consultation or consultation. For example, they request, "I'd like to book a consultation with a nutritionist this weekend at 3:00 PM." This request is sent to the server, which uses dedicated scheduling software to manage the specialist's schedule and confirm the appointment. The appointment details are sent to the user's device and notified.

[1738] Input: Appointment request (e.g., "I'd like to schedule a consultation with a nutritionist this weekend at 3pm")

[1739] Output: Confirmation of booking

[1740] Step 5:

[1741] Data linkage with IoT devices

[1742] When a user uses IoT devices such as a smartwatch or blood pressure monitor, health data is collected in real time from these devices. The data is transmitted to the device via Bluetooth or Wi-Fi. The device then sends the data to a server, which stores it in a database. For example, heart rate data from a smartwatch is automatically collected and recorded.

[1743] Input: Health data from IoT devices (e.g., heart rate, blood pressure)

[1744] Output: Health data is stored in a database

[1745] Step 6:

[1746] Manage your subscription

[1747] The server manages the user's subscription status. Before the free period ends, the server notifies the user and prompts them to subscribe. The user selects the desired plan within the app and enters their credit card information to complete the payment. The server verifies the payment information and updates the subscription status.

[1748] Input: Subscription request, credit card information

[1749] Output: Updated subscription status, payment confirmation notification

[1750] (Application example 1)

[1751] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1752] In modern society, many health problems, such as lifestyle-related diseases and obesity, are on the rise. In particular, busy lifestyles make it difficult to choose healthy meals. Other issues include the inability to easily access health consultations and medical treatment from experts, and a lack of appropriate advice based on individual health data. In response to these issues, there is a demand for fast and accurate health management support.

[1753] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1754] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with experts and online medical treatments, means for automatically collecting health data in cooperation with IoT devices, means for proposing health menus based on user data, and means for conducting health consultations with the user in a chat format. This allows users to receive dietary suggestions based on their own health information and easily consult with experts, thereby enabling them to achieve a healthy lifestyle.

[1755] The "means for receiving health data from the user" is an interface for sending and receiving health-related information (for example, weight, dietary content, amount of exercise, etc.) input by the user to the server.

[1756] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate answers to health-related questions asked by users.

[1757] The "database means for storing health data and health consultation history" is a database system for long-term storage and management of health data entered by users and the health consultation response history provided by the generation AI.

[1758] "Means for booking and conducting detailed consultations and online medical consultations with specialists" refers to a system that allows users to book and conduct detailed health consultations and online medical consultations with specialists.

[1759] "Means for automatically collecting health data in conjunction with IoT devices" refers to a function that works in conjunction with smartwatches and everyday health measurement devices to collect health data automatically obtained from them and send it to a server.

[1760] "Means for suggesting healthy menus based on user data" refers to a function in which AI uses health data provided by the user to generate and suggest healthy meal menus based on that data.

[1761] The "means for providing health consultations to users in chat format" is a system that allows users and generative AI models to exchange chat-style messages in real time and respond to health-related questions.

[1762] This invention is a system that supports users' health management, and aims to collect health data of individual users through collaboration with smartphones and IoT devices, and provide health consultations and dietary suggestions using a generative AI model. A detailed description of the system is provided below.

[1763] System configuration

[1764] This invention is mainly composed of three main elements: a server, a terminal, and a user.

[1765] Server Features

[1766] The server has the following main functions:

[1767] 1. Means of receiving health data from users:

[1768] The server receives basic health information (age, weight, health goals, allergy information, etc.) entered by the user through a smartphone application and stores it in a database.

[1769] 2. How to use generative AI models to answer health questions:

[1770] Leveraging OpenAI's ChatCompletion API, the system automatically generates answers to health-related questions from users. For example, if a user asks, "How balanced is your diet today?", the generative AI model instantly provides an answer.

[1771] 3. Database means for storing health data and health consultation history:

[1772] The server uses SQLite or another database management system to store the health data provided by the user and the history of answers provided by the generative AI model.

[1773] 4. How to book and conduct in-depth consultations with specialists and online consultations:

[1774] We provide a reservation system that has a function for linking with experts and allows users to consult with experts online, allowing users to easily obtain professional advice.

[1775] 5. Means of automatically collecting health data in conjunction with IoT devices:

[1776] It has the ability to automatically collect data from IoT devices such as smartwatches and blood pressure monitors, and send it to a server for storage.

[1777] 6. How to suggest healthy menus based on user data:

[1778] Based on the data provided by the user, the generative AI model will suggest healthy meals, for example, if the user's goal is to lose weight, it will provide a suitable meal plan.

[1779] 7. Means of providing health consultations to users through chat:

[1780] Users can chat with the generated AI model in real time via a smartphone app and ask for health advice.

[1781] Device Features

[1782] The device (smartphone, etc.) has the following functions:

[1783] 1. Data input interface:

[1784] It provides an interface that allows users to manually input their health data, making it easy to record daily weight, diet, exercise, etc.

[1785] 2. Health consultation chat function:

[1786] It provides an interface that allows users to chat with the generated AI and receive health advice.

[1787] 3. IoT device integration:

[1788] It works in conjunction with IoT devices such as smartwatches and blood pressure monitors, and has the ability to automatically collect data and send it to a server.

[1789] User operations

[1790] The user uses this system by following the steps below:

[1791] 1. Initial Setup:

[1792] Install the app and create an account, which starts by entering basic health information.

[1793] 2. Data entry and consultation:

[1794] Enter your daily health data and ask the AI ​​for health advice. For example, you can ask, "Could you please give me some advice about my recent diet?"

[1795] 3. Specialist consultations and online consultation appointments:

[1796] You can book consultations and appointments with specialists and receive professional advice via video call.

[1797] 4. Subscription:

[1798] After the free trial period ends, you can subscribe to continue using the service for detailed advice and online consultations.

[1799] Specific examples

[1800] Here is a concrete example of its use:

[1801] 1. Example of suggesting healthy menus based on user data:

[1802] After the user enters their health goals, they can ask, "How is your diet balanced today?" and the generative AI will suggest, "You're eating too many carbohydrates today. Try to eat more vegetables at your next meal."

[1803] 2. Example prompt:

[1804] User data: Age: 30, Weight: 70kg, Goal: Lose weight, Allergy: Peanuts

[1805] User Question: How balanced is your diet today?

[1806] Please provide your answer:

[1807] This allows users to understand their health status in real time and receive advice on appropriate diet and exercise.

[1808] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1809] Step 1: Initial Setup

[1810] Users download the smartphone app and create an account. The app prompts users to enter basic health information (age, weight, health goals, allergy information, etc.), which is then sent to a server and stored in a database.

[1811] Input: User's health information (age, weight, health goals, allergy information)

[1812] Data processing: Send input data to the server in JSON format

[1813] Output: User information is saved in the database

[1814] Step 2: Data entry

[1815] Users enter their daily health data (weight, diet, exercise, etc.) into the app, which then sends this data to a server and stores it in a database.

[1816] Input: User's daily health data (weight, diet, exercise)

[1817] Data processing: Send input data to the server in JSON format

[1818] Output: Daily health data is stored in a database

[1819] Step 3: IoT device integration

[1820] The app connects with users' IoT devices such as smartwatches and blood pressure monitors, allowing data collected from the devices to be automatically sent to the server.

[1821] Input: Health data from IoT devices (heart rate, steps, etc.)

[1822] Data processing: Automatically send data to the server

[1823] Output: Data from IoT devices is stored in a database

[1824] Step 4: Conducting health consultations

[1825] Users use the app's chat function to ask questions about their health. When a user enters a question, the question is sent to the server, which uses a generative AI model to generate an answer and sends it back to the user.

[1826] Input: User's health consultation question

[1827] Data computation: Generative AI models generate answers based on prompts

[1828] Output: The answer is returned to the user

[1829] Step 5: Healthy menu suggestions

[1830] When a user requests a healthy meal plan, they input their request into the app, and the server uses a generative AI model based on the user data to suggest an appropriate meal plan.

[1831] Input: User's health menu request

[1832] Data arithmetic: Generative AI models generate menus based on prompts

[1833] Output: A suggested meal menu is displayed to the user

[1834] Step 6: Book a consultation with a specialist

[1835] When a user wants to consult with a specialist, they make a reservation through the app. The server checks the specialist's schedule and confirms the reservation.

[1836] Input: User's consultation appointment request

[1837] Data processing: Matching booking requests with specialist schedules

[1838] Output: A confirmation of the booking is sent to the user.

[1839] Step 7: Manage your subscriptions

[1840] After the free trial period ends, users can subscribe to continue receiving detailed advice and treatment features. The app will manage users' subscription status and notify them when the trial period ends.

[1841] Input: Free period end information

[1842] Data Processing: Subscription Status Update

[1843] Output: A prompt to subscribe is sent to the user

[1844] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1845] This invention is a health management system that provides free health consultations utilizing generative AI and an emotion engine to assist users in managing their health. The system collects input data from users, uses a generative AI model and an emotion engine to answer health questions, and even provides in-depth consultations with experts and online medical treatments.

[1846] System configuration

[1847] Server Features

[1848] 1. Database Management

[1849] The server receives health data (e.g., weight, number of steps, dietary content, sleep time, etc.) and emotional data entered by the user and stores this data in a centrally managed database.

[1850] 2. Operation of generative AI

[1851] The server runs a generative AI model that automatically generates quick and appropriate answers to users' health inquiries, allowing users to easily get answers to their everyday health questions.

[1852] 3. Operating the Emotion Engine

[1853] The server runs an emotion engine, analyzes the data input by the user and the content of the health consultation, and recognizes the user's emotional state. This recognized emotional data is also stored in the database.

[1854] 4. Collaboration with experts

[1855] The server accepts requests for detailed consultations and online medical consultations with experts, and has the ability to check the experts' schedules and confirm appropriate appointments.

[1856] 5. IoT device data collection

[1857] The server works in conjunction with IoT devices such as smartwatches and blood pressure monitors, automatically collecting health data from these devices and storing it in a database.

[1858] Device Features

[1859] 1. Data input interface

[1860] The device provides an interface through which users can input their daily health and emotional data, which is then sent to a server and stored in a database.

[1861] 2. Health consultation chat function

[1862] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[1863] 3. Emotion data input assistance

[1864] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends this data to a server.

[1865] 4. IoT device integration

[1866] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[1867] User operations

[1868] 1. Initial Setup

[1869] Users download the app, create an account, and enter basic health information (age, weight, medical history, allergies, health goals, etc.). The app also configures facial recognition and voice analysis settings, and sets up how to input emotional data.

[1870] 2. Data entry and consultation

[1871] Users input their daily health data and ask the AI ​​for health advice. For example, they can ask questions such as, "How balanced is my diet today?" or "What kind of exercise should I do?" The emotion engine analyzes the user's emotions and can suggest, for example, "How to relax if you're feeling stressed."

[1872] 3. Consult with a specialist and book an online consultation

[1873] Users can book detailed consultations and online consultations with specialists, and if necessary, can make video calls with specialists to receive consultation details and advice.

[1874] 4. Subscription

[1875] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[1876] Specific examples

[1877] 1. Example of initial settings

[1878] A user installs the app and creates an account by entering their name, age, weight, gender, and allergy information. They then configure facial recognition and voice analysis. For example, a user might enter "30 years old, male, weighing 70 kg, allergy to peanuts, wanting to lose 5 kg" to enable the facial recognition feature.

[1879] 2. Examples of daily data entry and consultation

[1880] The user enters their daily weight and dietary information and asks in chat, "How is your diet balance today?" The generative AI responds, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance." The emotion engine recognizes signs of stress from the user's facial expressions and voice and provides additional advice, such as "Try taking deep breaths to relax."

[1881] 3. Example of collaboration with IoT devices

[1882] When a user is wearing a smartwatch, the heart rate data captured from the smartwatch is automatically sent to the device and stored on the server, without the user having to provide any specific input.

[1883] 4. Consultations with specialists and examples of treatment

[1884] A user requests, "I'd like to make an appointment with a nutritionist this weekend at 3:00 PM," and the server checks the specialist's schedule and confirms the appointment. At the appointed date and time, the user and the nutritionist contact each other via video call, and receive specific dietary advice. The user's emotional state is also provided to the nutritionist, enabling more personalized advice.

[1885] As described above, the present invention provides a system that allows users to manage their health on a daily basis and receive professional advice as needed, which promotes app usage and contributes to an increase in subscription subscribers.

[1886] The processing flow will be explained below.

[1887] Step 1: Initial Setup

[1888] 1. The user downloads and installs the app.

[1889] 2. The device will display the account creation screen when you first start it up.

[1890] 3. The user creates an account by entering their name, email address, password, age, weight, gender, and allergy information.

[1891] 4. The terminal sends the entered information to the server.

[1892] 5. The server stores the received information in its database and completes the account creation.

[1893] 6. The server sends a notification to the device that account creation is complete.

[1894] 7. The device displays a notification to the user that the account has been created.

[1895] Step 2: Configure facial recognition and voice analysis

[1896] 1. The device will display the initial setting screen for the emotion engine.

[1897] 2. The user configures the facial recognition and voice analysis settings (for example, by moving their face towards the camera or reading a specific phrase aloud).

[1898] 3. The device sends the settings to the server.

[1899] 4. The server saves the emotion engine configuration data in the database.

[1900] Step 3: Daily entry of health and emotional data

[1901] 1. The user launches the app and enters their daily health data, such as weight, number of steps, diet, and sleep time.

[1902] 2. The terminal sends the entered data to the server.

[1903] 3. The server stores the received data in a database.

[1904] 4. The device collects emotional data from the user's facial recognition and voice, and also sends this to the server.

[1905] 5. The server stores the emotion data in a database.

[1906] Step 4: Free health consultation

[1907] 1. The user activates the chat function within the app and enters a health consultation message into the generation AI.

[1908] 2. The device sends the user's question to the server.

[1909] 3. The server launches the generative AI model, analyzes the question, and generates an appropriate answer.

[1910] 4. The server sends the generated response to the terminal.

[1911] 5. The server starts the emotion engine and analyzes the user's emotion data.

[1912] 6. The server customizes the answer based on the sentiment data and generates appropriate follow-up advice.

[1913] 7. The server sends a customized answer and additional advice to the device.

[1914] 8. The device displays the user the answer from the generative AI and additional advice based on the emotional data.

[1915] Step 5: Consult with a specialist and online consultation

[1916] 1. Users can use the expert consultation function within the app to make detailed consultations or make appointments for online medical consultations.

[1917] 2. The device sends the user's request to the server.

[1918] 3. The server queries the expert's schedule to check available dates and times.

[1919] 4. The server sends the confirmed schedule information to the terminal.

[1920] 5. The terminal displays available reservation dates and times to the user and allows them to select one.

[1921] 6. The user selects the desired date and time, and the device sends the selection to the server.

[1922] 7. The server confirms the booking and notifies the user and the expert.

[1923] 8. The user launches the app at the scheduled time and begins the online consultation.

[1924] 9. The device provides video calling functionality to assist in the conduct of medical treatment.

[1925] 10. The server records the medical treatment details and advice in a database.

[1926] Step 6: Subscribe

[1927] 1. The server prepares and sends a subscription invitation to users whose free period is about to expire.

[1928] 2. The device will display instructions to the user via push notification or email.

[1929] 3. The user enters payment information, such as credit card information, within the app to sign up for a paid plan.

[1930] 4. The terminal sends the entered payment information to the server.

[1931] 5. The server processes the payment information and updates the subscription status to paid.

[1932] 6. The terminal notifies the user that subscription to a paid membership has been completed.

[1933] Through these steps, the system efficiently supports users in managing their health and provides a comprehensive approach that includes emotional data. The emotion engine analyzes the user's emotional state and customizes the generative AI model's answers, resulting in more personalized healthcare services.

[1934] Example 2

[1935] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1936] Conventional health management systems have difficulty collecting not only users' health data but also their emotional data and providing appropriate advice based on that data. Furthermore, the procedures for making appointments for consultations with specialists or online medical consultations are often complicated, placing a burden on users. Furthermore, insufficient integration with IoT devices results in inefficient data collection. This makes it difficult for users to effectively manage their health on a daily basis.

[1937] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1938] In this invention, the server includes means for receiving health data and emotional data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing health data and health consultation history, means for analyzing the user's emotional state using an emotion engine, means for booking and implementing detailed consultations with experts and online medical treatments, and means for automatically collecting health data in cooperation with IoT devices. This enables comprehensive health management based on health data and emotional data, and smooth cooperation with experts reduces the burden on the user and enables effective health management.

[1939] "Health data from the user" refers to data relating to the user's daily health condition, such as weight, number of steps, dietary content, and sleep time, which the user inputs himself or herself.

[1940] "Emotional data" is data about a user's emotional state collected through facial recognition and voice analysis of the user.

[1941] A "generative AI model" is an artificial intelligence model that generates answers in natural language in response to health inquiries from users. An example of this would be a chatbot.

[1942] A "means for answering health inquiries" is a method or system that enables a generative AI model to provide appropriate answers to health inquiries from users.

[1943] The "database means" is a data management system for storing the user's health data and health consultation history.

[1944] An "emotion engine" is a technology that analyzes input data and dialogue content from users to recognize their emotional state.

[1945] "Means for booking and conducting in-depth consultations and online medical consultations with specialists" refers to systems and functions that allow users to book a date and time for a consultation or medical consultation with a specialist and to conduct that appointment online.

[1946] "Means for automatically collecting health data in cooperation with IoT devices" refers to a method or system for automatically acquiring health data from IoT devices such as smartwatches and blood pressure monitors and transmitting that data to a server.

[1947] The present invention is a health management system that provides a free health consultation function utilizing a generative AI model and an emotion engine to assist users in managing their health. The system collects health and emotion data from users, uses the generative AI model and the emotion engine to answer health questions, and also provides detailed consultations with experts and online medical treatments. The following describes in detail the embodiments of the present invention.

[1948] server

[1949] The server performs the following functions:

[1950] 1. Data Collection and Management

[1951] The server receives the health and emotion data sent from the device and stores them in a centralized database, which is managed using MySQL or PostgreSQL, for example.

[1952] 2. Operation of generative AI models

[1953] When a health consultation from a user arrives at the server, the server generates a request to a generative AI model (e.g., GPT-4) and creates and sends an appropriate prompt. For example, in response to the question, "How is your diet balance today?", the server generates a prompt such as, "Please rate the user's diet balance today."

[1954] 3. Operating the Emotion Engine

[1955] The server runs an emotion engine (e.g., Affectiva or Azure Emotion Recognition API) that analyzes the user's input data and dialogue to recognize their emotional state. The recognized emotion data is also stored in a database.

[1956] 4. Scheduling with an expert

[1957] The server manages the schedules of the experts and confirms the reservation based on the reservation request from the user. The scheduling is done using the Google Calendar API.

[1958] 5. IoT device data collection

[1959] The server connects to IoT devices (e.g., smartwatches and blood pressure monitors) and automatically collects health data from these devices and stores it in a database. Firebase and AWS IoT are used here.

[1960] Terminal

[1961] The terminal provides the following user interface:

[1962] 1. Data input interface

[1963] The device provides an interface where users can input their daily health and emotional data. The input data is immediately sent to a server and stored in a database. This is implemented using a mobile app developed using Xamarin and React Native.

[1964] 2. Health consultation chat function

[1965] The device provides an interface where users can chat with the generated AI to ask health questions. The user's questions are sent to the server in real time, and the generated AI immediately returns answers.

[1966] 3. Auxiliary input of emotion data

[1967] The device performs facial recognition and voice analysis of the user, collects emotion data, and sends it to a server using OpenCV and Google Cloud Vision API.

[1968] 4. IoT device integration

[1969] The terminal works in conjunction with the user's IoT devices, such as a smartwatch or blood pressure monitor, and automatically collects data from these devices and sends it to a server.

[1970] User operations

[1971] 1. Initial settings of the app

[1972] Users download the app and create an account. They enter basic health information and configure facial recognition and voice analysis. For example, they might enter "30 years old, male, weighing 70 kg, allergy to peanuts, and want to lose 5 kg," and then activate the facial recognition feature.

[1973] 2. Daily data entry and consultation

[1974] Users input their daily health data and provide health advice to the generative AI model. For example, they can ask a question via chat, such as, "How is the balance of your diet today?" The generative AI will respond, "Your diet today is high in carbohydrates. If you eat a few more vegetables, it will be more balanced."

[1975] 3. Consultation and appointment with a specialist

[1976] The user can make an appointment for a detailed consultation or online medical consultation with a specialist and then complete the appointment. For example, the user can request, "I would like to make an appointment with a nutritionist at 3:00 PM this weekend," and the server will check the specialist's schedule and confirm the appointment.

[1977] 4. Subscription

[1978] After the free trial period ends, users can subscribe to continue using the service for detailed advice and online consultations.

[1979] In this way, the present invention utilizes a generative AI model and an emotion engine to provide a system that comprehensively supports users' health management. As a specific example, if a user inputs their daily weight and dietary details and asks in chat, "How is the balance of your diet today?", the generative AI can respond, "Your meal today is high in carbohydrates. Eating a few more vegetables would improve the balance," and the emotion engine can recognize signs of stress from the user's facial expressions and voice and provide additional advice, such as, "Try taking deep breaths to relax."

[1980] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1981] Step 1: User creates account

[1982] Users download the app and enter basic health information such as name, age, weight, gender, and allergies on the account creation screen. The data is then saved on the device and sent to the server, which then records the received data in a database.

[1983] Step 2: User performs initial setup

[1984] The user configures facial recognition and voice analysis settings. The device collects the user's facial images and voice samples and sends them to the server, where they are analyzed using an emotion engine and the results are stored in a database.

[1985] Step 3: Enter your daily health data

[1986] Users input their daily health data (weight, dietary habits, number of steps, sleep time, etc.) into the device's input interface. The device then sends the data to the server, which then stores the health data in a database.

[1987] Step 4: Enter your health consultation information

[1988] The user inputs a question using the health consultation chat function on the device. For example, they input a question such as, "How balanced is your diet today?" The device then sends this question to the server. The server receives the question and generates an appropriate prompt based on the user's input data.

[1989] Step 5: Request to the generative AI model

[1990] The server sends a prompt to the generative AI model. For example, it generates a prompt such as "Please rate the balance of the user's diet today" and sends it to the generative AI model. The generative AI model generates an answer based on this.

[1991] Step 6: Get the answer from the generative AI model

[1992] The generative AI model generates an answer based on the prompt and sends it back to the server, which then converts the answer into an appropriate format, such as "Your meal today is high in carbohydrates. Adding a few more vegetables would make it more balanced."

[1993] Step 7: Send the answer to the user

[1994] The server sends the answer received from the generative AI model to the device, which displays the answer on the user's chat screen. The user can view the answer and use it to manage their daily health.

[1995] Step 8: Analyze the sentiment data

[1996] The device constantly recognizes the user's face and analyzes their voice to collect emotional data. This data is sent to a server and analyzed by an emotion engine. The analysis results are stored in a database and used to provide advice to the user as needed.

[1997] Step 9: Book an appointment online with a specialist

[1998] The user inputs a request into the device to schedule a consultation with a specialist or an online medical consultation. For example, the user might request, "I'd like to schedule a consultation with a nutritionist this weekend at 3:00 PM." The device then sends this request to the server.

[1999] Step 10: Scheduling and Confirmation

[2000] The server checks the specialist's scheduling system and confirms the appointment based on the user's request. If the appointment is confirmed, the server notifies the user. The user can then have an online consultation with the specialist at the scheduled date and time.

[2001] Step 11: Conducting online consultations

[2002] Based on the scheduled date and time, users can receive online medical treatment and detailed health consultations through video calls with specialists. The server records the session information in a database for future reference.

[2003] Step 12: Manage your subscriptions

[2004] After the free trial period ends, users can choose whether or not to subscribe within the app. The device sends the user's subscription status to the server, which stores the information in a database. If users wish to continue using the app, a feature is provided to prompt them to subscribe.

[2005] (Application example 2)

[2006] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2007] Conventional health management systems collect and manage users' health data and can answer health questions using generative AI, but there were almost no systems designed for use in physical stores. As a result, there was a lack of a way for customers to receive health consultations easily and quickly in physical stores. It was also difficult to provide appropriate health advice in real time in physical stores.

[2008] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2009] In this invention, the server includes means for receiving health data from a user, means for responding to health consultations from the user using a generative AI model, database means for storing the health data and health consultation history, means for booking and implementing detailed consultations with specialists and online medical examinations, means for automatically collecting health data in cooperation with IoT devices, and means for providing health consultations and advice to users via smart devices installed in physical stores. This enables customers to easily and quickly receive health consultations in physical stores and appropriate health advice in real time.

[2010] The "means for receiving health data from the user" is an interface for transmitting health data such as weight, number of steps, dietary details, and sleep time input by the user to the server.

[2011] "Means for answering health inquiries from users using a generative AI model" is a function that utilizes a generative AI model to automatically generate appropriate answers to users' health-related questions.

[2012] The "database means for storing health data and health consultation history" is a database system for centrally managing and storing information about health data and health consultations entered by users.

[2013] "Means for reserving and conducting detailed consultations and online medical consultations with specialists" is a function that enables users to reserve and conduct detailed health consultations and online medical consultations with specialists.

[2014] "Means for automatically collecting health data in cooperation with IoT devices" refers to a function that automatically collects health data from IoT devices such as smartwatches and blood pressure monitors and sends it to a server.

[2015] "Means for providing health consultations and advice to users via smart devices installed in physical stores" refers to an interface that allows users to conduct health consultations and receive advice from the generating AI using smart glasses, head-mounted displays, tablets, or smartphones installed in physical stores.

[2016] This invention is a system that realizes health management support for customers in physical stores. Using smart devices installed in physical stores, customers can receive health consultations easily and quickly.

[2017] System configuration

[2018] This system is mainly composed of three entities: a server, a terminal, and a user.

[2019] Server Features

[2020] 1. Means of receiving health data from users

[2021] It receives health data such as weight, number of steps, dietary intake, and sleep time entered by the user and stores it in a database. This requires an internet connection and stable communication.

[2022] 2. A method for answering health questions from users using a generative AI model

[2023] The server runs a generative AI model (e.g., GPT) to automatically generate prompt and appropriate answers to health inquiries from users. Responses to questions are generated using prompt sentences.

[2024] 3. Database means for storing health data and health consultation history

[2025] The received health data and health consultation information are stored in a centralized database system, using database solutions such as MySQL or MongoDB.

[2026] 4. A way to schedule and conduct detailed consultations with specialists and online medical consultations

[2027] The server manages the schedules of specialists, allows users to book detailed health consultations and online consultations, and integrates video calling functionality to provide consultations and advice.

[2028] 5. Means of automatically collecting health data in conjunction with IoT devices

[2029] Health data is automatically collected from IoT devices such as smartwatches and blood pressure monitors and stored in a database.

[2030] Device Features

[2031] 1. Data input interface

[2032] It provides an interface where users can input their health data using smart devices (smart glasses, head-mounted displays, tablets, smartphones) in physical stores, and the input data is sent to a server.

[2033] 2. Health consultation chat function

[2034] The device provides an interface for users to chat with the AI ​​to discuss health issues. Questions are sent to the server in real time, and the AI ​​responds immediately.

[2035] Examples of prompts:

[2036] "What healthy foods should I eat after a workout?"

[2037] 3. Emotion data input assistance

[2038] The device collects emotional data from the user using methods such as facial recognition and voice analysis, and sends it to a server, allowing the AI ​​to reflect the user's emotional state in its responses.

[2039] User operations

[2040] 1. Initial Setup

[2041] Users download the app, create an account, enter basic health information, configure facial recognition and voice analysis settings, and set up how to input emotional data.

[2042] 2. Data entry and consultation

[2043] Users can use smart devices in physical stores to input their daily health data and also consult with the AI ​​to receive appropriate health advice.

[2044] Examples:

[2045] When a user types in, "What healthy foods should I eat after exercise?", the generative AI will answer, "I recommend a protein bar that is effective after exercise."

[2046] 3. Consult with a specialist and book an online consultation

[2047] Users can easily book and schedule detailed consultations with specialists or online medical consultations using the server's reservation management system.

[2048] Examples of technology implementation

[2049] Hardware Usage

[2050] Smart glasses, head-mounted displays, tablets, and smartphones will be installed in the physical store, and these devices will act as interfaces with users.

[2051] Software Usage

[2052] It uses Python's Flask as the server backend, MySQL as the database, and the open source GPT as the generative AI model.

[2053] These features allow users to seamlessly manage and consult about their health within a physical store, and they can also receive expert advice in real time, contributing to improving their health.

[2054] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2055] Step 1:

[2056] Users access smart devices installed in physical stores and input health data (weight, number of steps, dietary content, sleep time, etc.).

[2057] Input: Health data entered by the user through their smart device.

[2058] Output: The entered data is sent to the server.

[2059] How it works: An input interface is displayed on the screen of the smart device, and the user enters their health information item by item. After completing the input, the data is sent to the server in real time.

[2060] Step 2:

[2061] The server stores the received health data in a database.

[2062] Input: Health data submitted in step 1.

[2063] Output: Health data is stored in a database.

[2064] What happens: The server executes a query to properly structure the received data and store it in a MySQL database.

[2065] Step 3:

[2066] The user uses a smart device to provide health consultation to the generated AI model.

[2067] Input: A question that the user types into their smart device.

[2068] Output: The answer generated by the generative AI model.

[2069] How it works: The user types a prompt into their smart device, such as "What healthy foods should I eat after exercise?", and it is sent to the server, which then passes the question to the generative AI model.

[2070] Step 4:

[2071] The server uses the generative AI model to generate answers to the user's questions.

[2072] Input: The question entered by the user in step 3.

[2073] Output: The correct answer to the question.

[2074] How it works: The server inputs a question as a prompt to a generative AI model (e.g., GPT), and the model generates the optimal answer to that question. The generated answer is returned to the server.

[2075] Step 5:

[2076] The server sends the generated answer back to the smart device and displays it to the user.

[2077] Input: The answer generated in step 4.

[2078] Output: The answer is displayed on the user's smart device.

[2079] How it works: The server sends the answer obtained from the generative AI model to the smart device, which immediately displays the answer to the user for confirmation.

[2080] Step 6:

[2081] If necessary, the user books an online consultation with a specialist.

[2082] Input: Booking request entered by user into smart device.

[2083] Output: Booking confirmation and schedule confirmation.

[2084] Specific operation: The user inputs the desired date and time for consultation with a specialist through their smart device and sends it to the server. The server checks the specialist's schedule and, if the appointment is confirmed, notifies the user's smart device of the information.

[2085] Step 7:

[2086] Online consultations with specialists are conducted, providing detailed advice on the user's health condition.

[2087] Input: User's health condition and consultation details at the time of appointment.

[2088] Output: Expert advice.

[2089] Specific operation: At the scheduled time, the expert and the user will be connected via video call, where the expert will check the user's health condition and provide advice.

[2090] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2091] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2092] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2093] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2094] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2095] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2096] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2097] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2098] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2099] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2100] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2101] In...

Claims

1. means for receiving health data from a user; A means for answering health inquiries from users using the generative AI model; and a database means for storing health data and health consultation history; A means to book and conduct detailed consultations with specialists and online medical consultations, A means to automatically collect health data in conjunction with IoT devices, and A system including:

2. The system of claim 1 further comprising means for analyzing health conditions and making recommendations based on the health data stored in the database means.

3. The system of claim 1 , further comprising means for managing a user's subscription status and prompting the user to subscribe at the end of a free period.

Images