system

Abstract

We provide the system. [Solution] A server means for collecting user health data and generating medication alerts, A terminal means for measuring the user's facial recognition and biometric information, An artificial intelligence system that interacts with users based on the information it generates, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Sublingual immunotherapy, known as a treatment method for allergic rhinitis, has problems in that it requires long-term continuous treatment, imposing a significant psychological and economic burden on patients. Also, since self-management is required, it is difficult to avoid forgetting to take the medicine and to maintain the motivation for treatment. Furthermore, since users cannot easily grasp their own health status, it may be difficult to perceive the effect of the treatment. There is a need for a method to efficiently solve these problems.

Means for Solving the Problems

[0005] This invention includes a server that collects user health data and generates medication alerts at appropriate times. Furthermore, a terminal device performs facial recognition and measures the user's biometric information, and an artificial intelligence device interacts with the user based on the collected information. This provides a system that makes it easier for patients to continue treatment, prevents missed doses, and allows them to monitor their health status in real time. These means make it possible to reduce the burden of long-term treatment on patients and improve the success rate of treatment.

[0006] A "server system" is a computer system that collects users' health-related data and uses that data to generate alerts and information.

[0007] A "terminal device" is a device that is directly operated by the user and has the function of performing facial recognition and measuring biometric information, and transmitting that information to a server.

[0008] "Artificial intelligence tools" refer to systems that use natural language processing and machine learning techniques to generate user interactions and answers based on collected information.

[0009] "Facial recognition" is a technology that analyzes a user's facial image and detects specific features.

[0010] "Biometric information" refers to data that indicates the user's health status, and specifically includes heart rate and stress indicators.

[0011] "Health data" is a general term for information that indicates a user's physical condition and changes.

[0012] An "alert" is a notification or warning signal designed to prompt a user to take a specific action or pay attention to something. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, a numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention constructs a system that supports the continuation of sublingual immunotherapy by utilizing a server, terminal, and user interface. The server stores the user's registered medication schedule and personal health data, and generates real-time alerts and motivational messages tailored to the user. This system incorporates individual program logic and employs a highly accurate scheduling algorithm that provides notifications based on various conditions.

[0035] The device accepts user input and activates its camera for facial recognition when medication is taken. While the user holds the medication under their tongue, the device measures biosignals and analyzes heart rate and stress levels. For example, it analyzes whether the user is relaxed or stressed and sends this data to a server. The server then generates feedback on the user's health status and notifies the device. This feedback aims to improve the user's sense of security regarding their treatment.

[0036] Users can interact with the generative AI through a smartphone app. When a user enters a question within the app, that information is sent to a server. The server uses the generative AI to analyze the question and generate an appropriate answer. This AI provides the best possible answer for the user by referring to a vast amount of medical data and past conversation logs. For example, in response to the question, "Is it okay that I took my medication late today?", it can provide personalized advice such as, "Let's adjust the timing of future doses so that your progress is not affected."

[0037] This system aims to support users in making treatment a habit and improve the likelihood of continuing medication, thereby promoting long-term health improvement. In this way, the present invention reduces the psychological and physical burden on users and helps alleviate anxieties about health and performance between treatments.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The user launches the smartphone app and logs into their account. Here, they can check their medication schedule and personal information.

[0041] Step 2:

[0042] The device connects to the server and receives the user's current medication schedule and health data. This data is used to generate the next medication reminder.

[0043] Step 3:

[0044] The server generates medication alerts based on user data. It schedules notifications to be sent to the device based on pre-set times.

[0045] Step 4:

[0046] The device will notify the user of an alert at a specified time. This can be done via sound, vibration, or screen display.

[0047] Step 5:

[0048] The user checks the notification and prepares to start taking the medication by pressing the "Start Taking Medication" button within the app.

[0049] Step 6:

[0050] After confirming that the user is ready, the device activates its camera and takes a picture of the user's face. Face recognition is performed at this stage.

[0051] Step 7:

[0052] The device analyzes captured facial images to measure heart rate and stress levels. Biometric data is processed in real time.

[0053] Step 8:

[0054] The server checks the biometric data received from the terminal, generates a warning if any abnormal values ​​are found, and notifies the terminal.

[0055] Step 9:

[0056] Users can use the in-app chat function to ask the AI ​​questions about alerts and health status.

[0057] Step 10:

[0058] The server receives a question and uses a generative AI to perform natural language processing analysis. Based on the analysis results, it generates an appropriate answer.

[0059] Step 11:

[0060] The terminal displays the generated response to the user and provides further information as needed.

[0061] Step 12:

[0062] After reviewing the feedback, users can continue treatment by reviewing their next medication schedule or asking further questions.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Support is needed to improve the continuity of treatment in sublingual immunotherapy, but conventional methods lack appropriate feedback tailored to the individual circumstances and psychological state of the user. In addition, there is a need for means to provide timely medication alerts and personalized advice.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes information management means for collecting the user's medical information and generating medication warnings based on a high-precision scheduling algorithm, input means for performing facial recognition of the user and measuring biosignals, and generation AI means for engaging in dialogue with the user based on the analyzed information. This makes it possible to provide alerts and generate feedback tailored to the user's individual treatment status.

[0068] "Information management means" refers to a system element that has the function of collecting the user's medical information and generating medication warnings based on a highly accurate scheduling algorithm.

[0069] "Input means" refers to system elements used for facial recognition of the user and measurement of biosignals.

[0070] A "generative AI means" is a system element equipped with artificial intelligence functions for interacting with users based on analyzed information.

[0071] This invention relates to a system designed to support the continuation of sublingual immunotherapy, and includes a server, a terminal, and a user interface as its main components.

[0072] The server collects the user's medical information through information management systems and processes the stored data based on a high-precision scheduling algorithm to generate timely medication reminders. These reminders are sent to the user's smartphone or computer to help them proceed with their treatment as planned.

[0073] The device functions as an input device, using a camera and biosensors to recognize the user's face and measure biosignals. For example, when a user takes medication, the device uses facial recognition to identify the user and measures heart rate and other stress indicators using a heart rate sensor. This makes it possible to continuously monitor the user's current physical condition.

[0074] Artificial intelligence used as a generative AI tool analyzes data stored on a server and provides feedback and advice in response to user questions and requests. For example, if a user enters a question into the app such as, "How should I adjust my next dose?", the AI ​​can analyze the question and provide specific advice such as, "Let's try setting your next dose for 6 PM." Through such prompt-based dialogue, generative AI can provide optimal support to the user.

[0075] The system is designed to promote long-term health improvement by providing psychological and physical support through personalized motivational messages and feedback, enabling users to make treatment a habit. In this way, the present invention aims to provide users with a safer and more effective treatment environment.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] Users input their medication schedule and health information through a smartphone app.

[0079] Input: Dosage schedule, health information

[0080] Specific operation: Users register medical data such as daily medication times and allergy information using the app's input form.

[0081] Output: Saving user data to the database

[0082] Step 2:

[0083] The server receives user input data and stores the data using information management tools.

[0084] Input: User-entered medication schedule and health information

[0085] Specific operation: The server updates the database and saves it as data necessary for configuring a high-precision scheduling algorithm.

[0086] Output: Generate the next alert using the saved data.

[0087] Step 3:

[0088] The server analyzes the stored data and generates medication alerts based on a highly accurate scheduling algorithm.

[0089] Input: Saved user medication schedule

[0090] Specific operation: The server executes a scheduling algorithm and generates an alert based on the next dose timing.

[0091] Output: Generated medication alert

[0092] Step 4:

[0093] The device receives medication alerts from the server and notifies the user.

[0094] Input: Medication alert sent from the server

[0095] Specific action: The device displays an alert message to the user via push notification.

[0096] Output: Alert notification to the user

[0097] Step 5:

[0098] When a user takes medication, the device performs facial recognition and measures biometric information.

[0099] Input: User initiates medication

[0100] Specific actions: The device's camera is activated, facial recognition technology is used to verify the user's identity, and biometric sensors are used to measure heart rate and other parameters.

[0101] Output: Measured biometric data

[0102] Step 6:

[0103] The server analyzes biometric data transmitted from the terminal and generates feedback on the user's health status.

[0104] Input: Biometric data transmitted from the device

[0105] Specific actions: Analyze biometric data to assess the user's stress and health status. Create feedback messages based on the assessment results.

[0106] Output: Generated feedback message

[0107] Step 7:

[0108] Artificial intelligence, used as a generative AI tool, analyzes user prompts and provides appropriate responses.

[0109] Input: Prompt text entered by the user in the app

[0110] Specific operation: The AI ​​analyzes the prompt and generates appropriate advice by referring to past data. An example prompt is "How should I adjust the timing of my next dose?"

[0111] Output: AI-generated answers and advice

[0112] Step 8:

[0113] The server notifies the user's device of the feedback messages and AI responses generated by the server.

[0114] Input: Generated feedback message, AI response

[0115] Specific action: The server sends a notification to the terminal, allowing the user to check the message.

[0116] Output: Notification of feedback and advice to the user.

[0117] (Application Example 1)

[0118] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0119] Systems designed to support users in taking medications appropriately are required to promote continued treatment and reduce psychological and physical burden by providing real-time monitoring of health status and personalized feedback. However, conventional systems have the challenge of not being able to accurately evaluate physical reactions and changes in health status during medication administration and take appropriate action on the spot.

[0120] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0121] In this invention, the server includes an information processing device that collects the user's physical information and generates medication notifications; a terminal device that acquires the user's identification information and biometric information; an intelligent engine means that communicates with the user based on the generated information; and a transaction function that purchases medical-related products using the user's approval means. This allows the user to monitor their health status in real time, receive appropriate feedback, and easily purchase necessary medical products.

[0122] A "user" refers to an individual who uses the system to take medication and manages their health status and purchases medications in the process.

[0123] "Physical information" refers to data indicating the user's health status, including physiological data acquired in real time, such as heart rate and stress level.

[0124] A "medication notification" is a reminder generated to inform the user of the appropriate timing for taking their medication.

[0125] An "information processing device" refers to a computer system that collects a user's physical information and processes it to generate medication notifications.

[0126] "Identification information" refers to data used to identify users through methods such as facial recognition.

[0127] "Terminal device" refers to a terminal device used to acquire user identification information and biometric information.

[0128] "Biometric information" refers to information that includes data about the user's physical condition, such as heart rate and stress level.

[0129] An "intelligent engine means" is a processing system that utilizes artificial intelligence to communicate with users based on the information it generates.

[0130] "Approval process" refers to the process used to confirm a user's intention to purchase medical-related products.

[0131] The "transaction function" refers to a system feature used by users to purchase medical-related products.

[0132] One embodiment of this invention is a comprehensive system in which a user, server, and terminal work together to provide a user with the ability to take medication appropriately and manage their health. The specific embodiment is described below.

[0133] Users log in to the system using a smartphone, which is the end device, and its facial recognition function. The smartphone's camera and sensors for biometric information sensing are used for facial recognition. This device acquires the user's identification information and physical information, such as heart rate and stress level.

[0134] The server operates as an information processing unit using Python and the Flask framework, storing and analyzing real-time acquired user physical information. Based on this information, it generates medication notifications and sends reminders to the user. Furthermore, the server utilizes a generative AI model as an intelligent engine to generate feedback and motivational messages regarding the user's health status.

[0135] For example, if a user has not yet taken their medication today, the server analyzes the user's health data and past medication patterns, and in response to the question, "Is it okay that I took my medication late today?", it advises, "Let's adjust your future medication times so that your progress is not affected."

[0136] Example prompt: "User A took medication today. Their stress level is high; please generate specific advice for relaxation."

[0137] In this way, users can take their medication at the appropriate time and easily purchase medical products with the support of an intelligent engine.

[0138] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0139] Step 1:

[0140] The device activates the facial recognition function when the user operates their smartphone. The input is camera data from the smartphone, and the facial recognition algorithm is used to obtain the user's identification information. This allows the device to log the user in.

[0141] Step 2:

[0142] The device acquires the user's biometric information using sensors. The input data includes heart rate and stress levels, which are measured and temporarily stored within the device. The acquired biometric information is used to assess the user's physical condition.

[0143] Step 3:

[0144] The server receives the user's biometric and identification information from the terminal. The input is data transmitted from the terminal, which is stored in a database and analyzed by an information processing device. The output of the analysis is a health assessment of the user.

[0145] Step 4:

[0146] The server generates medication reminders for the user based on a health assessment. The input is the analyzed health status, and a scheduling algorithm is used to determine and notify the user of the next dose timing. The output is a reminder sent to the user's smartphone.

[0147] Step 5:

[0148] The server activates an intelligent engine and uses a generative AI model to generate feedback in response to the user's question, "I was late taking my medication today, is that okay?". The input is the user's past medication data and health assessment, and the output is personalized advice.

[0149] Step 6:

[0150] The device displays feedback and notifications from the server to the user. The input consists of reminders and advice received from the server, which are displayed in an easy-to-understand format, allowing the user to make decisions based on that information.

[0151] Step 7:

[0152] Users purchase necessary medical products using an in-app approval mechanism. Inputs include recommended products from the server and user approval; output is a purchase completion notification. The terminal records purchase history for future recommendations.

[0153] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0154] This invention enhances support for continuing sublingual immunotherapy by using a system that integrates the user, terminal, server, and emotion engine. The server manages information from the user and generates medication alerts and accumulates the user's emotional data based on that data. Through this, the aim is to maintain the user's physical and mental health.

[0155] The device is responsible for collecting user health data and uses facial recognition technology and biometric data measurement functions to understand the user's current state. Furthermore, by incorporating an emotion engine, it recognizes the user's emotions in real time from facial analysis and biometric data. For example, if the user shows an anxious expression, the emotion engine detects this state and sends that information to the server.

[0156] Based on the received emotional data, the server dynamically modifies the content of advice and motivational messages provided to the user via a generating AI. For example, if the emotional engine determines that the user is feeling down, the server can generate a positive and encouraging message and display it to the user through their device. In this way, personalized support that takes the user's emotions into account is provided, aiming to motivate them to continue treatment.

[0157] By using a device equipped with an emotion engine, users can visualize their own emotions and health status. The application provides feedback to the user and helps maintain a balance between health and emotions as part of their daily treatment. For example, when a user is feeling stressed, the application can suggest relaxation methods and simple exercises, making the treatment process more palatable.

[0158] This system provides emotional support to users undergoing sublingual immunotherapy, creating an environment that maximizes the effectiveness of the treatment. Thus, the configuration disclosed herein not only enhances conventional treatment support systems but also aims for more comprehensive user support through a new approach that utilizes emotion recognition.

[0159] The following describes the processing flow.

[0160] Step 1:

[0161] The user launches the smartphone app and prepares to check their health status. They log in and access the app's main screen.

[0162] Step 2:

[0163] The device activates its camera and takes a picture of the user's face. Using facial recognition technology, it identifies the user's face and acquires the image data.

[0164] Step 3:

[0165] The device analyzes facial features using the acquired facial image, and the emotion engine recognizes the user's emotions from their facial expressions. In this process, it detects facial expression patterns such as smiles and anxiety.

[0166] Step 4:

[0167] The device uses sensors to measure biometric information such as the user's heart rate and skin potential. An emotion engine analyzes this data to supplement the user's emotional state.

[0168] Step 5:

[0169] The device sends the user's emotional state, as recognized by the emotion engine, to the server. This information includes data on the type and intensity of the emotion.

[0170] Step 6:

[0171] The server integrates received emotional and health data to assess the user's overall health status. Based on the analysis results, a generative AI generates optimal feedback.

[0172] Step 7:

[0173] The server sends the generated feedback to the terminal and displays it to the user. This includes advice on the user's current health status and motivational messages tailored to their emotions.

[0174] Step 8:

[0175] Users review the displayed feedback and, if necessary, perform the recommended relaxation methods or exercises.

[0176] Step 9:

[0177] Users maintain motivation to continue treatment by continuing to manage their health based on daily treatment and feedback.

[0178] (Example 2)

[0179] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0180] A challenge in continuing sublingual immunotherapy is the lack of a system that provides support that considers not only the user's physical health but also their emotional health. Conventional systems have been unable to adequately grasp the user's emotional state, limiting their ability to provide personalized support.

[0181] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0182] In this invention, the server includes terminal means for collecting the user's biometric data and performing emotion analysis, means for managing the user's emotion data and biometric information and dynamically generating messages, and artificial intelligence means for interacting with the user based on the generated messages. This makes it possible to analyze the user's emotional state in real time and provide personalized health advice and motivational messages based on that analysis.

[0183] "User biometric data" is a general term for data that indicates the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0184] "Emotional analysis" is a process that analyzes and determines a user's emotional state based on their facial expressions and biometric data.

[0185] "Terminal means" refers to electronic devices used to collect the user's biometric data and analyze their emotional state.

[0186] A "server system" is a central processing unit that receives data transmitted from terminals and performs data analysis and information generation.

[0187] "Dynamically generating messages" means creating messages with content that is optimal for the user's current emotional state at any given time.

[0188] "Artificial intelligence means" refers to a program or device that uses generated messages to conduct automated conversations with users.

[0189] "Personalized health advice" refers to individualized health management guidance provided based on the user's emotional state and health data.

[0190] A "motivational message" is a message intended to psychologically motivate or encourage a user.

[0191] This invention is a system that comprehensively supports the physical and emotional health of users undergoing sublingual immunotherapy. The system consists of a user, a terminal, a server, an emotion analysis engine, and a generative AI model.

[0192] The device uses built-in sensors to collect the user's biometric data in real time. This provides physiological indicators such as heart rate and blood pressure. Additionally, facial recognition technology using a camera analyzes the user's facial expressions and determines their emotional state. All of this data is input into the device's emotion analysis engine.

[0193] The server receives emotional data and biometric information transmitted from the terminal and stores it in a database. Based on this information, a generative AI model operates to dynamically generate personalized advice and encouraging messages for the user. For example, a prompt such as "The user appears to be stressed. Please suggest ways to relax." is input to the generative AI model. This process allows the server to generate the most appropriate message based on the user's state.

[0194] Users receive these messages through their devices, visualizing their emotional and physical state. Providing daily feedback to the system enables more precise personalization. For example, if a user is tagged as "anxious," the device can display specific suggestions such as, "We recommend taking a deep breath and going for a short walk."

[0195] In this way, the present invention provides a novel approach to effectively support users' sublingual immunotherapy, enabling the maintenance of overall health.

[0196] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0197] Step 1:

[0198] The device collects the user's biometric information and facial expression data. As input, it collects the user's heart rate, blood pressure, and facial image data. This information is acquired through sensors and a camera. As output, a dataset containing biometric information and image data is generated. Specifically, when the user stands in front of the device, the sensors automatically activate and data acquisition begins.

[0199] Step 2:

[0200] The device inputs collected data into an emotion analysis engine to analyze the user's emotional state. The input consists of collected biometric information and facial expression data, which the emotion analysis engine processes. Specifically, a facial recognition algorithm determines emotions from facial expressions. The output is an analysis result indicating the user's emotional state. The analysis result is expressed as an emotion label such as anxiety, joy, or stress.

[0201] Step 3:

[0202] The terminal sends analyzed emotion data to the server. The input is analyzed data indicating the user's emotional state. This data is processed and then sent to the server to accumulate data. The output is the emotion data stored on the server. Specifically, data is sent from the terminal to the server periodically.

[0203] Step 4:

[0204] The server uses a generative AI model based on received emotional data to generate messages for the user. The input is data on the user's emotional state. By inputting "The user is feeling anxious. Please suggest ways to relax." as a prompt into the generative AI model, a message containing advice is obtained as output. The generated message is customized based on the user's individual profile.

[0205] Step 5:

[0206] The terminal delivers messages sent from the server to the user. The input is the generated message from the server. The output is the message displayed on the terminal's screen. Specifically, the terminal notifies the user of the message and provides an interface for the user to confirm it. The user confirms the message, and feedback is sent back to the server as needed.

[0207] (Application Example 2)

[0208] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0209] In sublingual immunotherapy, comprehensive management of the user's physical and mental health is crucial to supporting their continued treatment. However, conventional systems lack support that takes emotional states into account, making it difficult to address the individual needs of users. Therefore, there is a need for methods that provide personalized support based on the user's emotions and health status.

[0210] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0211] In this invention, the server includes information processing means for collecting user health data and generating medication alerts; device means for recognizing the user's face and measuring biometric information; intelligent engine means for interacting with the user based on the generated information; and presentation means for analyzing the user's emotional state and suggesting appropriate products or services. This enables personalized support and product suggestions based on the user's emotional state.

[0212] "Information processing means" refers to a device or system that has the function of generating appropriate medication alerts by collecting and analyzing user health data.

[0213] A "device means" is a device that recognizes the user's face and measures biometric information to understand the user's latest health status.

[0214] An "intelligent engine" is an artificial intelligence system that interacts with users based on collected information and provides appropriate advice and feedback.

[0215] A "presentation method" is a system that analyzes the user's emotional state and, based on the results, proposes the most suitable products and services in real time.

[0216] To implement this invention, the system is primarily configured based on the interaction between servers, terminals, and users.

[0217] First, the server acts as an information processing device, storing and managing health data collected from users and generating appropriate medication alerts. The server uses a generative AI model to perform necessary data analysis based on the user's health status and emotions. High-speed server equipment is used for this process.

[0218] The device recognizes the user's face and measures biometric information. Computer vision technologies such as OpenCV using a camera are employed for face recognition, and an emotion recognition model based on TENSORFLOW® is incorporated to analyze the user's emotional state. This makes it possible to assess the user's health status in real time.

[0219] Furthermore, the server, acting as an intelligent engine, generates optimal advice based on the recognized user's emotions and health data. This generation is based on a generative AI model and enables dynamic messaging using prompts. For example, if a user is showing signs of stress, inputting a prompt such as, "The customer appears anxious. Please create a suggestion for products that will have a relaxing effect," will allow the generative AI to provide an appropriate message.

[0220] As a means of presentation, the system makes suggestions on the device side based on the user's emotional state. The display is done using devices such as smart glasses or digital displays, allowing for real-time presentation of necessary products and services. Therefore, by using this system, users can receive personalized support, enabling continuous support for sublingual immunotherapy and effective product recommendations.

[0221] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0222] Step 1:

[0223] The device collects the user's health data. As input, it acquires the user's biometric information and facial images through its camera and sensors. The device uses a facial recognition algorithm to identify the user, compares and analyzes the obtained biometric data against pre-set baseline values, and sends the results to a server.

[0224] Step 2:

[0225] The server analyzes the received data to assess the user's health and emotional state. Using health data sent from the terminal as input, it recognizes the user's emotions through a generative AI model. The server utilizes text analysis and machine learning algorithms to quantify, for example, stress levels, and passes the results to the generative AI model.

[0226] Step 3:

[0227] The server uses a generative AI model to generate optimal advice and product suggestions for the user. The input is the results of the health status and emotion assessment mentioned earlier. The generative AI model utilizes preset prompts to generate suggestions for products that have a relaxing effect when the user shows signs of anxiety. For example, it uses the prompt, "The customer is showing signs of anxiety. Please create suggestions for products that have a relaxing effect."

[0228] Step 4:

[0229] The server sends generated advice and suggestions to the terminal, which then presents them to the user. It receives generated text messages as input and displays them on the terminal's display or smart glasses. The terminal provides information visually to the user through its user interface, allowing the user to decide on actions based on the suggestions.

[0230] Step 5:

[0231] The user reviews the presented products or advice and takes action as needed. Based on the information received, the user checks products with relaxing effects and makes purchases in stores or orders online. The user's response is fed back into the system via the terminal and used to improve the accuracy of future suggestions.

[0232] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0233] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0234] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0235] [Second Embodiment]

[0236] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0237] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0238] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0239] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0240] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0241] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0242] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0243] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0244] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0245] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0246] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0247] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0248] This invention constructs a system that supports the continuation of sublingual immunotherapy by utilizing a server, terminal, and user interface. The server stores the user's registered medication schedule and personal health data, and generates real-time alerts and motivational messages tailored to the user. This system incorporates individual program logic and employs a highly accurate scheduling algorithm that provides notifications based on various conditions.

[0249] The device accepts user input and activates its camera for facial recognition when medication is taken. While the user holds the medication under their tongue, the device measures biosignals and analyzes heart rate and stress levels. For example, it analyzes whether the user is relaxed or stressed and sends this data to a server. The server then generates feedback on the user's health status and notifies the device. This feedback aims to improve the user's sense of security regarding their treatment.

[0250] Users can interact with the generative AI through a smartphone app. When a user enters a question within the app, that information is sent to a server. The server uses the generative AI to analyze the question and generate an appropriate answer. This AI provides the best possible answer for the user by referring to a vast amount of medical data and past conversation logs. For example, in response to the question, "Is it okay that I took my medication late today?", it can provide personalized advice such as, "Let's adjust the timing of future doses so that your progress is not affected."

[0251] This system aims to support users in making treatment a habit and improve the likelihood of continuing medication, thereby promoting long-term health improvement. In this way, the present invention reduces the psychological and physical burden on users and helps alleviate anxieties about health and performance between treatments.

[0252] The following describes the processing flow.

[0253] Step 1:

[0254] The user launches the smartphone app and logs into their account. Here, they can check their medication schedule and personal information.

[0255] Step 2:

[0256] The device connects to the server and receives the user's current medication schedule and health data. This data is used to generate the next medication reminder.

[0257] Step 3:

[0258] The server generates medication alerts based on user data. It schedules notifications to be sent to the device based on pre-set times.

[0259] Step 4:

[0260] The device will notify the user of an alert at a specified time. This can be done via sound, vibration, or screen display.

[0261] Step 5:

[0262] The user checks the notification and prepares to start taking the medication by pressing the "Start Taking Medication" button within the app.

[0263] Step 6:

[0264] After confirming that the user is ready, the device activates its camera and takes a picture of the user's face. Face recognition is performed at this stage.

[0265] Step 7:

[0266] The device analyzes captured facial images to measure heart rate and stress levels. Biometric data is processed in real time.

[0267] Step 8:

[0268] The server checks the biometric data received from the terminal, generates a warning if any abnormal values ​​are found, and notifies the terminal.

[0269] Step 9:

[0270] Users can use the in-app chat function to ask the AI ​​questions about alerts and health status.

[0271] Step 10:

[0272] The server receives a question and uses a generative AI to perform natural language processing analysis. Based on the analysis results, it generates an appropriate answer.

[0273] Step 11:

[0274] The terminal displays the generated response to the user and provides further information as needed.

[0275] Step 12:

[0276] After reviewing the feedback, users can continue treatment by reviewing their next medication schedule or asking further questions.

[0277] (Example 1)

[0278] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0279] Support is needed to improve the continuity of treatment in sublingual immunotherapy, but conventional methods lack appropriate feedback tailored to the individual circumstances and psychological state of the user. In addition, there is a need for means to provide timely medication alerts and personalized advice.

[0280] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0281] In this invention, the server includes an information management means for collecting the user's medical information and generating a dosing warning based on a high-precision scheduling algorithm, an input means for performing face recognition of the user and measuring biometric signals, and a generating AI means for interacting with the user based on the analyzed information. This enables the provision of alerts and the generation of feedback according to the user's individual treatment situation.

[0282] The "information management means" is a system element that has the function of collecting the user's medical information and generating a dosing warning based on a high-precision scheduling algorithm.

[0283] The "input means" is a system element for performing face recognition of the user and measuring biometric signals.

[0284] The "generating AI means" is a system element equipped with an artificial intelligence function for interacting with the user based on the analyzed information.

[0285] The present invention is a system designed to support the continuation of sublingual immunotherapy, and mainly includes a server, a terminal, and a user interface as its components.

[0286] The server collects the user's medical information through the information management means and processes the stored data based on a high-precision scheduling algorithm to generate timely dosing warnings. These warnings are notified to the user's smartphone or computer to assist the user in proceeding with the treatment as planned.

[0287] The terminal functions as an input means and uses a camera and a biometric sensor to perform face recognition of the user and measure biometric signals. For example, when the user takes medicine, the terminal uses face recognition to confirm the user and measures the heart rate and other stress indicators with a heart rate sensor. This enables continuous monitoring of the user's current physical condition.

[0288] Artificial intelligence used as a generative AI tool analyzes data stored on a server and provides feedback and advice in response to user questions and requests. For example, if a user enters a question into the app such as, "How should I adjust my next dose?", the AI ​​can analyze the question and provide specific advice such as, "Let's try setting your next dose for 6 PM." Through such prompt-based dialogue, generative AI can provide optimal support to the user.

[0289] The system is designed to promote long-term health improvement by providing psychological and physical support through personalized motivational messages and feedback, enabling users to make treatment a habit. In this way, the present invention aims to provide users with a safer and more effective treatment environment.

[0290] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0291] Step 1:

[0292] Users input their medication schedule and health information through a smartphone app.

[0293] Input: Dosage schedule, health information

[0294] Specific operation: Users register medical data such as daily medication times and allergy information using the app's input form.

[0295] Output: Saving user data to the database

[0296] Step 2:

[0297] The server receives user input data and stores the data using information management tools.

[0298] Input: User-entered medication schedule and health information

[0299] Specific operation: The server updates the database and stores it as data required for setting a high-precision scheduling algorithm.

[0300] Output: Generation of the next alert using the stored data

[0301] Step 3:

[0302] The server analyzes the stored data and generates a medication alert based on a high-precision scheduling algorithm.

[0303] Input: The stored medication schedule of the user

[0304] Specific operation: The server executes a scheduling algorithm and generates an alert based on the next medication timing.

[0305] Output: The generated medication alert

[0306] Step 4:

[0307] The terminal receives the medication alert from the server and notifies the user.

[0308] Input: The medication alert sent from the server

[0309] Specific operation: The terminal displays an alert message to the user through a push notification.

[0310] Output: Alert notification to the user

[0311] Step 5:

[0312] When the user takes the medicine, the terminal performs face recognition and biometric information measurement.

[0313] Input: The start of taking medicine by the user

[0314] Specific actions: The device's camera is activated, facial recognition technology is used to verify the user's identity, and biometric sensors are used to measure heart rate and other parameters.

[0315] Output: Measured biometric data

[0316] Step 6:

[0317] The server analyzes biometric data transmitted from the terminal and generates feedback on the user's health status.

[0318] Input: Biometric data transmitted from the device

[0319] Specific actions: Analyze biometric data to assess the user's stress and health status. Create feedback messages based on the assessment results.

[0320] Output: Generated feedback message

[0321] Step 7:

[0322] Artificial intelligence, used as a generative AI tool, analyzes user prompts and provides appropriate responses.

[0323] Input: Prompt text entered by the user in the app

[0324] Specific operation: The AI ​​analyzes the prompt and generates appropriate advice by referring to past data. An example prompt is "How should I adjust the timing of my next dose?"

[0325] Output: AI-generated answers and advice

[0326] Step 8:

[0327] The server notifies the user's device of the feedback messages and AI responses generated by the server.

[0328] Input: Generated feedback message, AI response

[0329] Specific action: The server sends a notification to the terminal, allowing the user to check the message.

[0330] Output: Notification of feedback and advice to the user.

[0331] (Application Example 1)

[0332] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0333] Systems designed to support users in taking medications appropriately are required to promote continued treatment and reduce psychological and physical burden by providing real-time monitoring of health status and personalized feedback. However, conventional systems have the challenge of not being able to accurately evaluate physical reactions and changes in health status during medication administration and take appropriate action on the spot.

[0334] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0335] In this invention, the server includes an information processing device that collects the user's physical information and generates medication notifications; a terminal device that acquires the user's identification information and biometric information; an intelligent engine means that communicates with the user based on the generated information; and a transaction function that purchases medical-related products using the user's approval means. This allows the user to monitor their health status in real time, receive appropriate feedback, and easily purchase necessary medical products.

[0336] A "user" refers to an individual who uses the system to take medication and manages their health status and purchases medications in the process.

[0337] "Physical information" refers to data indicating the user's health status, including physiological data acquired in real time, such as heart rate and stress level.

[0338] A "medication notification" is a reminder generated to inform the user of the appropriate timing for taking their medication.

[0339] An "information processing device" refers to a computer system that collects a user's physical information and processes it to generate medication notifications.

[0340] "Identification information" refers to data used to identify users through methods such as facial recognition.

[0341] "Terminal device" refers to a terminal device used to acquire user identification information and biometric information.

[0342] "Biometric information" refers to information that includes data about the user's physical condition, such as heart rate and stress level.

[0343] An "intelligent engine means" is a processing system that utilizes artificial intelligence to communicate with users based on the information it generates.

[0344] "Approval process" refers to the process used to confirm a user's intention to purchase medical-related products.

[0345] The "transaction function" refers to a system feature used by users to purchase medical-related products.

[0346] One embodiment of this invention is a comprehensive system in which a user, server, and terminal work together to provide a user with the ability to take medication appropriately and manage their health. The specific embodiment is described below.

[0347] Users log in to the system using a smartphone, which is the end device, and its facial recognition function. The smartphone's camera and sensors for biometric information sensing are used for facial recognition. This device acquires the user's identification information and physical information, such as heart rate and stress level.

[0348] The server operates as an information processing unit using Python and the Flask framework, storing and analyzing real-time acquired user physical information. Based on this information, it generates medication notifications and sends reminders to the user. Furthermore, the server utilizes a generative AI model as an intelligent engine to generate feedback and motivational messages regarding the user's health status.

[0349] For example, if a user has not yet taken their medication today, the server analyzes the user's health data and past medication patterns, and in response to the question, "Is it okay that I took my medication late today?", it advises, "Let's adjust your future medication times so that your progress is not affected."

[0350] Example prompt: "User A took medication today. Their stress level is high; please generate specific advice for relaxation."

[0351] In this way, users can take their medication at the appropriate time and easily purchase medical products with the support of an intelligent engine.

[0352] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0353] Step 1:

[0354] The device activates the facial recognition function when the user operates their smartphone. The input is camera data from the smartphone, and the facial recognition algorithm is used to obtain the user's identification information. This allows the device to log the user in.

[0355] Step 2:

[0356] The device acquires the user's biometric information using sensors. The input data includes heart rate and stress levels, which are measured and temporarily stored within the device. The acquired biometric information is used to assess the user's physical condition.

[0357] Step 3:

[0358] The server receives the user's biometric and identification information from the terminal. The input is data transmitted from the terminal, which is stored in a database and analyzed by an information processing device. The output of the analysis is a health assessment of the user.

[0359] Step 4:

[0360] The server generates medication reminders for the user based on a health assessment. The input is the analyzed health status, and a scheduling algorithm is used to determine and notify the user of the next dose timing. The output is a reminder sent to the user's smartphone.

[0361] Step 5:

[0362] The server activates an intelligent engine and uses a generative AI model to generate feedback in response to the user's question, "I was late taking my medication today, is that okay?". The input is the user's past medication data and health assessment, and the output is personalized advice.

[0363] Step 6:

[0364] The device displays feedback and notifications from the server to the user. The input consists of reminders and advice received from the server, which are displayed in an easy-to-understand format, allowing the user to make decisions based on that information.

[0365] Step 7:

[0366] Users purchase necessary medical products using an in-app approval mechanism. Inputs include recommended products from the server and user approval; output is a purchase completion notification. The terminal records purchase history for future recommendations.

[0367] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0368] This invention enhances support for continuing sublingual immunotherapy by using a system that integrates the user, terminal, server, and emotion engine. The server manages information from the user and generates medication alerts and accumulates the user's emotional data based on that data. Through this, the aim is to maintain the user's physical and mental health.

[0369] The device is responsible for collecting user health data and uses facial recognition technology and biometric data measurement functions to understand the user's current state. Furthermore, by incorporating an emotion engine, it recognizes the user's emotions in real time from facial analysis and biometric data. For example, if the user shows an anxious expression, the emotion engine detects this state and sends that information to the server.

[0370] Based on the received emotional data, the server dynamically modifies the content of advice and motivational messages provided to the user via a generating AI. For example, if the emotional engine determines that the user is feeling down, the server can generate a positive and encouraging message and display it to the user through their device. In this way, personalized support that takes the user's emotions into account is provided, aiming to motivate them to continue treatment.

[0371] By using a device equipped with an emotion engine, users can visualize their own emotions and health status. The application provides feedback to the user and helps maintain a balance between health and emotions as part of their daily treatment. For example, when a user is feeling stressed, the application can suggest relaxation methods and simple exercises, making the treatment process more palatable.

[0372] This system provides emotional support to users undergoing sublingual immunotherapy, creating an environment that maximizes the effectiveness of the treatment. Thus, the configuration disclosed herein not only enhances conventional treatment support systems but also aims for more comprehensive user support through a new approach that utilizes emotion recognition.

[0373] The following describes the processing flow.

[0374] Step 1:

[0375] The user launches the smartphone app and prepares to check their health status. They log in and access the app's main screen.

[0376] Step 2:

[0377] The device activates its camera and takes a picture of the user's face. Using facial recognition technology, it identifies the user's face and acquires the image data.

[0378] Step 3:

[0379] The device analyzes facial features using the acquired facial image, and the emotion engine recognizes the user's emotions from their facial expressions. In this process, it detects facial expression patterns such as smiles and anxiety.

[0380] Step 4:

[0381] The device uses sensors to measure biometric information such as the user's heart rate and skin potential. An emotion engine analyzes this data to supplement the user's emotional state.

[0382] Step 5:

[0383] The device sends the user's emotional state, as recognized by the emotion engine, to the server. This information includes data on the type and intensity of the emotion.

[0384] Step 6:

[0385] The server integrates received emotional and health data to assess the user's overall health status. Based on the analysis results, a generative AI generates optimal feedback.

[0386] Step 7:

[0387] The server sends the generated feedback to the terminal and displays it to the user. This includes advice on the user's current health status and motivational messages tailored to their emotions.

[0388] Step 8:

[0389] Users review the displayed feedback and, if necessary, perform the recommended relaxation methods or exercises.

[0390] Step 9:

[0391] Users maintain motivation to continue treatment by continuing to manage their health based on daily treatment and feedback.

[0392] (Example 2)

[0393] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0394] A challenge in continuing sublingual immunotherapy is the lack of a system that provides support that considers not only the user's physical health but also their emotional health. Conventional systems have been unable to adequately grasp the user's emotional state, limiting their ability to provide personalized support.

[0395] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0396] In this invention, the server includes terminal means for collecting the user's biometric data and performing emotion analysis, means for managing the user's emotion data and biometric information and dynamically generating messages, and artificial intelligence means for interacting with the user based on the generated messages. This makes it possible to analyze the user's emotional state in real time and provide personalized health advice and motivational messages based on that analysis.

[0397] "User biometric data" is a general term for data that indicates the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0398] "Emotional analysis" is a process that analyzes and determines a user's emotional state based on their facial expressions and biometric data.

[0399] "Terminal means" refers to electronic devices used to collect the user's biometric data and analyze their emotional state.

[0400] A "server system" is a central processing unit that receives data transmitted from terminals and performs data analysis and information generation.

[0401] "Dynamically generating messages" means creating messages with content that is optimal for the user's current emotional state at any given time.

[0402] "Artificial intelligence means" refers to a program or device that uses generated messages to conduct automated conversations with users.

[0403] "Personalized health advice" refers to individualized health management guidance provided based on the user's emotional state and health data.

[0404] A "motivational message" is a message intended to psychologically motivate or encourage a user.

[0405] This invention is a system that comprehensively supports the physical and emotional health of users undergoing sublingual immunotherapy. The system consists of a user, a terminal, a server, an emotion analysis engine, and a generative AI model.

[0406] The device uses built-in sensors to collect the user's biometric data in real time. This provides physiological indicators such as heart rate and blood pressure. Additionally, facial recognition technology using a camera analyzes the user's facial expressions and determines their emotional state. All of this data is input into the device's emotion analysis engine.

[0407] The server receives emotional data and biometric information transmitted from the terminal and stores it in a database. Based on this information, a generative AI model operates to dynamically generate personalized advice and encouraging messages for the user. For example, a prompt such as "The user appears to be stressed. Please suggest ways to relax." is input to the generative AI model. This process allows the server to generate the most appropriate message based on the user's state.

[0408] Users receive these messages through their devices, visualizing their emotional and physical state. Providing daily feedback to the system enables more precise personalization. For example, if a user is tagged as "anxious," the device can display specific suggestions such as, "We recommend taking a deep breath and going for a short walk."

[0409] In this way, the present invention provides a novel approach to effectively support users' sublingual immunotherapy, enabling the maintenance of overall health.

[0410] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0411] Step 1:

[0412] The device collects the user's biometric information and facial expression data. As input, it collects the user's heart rate, blood pressure, and facial image data. This information is acquired through sensors and a camera. As output, a dataset containing biometric information and image data is generated. Specifically, when the user stands in front of the device, the sensors automatically activate and data acquisition begins.

[0413] Step 2:

[0414] The device inputs collected data into an emotion analysis engine to analyze the user's emotional state. The input consists of collected biometric information and facial expression data, which the emotion analysis engine processes. Specifically, a facial recognition algorithm determines emotions from facial expressions. The output is an analysis result indicating the user's emotional state. The analysis result is expressed as an emotion label such as anxiety, joy, or stress.

[0415] Step 3:

[0416] The terminal sends analyzed emotion data to the server. The input is analyzed data indicating the user's emotional state. This data is processed and then sent to the server to accumulate data. The output is the emotion data stored on the server. Specifically, data is sent from the terminal to the server periodically.

[0417] Step 4:

[0418] The server uses a generative AI model based on received emotional data to generate messages for the user. The input is data on the user's emotional state. By inputting "The user is feeling anxious. Please suggest ways to relax." as a prompt into the generative AI model, a message containing advice is obtained as output. The generated message is customized based on the user's individual profile.

[0419] Step 5:

[0420] The terminal delivers messages sent from the server to the user. The input is the generated message from the server. The output is the message displayed on the terminal's screen. Specifically, the terminal notifies the user of the message and provides an interface for the user to confirm it. The user confirms the message, and feedback is sent back to the server as needed.

[0421] (Application Example 2)

[0422] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0423] In sublingual immunotherapy, comprehensive management of the user's physical and mental health is crucial to supporting their continued treatment. However, conventional systems lack support that takes emotional states into account, making it difficult to address the individual needs of users. Therefore, there is a need for methods that provide personalized support based on the user's emotions and health status.

[0424] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0425] In this invention, the server includes information processing means for collecting user health data and generating medication alerts; device means for recognizing the user's face and measuring biometric information; intelligent engine means for interacting with the user based on the generated information; and presentation means for analyzing the user's emotional state and suggesting appropriate products or services. This enables personalized support and product suggestions based on the user's emotional state.

[0426] "Information processing means" refers to a device or system that has the function of generating appropriate medication alerts by collecting and analyzing user health data.

[0427] A "device means" is a device that recognizes the user's face and measures biometric information to understand the user's latest health status.

[0428] An "intelligent engine" is an artificial intelligence system that interacts with users based on collected information and provides appropriate advice and feedback.

[0429] A "presentation method" is a system that analyzes the user's emotional state and, based on the results, proposes the most suitable products and services in real time.

[0430] To implement this invention, the system is primarily configured based on the interaction between servers, terminals, and users.

[0431] First, the server acts as an information processing device, storing and managing health data collected from users and generating appropriate medication alerts. The server uses a generative AI model to perform necessary data analysis based on the user's health status and emotions. High-speed server equipment is used for this process.

[0432] The device recognizes the user's face and measures biometric information. Computer vision technologies such as OpenCV using a camera are employed for face recognition, and a TensorFlow-based emotion recognition model is incorporated to analyze the user's emotional state. This enables real-time assessment of the user's health status.

[0433] Furthermore, the server, acting as an intelligent engine, generates optimal advice based on the recognized user's emotions and health data. This generation is based on a generative AI model and enables dynamic messaging using prompts. For example, if a user is showing signs of stress, inputting a prompt such as, "The customer appears anxious. Please create a suggestion for products that will have a relaxing effect," will allow the generative AI to provide an appropriate message.

[0434] As a means of presentation, the system makes suggestions on the device side based on the user's emotional state. The display is done using devices such as smart glasses or digital displays, allowing for real-time presentation of necessary products and services. Therefore, by using this system, users can receive personalized support, enabling continuous support for sublingual immunotherapy and effective product recommendations.

[0435] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0436] Step 1:

[0437] The device collects the user's health data. As input, it acquires the user's biometric information and facial images through its camera and sensors. The device uses a facial recognition algorithm to identify the user, compares and analyzes the obtained biometric data against pre-set baseline values, and sends the results to a server.

[0438] Step 2:

[0439] The server analyzes the received data to assess the user's health and emotional state. Using health data sent from the terminal as input, it recognizes the user's emotions through a generative AI model. The server utilizes text analysis and machine learning algorithms to quantify, for example, stress levels, and passes the results to the generative AI model.

[0440] Step 3:

[0441] The server uses a generative AI model to generate optimal advice and product suggestions for the user. The input is the results of the health status and emotion assessment mentioned earlier. The generative AI model utilizes preset prompts to generate suggestions for products that have a relaxing effect when the user shows signs of anxiety. For example, it uses the prompt, "The customer is showing signs of anxiety. Please create suggestions for products that have a relaxing effect."

[0442] Step 4:

[0443] The server sends generated advice and suggestions to the terminal, which then presents them to the user. It receives generated text messages as input and displays them on the terminal's display or smart glasses. The terminal provides information visually to the user through its user interface, allowing the user to decide on actions based on the suggestions.

[0444] Step 5:

[0445] The user reviews the presented products or advice and takes action as needed. Based on the information received, the user checks products with relaxing effects and makes purchases in stores or orders online. The user's response is fed back into the system via the terminal and used to improve the accuracy of future suggestions.

[0446] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0447] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0448] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0449] [Third Embodiment]

[0450] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0451] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0452] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0453] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0454] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0455] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0456] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0457] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0458] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0459] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0460] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0461] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0462] This invention constructs a system that supports the continuation of sublingual immunotherapy by utilizing a server, terminal, and user interface. The server stores the user's registered medication schedule and personal health data, and generates real-time alerts and motivational messages tailored to the user. This system incorporates individual program logic and employs a highly accurate scheduling algorithm that provides notifications based on various conditions.

[0463] The device accepts user input and activates its camera for facial recognition when medication is taken. While the user holds the medication under their tongue, the device measures biosignals and analyzes heart rate and stress levels. For example, it analyzes whether the user is relaxed or stressed and sends this data to a server. The server then generates feedback on the user's health status and notifies the device. This feedback aims to improve the user's sense of security regarding their treatment.

[0464] Users can interact with the generative AI through a smartphone app. When a user enters a question within the app, that information is sent to a server. The server uses the generative AI to analyze the question and generate an appropriate answer. This AI provides the best possible answer for the user by referring to a vast amount of medical data and past conversation logs. For example, in response to the question, "Is it okay that I took my medication late today?", it can provide personalized advice such as, "Let's adjust the timing of future doses so that your progress is not affected."

[0465] This system aims to support users in making treatment a habit and improve the likelihood of continuing medication, thereby promoting long-term health improvement. In this way, the present invention reduces the psychological and physical burden on users and helps alleviate anxieties about health and performance between treatments.

[0466] The following describes the processing flow.

[0467] Step 1:

[0468] The user launches the smartphone app and logs into their account. Here, they can check their medication schedule and personal information.

[0469] Step 2:

[0470] The device connects to the server and receives the user's current medication schedule and health data. This data is used to generate the next medication reminder.

[0471] Step 3:

[0472] The server generates medication alerts based on user data. It schedules notifications to be sent to the device based on pre-set times.

[0473] Step 4:

[0474] The device will notify the user of an alert at a specified time. This can be done via sound, vibration, or screen display.

[0475] Step 5:

[0476] The user checks the notification and prepares to start taking the medication by pressing the "Start Taking Medication" button within the app.

[0477] Step 6:

[0478] After confirming that the user is ready, the device activates its camera and takes a picture of the user's face. Face recognition is performed at this stage.

[0479] Step 7:

[0480] The device analyzes captured facial images to measure heart rate and stress levels. Biometric data is processed in real time.

[0481] Step 8:

[0482] The server checks the biometric data received from the terminal, generates a warning if any abnormal values ​​are found, and notifies the terminal.

[0483] Step 9:

[0484] Users can use the in-app chat function to ask the AI ​​questions about alerts and health status.

[0485] Step 10:

[0486] The server receives a question and uses a generative AI to perform natural language processing analysis. Based on the analysis results, it generates an appropriate answer.

[0487] Step 11:

[0488] The terminal displays the generated response to the user and provides further information as needed.

[0489] Step 12:

[0490] After reviewing the feedback, users can continue treatment by reviewing their next medication schedule or asking further questions.

[0491] (Example 1)

[0492] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0493] Support is needed to improve the continuity of treatment in sublingual immunotherapy, but conventional methods lack appropriate feedback tailored to the individual circumstances and psychological state of the user. In addition, there is a need for means to provide timely medication alerts and personalized advice.

[0494] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0495] In this invention, the server includes information management means for collecting the user's medical information and generating medication warnings based on a high-precision scheduling algorithm, input means for performing facial recognition of the user and measuring biosignals, and generation AI means for engaging in dialogue with the user based on the analyzed information. This makes it possible to provide alerts and generate feedback tailored to the user's individual treatment status.

[0496] "Information management means" refers to a system element that has the function of collecting the user's medical information and generating medication warnings based on a highly accurate scheduling algorithm.

[0497] "Input means" refers to system elements used for facial recognition of the user and measurement of biosignals.

[0498] A "generative AI means" is a system element equipped with artificial intelligence functions for interacting with users based on analyzed information.

[0499] This invention relates to a system designed to support the continuation of sublingual immunotherapy, and includes a server, a terminal, and a user interface as its main components.

[0500] The server collects the user's medical information through information management systems and processes the stored data based on a high-precision scheduling algorithm to generate timely medication reminders. These reminders are sent to the user's smartphone or computer to help them proceed with their treatment as planned.

[0501] The device functions as an input device, using a camera and biosensors to recognize the user's face and measure biosignals. For example, when a user takes medication, the device uses facial recognition to identify the user and measures heart rate and other stress indicators using a heart rate sensor. This makes it possible to continuously monitor the user's current physical condition.

[0502] Artificial intelligence used as a generative AI tool analyzes data stored on a server and provides feedback and advice in response to user questions and requests. For example, if a user enters a question into the app such as, "How should I adjust my next dose?", the AI ​​can analyze the question and provide specific advice such as, "Let's try setting your next dose for 6 PM." Through such prompt-based dialogue, generative AI can provide optimal support to the user.

[0503] The system is designed to promote long-term health improvement by providing psychological and physical support through personalized motivational messages and feedback, enabling users to make treatment a habit. In this way, the present invention aims to provide users with a safer and more effective treatment environment.

[0504] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0505] Step 1:

[0506] Users input their medication schedule and health information through a smartphone app.

[0507] Input: Dosage schedule, health information

[0508] Specific operation: Users register medical data such as daily medication times and allergy information using the app's input form.

[0509] Output: Saving user data to the database

[0510] Step 2:

[0511] The server receives user input data and stores the data using information management tools.

[0512] Input: User-entered medication schedule and health information

[0513] Specific operation: The server updates the database and saves it as data necessary for configuring a high-precision scheduling algorithm.

[0514] Output: Generate the next alert using the saved data.

[0515] Step 3:

[0516] The server analyzes the stored data and generates medication alerts based on a highly accurate scheduling algorithm.

[0517] Input: Saved user medication schedule

[0518] Specific operation: The server executes a scheduling algorithm and generates an alert based on the next dose timing.

[0519] Output: Generated medication alert

[0520] Step 4:

[0521] The device receives medication alerts from the server and notifies the user.

[0522] Input: Medication alert sent from the server

[0523] Specific action: The device displays an alert message to the user via push notification.

[0524] Output: Alert notification to the user

[0525] Step 5:

[0526] When a user takes medication, the device performs facial recognition and measures biometric information.

[0527] Input: User initiates medication

[0528] Specific actions: The device's camera is activated, facial recognition technology is used to verify the user's identity, and biometric sensors are used to measure heart rate and other parameters.

[0529] Output: Measured biometric data

[0530] Step 6:

[0531] The server analyzes biometric data transmitted from the terminal and generates feedback on the user's health status.

[0532] Input: Biometric data transmitted from the device

[0533] Specific actions: Analyze biometric data to assess the user's stress and health status. Create feedback messages based on the assessment results.

[0534] Output: Generated feedback message

[0535] Step 7:

[0536] Artificial intelligence, used as a generative AI tool, analyzes user prompts and provides appropriate responses.

[0537] Input: Prompt text entered by the user in the app

[0538] Specific operation: The AI ​​analyzes the prompt and generates appropriate advice by referring to past data. An example prompt is "How should I adjust the timing of my next dose?"

[0539] Output: AI-generated answers and advice

[0540] Step 8:

[0541] The server notifies the user's device of the feedback messages and AI responses generated by the server.

[0542] Input: Generated feedback message, AI response

[0543] Specific action: The server sends a notification to the terminal, allowing the user to check the message.

[0544] Output: Notification of feedback and advice to the user.

[0545] (Application Example 1)

[0546] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0547] Systems designed to support users in taking medications appropriately are required to promote continued treatment and reduce psychological and physical burden by providing real-time monitoring of health status and personalized feedback. However, conventional systems have the challenge of not being able to accurately evaluate physical reactions and changes in health status during medication administration and take appropriate action on the spot.

[0548] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0549] In this invention, the server includes an information processing device that collects the user's physical information and generates medication notifications; a terminal device that acquires the user's identification information and biometric information; an intelligent engine means that communicates with the user based on the generated information; and a transaction function that purchases medical-related products using the user's approval means. This allows the user to monitor their health status in real time, receive appropriate feedback, and easily purchase necessary medical products.

[0550] A "user" refers to an individual who uses the system to take medication and manages their health status and purchases medications in the process.

[0551] "Physical information" refers to data indicating the user's health status, including physiological data acquired in real time, such as heart rate and stress level.

[0552] A "medication notification" is a reminder generated to inform the user of the appropriate timing for taking their medication.

[0553] An "information processing device" refers to a computer system that collects a user's physical information and processes it to generate medication notifications.

[0554] "Identification information" refers to data used to identify users through methods such as facial recognition.

[0555] "Terminal device" refers to a terminal device used to acquire user identification information and biometric information.

[0556] "Biometric information" refers to information that includes data about the user's physical condition, such as heart rate and stress level.

[0557] An "intelligent engine means" is a processing system that utilizes artificial intelligence to communicate with users based on the information it generates.

[0558] "Approval process" refers to the process used to confirm a user's intention to purchase medical-related products.

[0559] The "transaction function" refers to a system feature used by users to purchase medical-related products.

[0560] One embodiment of this invention is a comprehensive system in which a user, server, and terminal work together to provide a user with the ability to take medication appropriately and manage their health. The specific embodiment is described below.

[0561] Users log in to the system using a smartphone, which is the end device, and its facial recognition function. The smartphone's camera and sensors for biometric information sensing are used for facial recognition. This device acquires the user's identification information and physical information, such as heart rate and stress level.

[0562] The server operates as an information processing unit using Python and the Flask framework, storing and analyzing real-time acquired user physical information. Based on this information, it generates medication notifications and sends reminders to the user. Furthermore, the server utilizes a generative AI model as an intelligent engine to generate feedback and motivational messages regarding the user's health status.

[0563] For example, if a user has not yet taken their medication today, the server analyzes the user's health data and past medication patterns, and in response to the question, "Is it okay that I took my medication late today?", it advises, "Let's adjust your future medication times so that your progress is not affected."

[0564] Example prompt: "User A took medication today. Their stress level is high; please generate specific advice for relaxation."

[0565] In this way, users can take their medication at the appropriate time and easily purchase medical products with the support of an intelligent engine.

[0566] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0567] Step 1:

[0568] The device activates the facial recognition function when the user operates their smartphone. The input is camera data from the smartphone, and the facial recognition algorithm is used to obtain the user's identification information. This allows the device to log the user in.

[0569] Step 2:

[0570] The device acquires the user's biometric information using sensors. The input data includes heart rate and stress levels, which are measured and temporarily stored within the device. The acquired biometric information is used to assess the user's physical condition.

[0571] Step 3:

[0572] The server receives the user's biometric and identification information from the terminal. The input is data transmitted from the terminal, which is stored in a database and analyzed by an information processing device. The output of the analysis is a health assessment of the user.

[0573] Step 4:

[0574] The server generates medication reminders for the user based on a health assessment. The input is the analyzed health status, and a scheduling algorithm is used to determine and notify the user of the next dose timing. The output is a reminder sent to the user's smartphone.

[0575] Step 5:

[0576] The server activates an intelligent engine and uses a generative AI model to generate feedback in response to the user's question, "I was late taking my medication today, is that okay?". The input is the user's past medication data and health assessment, and the output is personalized advice.

[0577] Step 6:

[0578] The device displays feedback and notifications from the server to the user. The input consists of reminders and advice received from the server, which are displayed in an easy-to-understand format, allowing the user to make decisions based on that information.

[0579] Step 7:

[0580] Users purchase necessary medical products using an in-app approval mechanism. Inputs include recommended products from the server and user approval; output is a purchase completion notification. The terminal records purchase history for future recommendations.

[0581] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0582] This invention enhances support for continuing sublingual immunotherapy by using a system that integrates the user, terminal, server, and emotion engine. The server manages information from the user and generates medication alerts and accumulates the user's emotional data based on that data. Through this, the aim is to maintain the user's physical and mental health.

[0583] The device is responsible for collecting user health data and uses facial recognition technology and biometric data measurement functions to understand the user's current state. Furthermore, by incorporating an emotion engine, it recognizes the user's emotions in real time from facial analysis and biometric data. For example, if the user shows an anxious expression, the emotion engine detects this state and sends that information to the server.

[0584] Based on the received emotional data, the server dynamically modifies the content of advice and motivational messages provided to the user via a generating AI. For example, if the emotional engine determines that the user is feeling down, the server can generate a positive and encouraging message and display it to the user through their device. In this way, personalized support that takes the user's emotions into account is provided, aiming to motivate them to continue treatment.

[0585] By using a device equipped with an emotion engine, users can visualize their own emotions and health status. The application provides feedback to the user and helps maintain a balance between health and emotions as part of their daily treatment. For example, when a user is feeling stressed, the application can suggest relaxation methods and simple exercises, making the treatment process more palatable.

[0586] This system provides emotional support to users undergoing sublingual immunotherapy, creating an environment that maximizes the effectiveness of the treatment. Thus, the configuration disclosed herein not only enhances conventional treatment support systems but also aims for more comprehensive user support through a new approach that utilizes emotion recognition.

[0587] The following describes the processing flow.

[0588] Step 1:

[0589] The user launches the smartphone app and prepares to check their health status. They log in and access the app's main screen.

[0590] Step 2:

[0591] The device activates its camera and takes a picture of the user's face. Using facial recognition technology, it identifies the user's face and acquires the image data.

[0592] Step 3:

[0593] The device analyzes facial features using the acquired facial image, and the emotion engine recognizes the user's emotions from their facial expressions. In this process, it detects facial expression patterns such as smiles and anxiety.

[0594] Step 4:

[0595] The device uses sensors to measure biometric information such as the user's heart rate and skin potential. An emotion engine analyzes this data to supplement the user's emotional state.

[0596] Step 5:

[0597] The device sends the user's emotional state, as recognized by the emotion engine, to the server. This information includes data on the type and intensity of the emotion.

[0598] Step 6:

[0599] The server integrates received emotional and health data to assess the user's overall health status. Based on the analysis results, a generative AI generates optimal feedback.

[0600] Step 7:

[0601] The server sends the generated feedback to the terminal and displays it to the user. This includes advice on the user's current health status and motivational messages tailored to their emotions.

[0602] Step 8:

[0603] Users review the displayed feedback and, if necessary, perform the recommended relaxation methods or exercises.

[0604] Step 9:

[0605] Users maintain motivation to continue treatment by continuing to manage their health based on daily treatment and feedback.

[0606] (Example 2)

[0607] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0608] A challenge in continuing sublingual immunotherapy is the lack of a system that provides support that considers not only the user's physical health but also their emotional health. Conventional systems have been unable to adequately grasp the user's emotional state, limiting their ability to provide personalized support.

[0609] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0610] In this invention, the server includes terminal means for collecting the user's biometric data and performing emotion analysis, means for managing the user's emotion data and biometric information and dynamically generating messages, and artificial intelligence means for interacting with the user based on the generated messages. This makes it possible to analyze the user's emotional state in real time and provide personalized health advice and motivational messages based on that analysis.

[0611] "User biometric data" is a general term for data that indicates the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0612] "Emotional analysis" is a process that analyzes and determines a user's emotional state based on their facial expressions and biometric data.

[0613] "Terminal means" refers to electronic devices used to collect the user's biometric data and analyze their emotional state.

[0614] A "server system" is a central processing unit that receives data transmitted from terminals and performs data analysis and information generation.

[0615] "Dynamically generating messages" means creating messages with content that is optimal for the user's current emotional state at any given time.

[0616] "Artificial intelligence means" refers to a program or device that uses generated messages to conduct automated conversations with users.

[0617] "Personalized health advice" refers to individualized health management guidance provided based on the user's emotional state and health data.

[0618] A "motivational message" is a message intended to psychologically motivate or encourage a user.

[0619] This invention is a system that comprehensively supports the physical and emotional health of users undergoing sublingual immunotherapy. The system consists of a user, a terminal, a server, an emotion analysis engine, and a generative AI model.

[0620] The device uses built-in sensors to collect the user's biometric data in real time. This provides physiological indicators such as heart rate and blood pressure. Additionally, facial recognition technology using a camera analyzes the user's facial expressions and determines their emotional state. All of this data is input into the device's emotion analysis engine.

[0621] The server receives emotional data and biometric information transmitted from the terminal and stores it in a database. Based on this information, a generative AI model operates to dynamically generate personalized advice and encouraging messages for the user. For example, a prompt such as "The user appears to be stressed. Please suggest ways to relax." is input to the generative AI model. This process allows the server to generate the most appropriate message based on the user's state.

[0622] Users receive these messages through their devices, visualizing their emotional and physical state. Providing daily feedback to the system enables more precise personalization. For example, if a user is tagged as "anxious," the device can display specific suggestions such as, "We recommend taking a deep breath and going for a short walk."

[0623] In this way, the present invention provides a novel approach to effectively support users' sublingual immunotherapy, enabling the maintenance of overall health.

[0624] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0625] Step 1:

[0626] The device collects the user's biometric information and facial expression data. As input, it collects the user's heart rate, blood pressure, and facial image data. This information is acquired through sensors and a camera. As output, a dataset containing biometric information and image data is generated. Specifically, when the user stands in front of the device, the sensors automatically activate and data acquisition begins.

[0627] Step 2:

[0628] The device inputs collected data into an emotion analysis engine to analyze the user's emotional state. The input consists of collected biometric information and facial expression data, which the emotion analysis engine processes. Specifically, a facial recognition algorithm determines emotions from facial expressions. The output is an analysis result indicating the user's emotional state. The analysis result is expressed as an emotion label such as anxiety, joy, or stress.

[0629] Step 3:

[0630] The terminal sends analyzed emotion data to the server. The input is analyzed data indicating the user's emotional state. This data is processed and then sent to the server to accumulate data. The output is the emotion data stored on the server. Specifically, data is sent from the terminal to the server periodically.

[0631] Step 4:

[0632] The server uses a generative AI model based on received emotional data to generate messages for the user. The input is data on the user's emotional state. By inputting "The user is feeling anxious. Please suggest ways to relax." as a prompt into the generative AI model, a message containing advice is obtained as output. The generated message is customized based on the user's individual profile.

[0633] Step 5:

[0634] The terminal delivers messages sent from the server to the user. The input is the generated message from the server. The output is the message displayed on the terminal's screen. Specifically, the terminal notifies the user of the message and provides an interface for the user to confirm it. The user confirms the message, and feedback is sent back to the server as needed.

[0635] (Application Example 2)

[0636] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0637] In sublingual immunotherapy, comprehensive management of the user's physical and mental health is crucial to supporting their continued treatment. However, conventional systems lack support that takes emotional states into account, making it difficult to address the individual needs of users. Therefore, there is a need for methods that provide personalized support based on the user's emotions and health status.

[0638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0639] In this invention, the server includes information processing means for collecting user health data and generating medication alerts; device means for recognizing the user's face and measuring biometric information; intelligent engine means for interacting with the user based on the generated information; and presentation means for analyzing the user's emotional state and suggesting appropriate products or services. This enables personalized support and product suggestions based on the user's emotional state.

[0640] "Information processing means" refers to a device or system that has the function of generating appropriate medication alerts by collecting and analyzing user health data.

[0641] A "device means" is a device that recognizes the user's face and measures biometric information to understand the user's latest health status.

[0642] An "intelligent engine" is an artificial intelligence system that interacts with users based on collected information and provides appropriate advice and feedback.

[0643] A "presentation method" is a system that analyzes the user's emotional state and, based on the results, proposes the most suitable products and services in real time.

[0644] To implement this invention, the system is primarily configured based on the interaction between servers, terminals, and users.

[0645] First, the server acts as an information processing device, storing and managing health data collected from users and generating appropriate medication alerts. The server uses a generative AI model to perform necessary data analysis based on the user's health status and emotions. High-speed server equipment is used for this process.

[0646] The device recognizes the user's face and measures biometric information. Computer vision technologies such as OpenCV using a camera are employed for face recognition, and a TensorFlow-based emotion recognition model is incorporated to analyze the user's emotional state. This enables real-time assessment of the user's health status.

[0647] Furthermore, the server, acting as an intelligent engine, generates optimal advice based on the recognized user's emotions and health data. This generation is based on a generative AI model and enables dynamic messaging using prompts. For example, if a user is showing signs of stress, inputting a prompt such as, "The customer appears anxious. Please create a suggestion for products that will have a relaxing effect," will allow the generative AI to provide an appropriate message.

[0648] As a means of presentation, the system makes suggestions on the device side based on the user's emotional state. The display is done using devices such as smart glasses or digital displays, allowing for real-time presentation of necessary products and services. Therefore, by using this system, users can receive personalized support, enabling continuous support for sublingual immunotherapy and effective product recommendations.

[0649] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0650] Step 1:

[0651] The device collects the user's health data. As input, it acquires the user's biometric information and facial images through its camera and sensors. The device uses a facial recognition algorithm to identify the user, compares and analyzes the obtained biometric data against pre-set baseline values, and sends the results to a server.

[0652] Step 2:

[0653] The server analyzes the received data to assess the user's health and emotional state. Using health data sent from the terminal as input, it recognizes the user's emotions through a generative AI model. The server utilizes text analysis and machine learning algorithms to quantify, for example, stress levels, and passes the results to the generative AI model.

[0654] Step 3:

[0655] The server uses a generative AI model to generate optimal advice and product suggestions for the user. The input is the results of the health status and emotion assessment mentioned earlier. The generative AI model utilizes preset prompts to generate suggestions for products that have a relaxing effect when the user shows signs of anxiety. For example, it uses the prompt, "The customer is showing signs of anxiety. Please create suggestions for products that have a relaxing effect."

[0656] Step 4:

[0657] The server sends generated advice and suggestions to the terminal, which then presents them to the user. It receives generated text messages as input and displays them on the terminal's display or smart glasses. The terminal provides information visually to the user through its user interface, allowing the user to decide on actions based on the suggestions.

[0658] Step 5:

[0659] The user reviews the presented products or advice and takes action as needed. Based on the information received, the user checks products with relaxing effects and makes purchases in stores or orders online. The user's response is fed back into the system via the terminal and used to improve the accuracy of future suggestions.

[0660] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0661] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0662] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0663] [Fourth Embodiment]

[0664] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0665] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0666] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0667] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0668] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0669] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0670] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0671] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0672] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0673] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0674] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0675] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0676] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0677] This invention constructs a system that supports the continuation of sublingual immunotherapy by utilizing a server, terminal, and user interface. The server stores the user's registered medication schedule and personal health data, and generates real-time alerts and motivational messages tailored to the user. This system incorporates individual program logic and employs a highly accurate scheduling algorithm that provides notifications based on various conditions.

[0678] The device accepts user input and activates its camera for facial recognition when medication is taken. While the user holds the medication under their tongue, the device measures biosignals and analyzes heart rate and stress levels. For example, it analyzes whether the user is relaxed or stressed and sends this data to a server. The server then generates feedback on the user's health status and notifies the device. This feedback aims to improve the user's sense of security regarding their treatment.

[0679] Users can interact with the generative AI through a smartphone app. When a user enters a question within the app, that information is sent to a server. The server uses the generative AI to analyze the question and generate an appropriate answer. This AI provides the best possible answer for the user by referring to a vast amount of medical data and past conversation logs. For example, in response to the question, "Is it okay that I took my medication late today?", it can provide personalized advice such as, "Let's adjust the timing of future doses so that your progress is not affected."

[0680] This system aims to support users in making treatment a habit and improve the likelihood of continuing medication, thereby promoting long-term health improvement. In this way, the present invention reduces the psychological and physical burden on users and helps alleviate anxieties about health and performance between treatments.

[0681] The following describes the processing flow.

[0682] Step 1:

[0683] The user launches the smartphone app and logs into their account. Here, they can check their medication schedule and personal information.

[0684] Step 2:

[0685] The device connects to the server and receives the user's current medication schedule and health data. This data is used to generate the next medication reminder.

[0686] Step 3:

[0687] The server generates medication alerts based on user data. It schedules notifications to be sent to the device based on pre-set times.

[0688] Step 4:

[0689] The device will notify the user of an alert at a specified time. This can be done via sound, vibration, or screen display.

[0690] Step 5:

[0691] The user checks the notification and prepares to start taking the medication by pressing the "Start Taking Medication" button within the app.

[0692] Step 6:

[0693] After confirming that the user is ready, the device activates its camera and takes a picture of the user's face. Face recognition is performed at this stage.

[0694] Step 7:

[0695] The device analyzes captured facial images to measure heart rate and stress levels. Biometric data is processed in real time.

[0696] Step 8:

[0697] The server checks the biometric data received from the terminal, generates a warning if any abnormal values ​​are found, and notifies the terminal.

[0698] Step 9:

[0699] Users can use the in-app chat function to ask the AI ​​questions about alerts and health status.

[0700] Step 10:

[0701] The server receives a question and uses a generative AI to perform natural language processing analysis. Based on the analysis results, it generates an appropriate answer.

[0702] Step 11:

[0703] The terminal displays the generated response to the user and provides further information as needed.

[0704] Step 12:

[0705] After reviewing the feedback, users can continue treatment by reviewing their next medication schedule or asking further questions.

[0706] (Example 1)

[0707] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0708] Support is needed to improve the continuity of treatment in sublingual immunotherapy, but conventional methods lack appropriate feedback tailored to the individual circumstances and psychological state of the user. In addition, there is a need for means to provide timely medication alerts and personalized advice.

[0709] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0710] In this invention, the server includes information management means for collecting the user's medical information and generating medication warnings based on a high-precision scheduling algorithm, input means for performing facial recognition of the user and measuring biosignals, and generation AI means for engaging in dialogue with the user based on the analyzed information. This makes it possible to provide alerts and generate feedback tailored to the user's individual treatment status.

[0711] "Information management means" refers to a system element that has the function of collecting the user's medical information and generating medication warnings based on a highly accurate scheduling algorithm.

[0712] "Input means" refers to system elements used for facial recognition of the user and measurement of biosignals.

[0713] A "generative AI means" is a system element equipped with artificial intelligence functions for interacting with users based on analyzed information.

[0714] This invention relates to a system designed to support the continuation of sublingual immunotherapy, and includes a server, a terminal, and a user interface as its main components.

[0715] The server collects the user's medical information through information management systems and processes the stored data based on a high-precision scheduling algorithm to generate timely medication reminders. These reminders are sent to the user's smartphone or computer to help them proceed with their treatment as planned.

[0716] The device functions as an input device, using a camera and biosensors to recognize the user's face and measure biosignals. For example, when a user takes medication, the device uses facial recognition to identify the user and measures heart rate and other stress indicators using a heart rate sensor. This makes it possible to continuously monitor the user's current physical condition.

[0717] Artificial intelligence used as a generative AI tool analyzes data stored on a server and provides feedback and advice in response to user questions and requests. For example, if a user enters a question into the app such as, "How should I adjust my next dose?", the AI ​​can analyze the question and provide specific advice such as, "Let's try setting your next dose for 6 PM." Through such prompt-based dialogue, generative AI can provide optimal support to the user.

[0718] The system is designed to promote long-term health improvement by providing psychological and physical support through personalized motivational messages and feedback, enabling users to make treatment a habit. In this way, the present invention aims to provide users with a safer and more effective treatment environment.

[0719] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0720] Step 1:

[0721] Users input their medication schedule and health information through a smartphone app.

[0722] Input: Dosage schedule, health information

[0723] Specific operation: Users register medical data such as daily medication times and allergy information using the app's input form.

[0724] Output: Saving user data to the database

[0725] Step 2:

[0726] The server receives user input data and stores the data using information management tools.

[0727] Input: User-entered medication schedule and health information

[0728] Specific operation: The server updates the database and saves it as data necessary for configuring a high-precision scheduling algorithm.

[0729] Output: Generate the next alert using the saved data.

[0730] Step 3:

[0731] The server analyzes the stored data and generates medication alerts based on a highly accurate scheduling algorithm.

[0732] Input: Saved user medication schedule

[0733] Specific operation: The server executes a scheduling algorithm and generates an alert based on the next dose timing.

[0734] Output: Generated medication alert

[0735] Step 4:

[0736] The device receives medication alerts from the server and notifies the user.

[0737] Input: Medication alert sent from the server

[0738] Specific action: The device displays an alert message to the user via push notification.

[0739] Output: Alert notification to the user

[0740] Step 5:

[0741] When a user takes medication, the device performs facial recognition and measures biometric information.

[0742] Input: User initiates medication

[0743] Specific actions: The device's camera is activated, facial recognition technology is used to verify the user's identity, and biometric sensors are used to measure heart rate and other parameters.

[0744] Output: Measured biometric data

[0745] Step 6:

[0746] The server analyzes biometric data transmitted from the terminal and generates feedback on the user's health status.

[0747] Input: Biometric data transmitted from the device

[0748] Specific actions: Analyze biometric data to assess the user's stress and health status. Create feedback messages based on the assessment results.

[0749] Output: Generated feedback message

[0750] Step 7:

[0751] Artificial intelligence, used as a generative AI tool, analyzes user prompts and provides appropriate responses.

[0752] Input: Prompt text entered by the user in the app

[0753] Specific operation: The AI ​​analyzes the prompt and generates appropriate advice by referring to past data. An example prompt is "How should I adjust the timing of my next dose?"

[0754] Output: AI-generated answers and advice

[0755] Step 8:

[0756] The server notifies the user's device of the feedback messages and AI responses generated by the server.

[0757] Input: Generated feedback message, AI response

[0758] Specific action: The server sends a notification to the terminal, allowing the user to check the message.

[0759] Output: Notification of feedback and advice to the user.

[0760] (Application Example 1)

[0761] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0762] Systems designed to support users in taking medications appropriately are required to promote continued treatment and reduce psychological and physical burden by providing real-time monitoring of health status and personalized feedback. However, conventional systems have the challenge of not being able to accurately evaluate physical reactions and changes in health status during medication administration and take appropriate action on the spot.

[0763] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0764] In this invention, the server includes an information processing device that collects the user's physical information and generates medication notifications; a terminal device that acquires the user's identification information and biometric information; an intelligent engine means that communicates with the user based on the generated information; and a transaction function that purchases medical-related products using the user's approval means. This allows the user to monitor their health status in real time, receive appropriate feedback, and easily purchase necessary medical products.

[0765] A "user" refers to an individual who uses the system to take medication and manages their health status and purchases medications in the process.

[0766] "Physical information" refers to data indicating the user's health status, including physiological data acquired in real time, such as heart rate and stress level.

[0767] A "medication notification" is a reminder generated to inform the user of the appropriate timing for taking their medication.

[0768] An "information processing device" refers to a computer system that collects a user's physical information and processes it to generate medication notifications.

[0769] "Identification information" refers to data used to identify users through methods such as facial recognition.

[0770] "Terminal device" refers to a terminal device used to acquire user identification information and biometric information.

[0771] "Biometric information" refers to information that includes data about the user's physical condition, such as heart rate and stress level.

[0772] An "intelligent engine means" is a processing system that utilizes artificial intelligence to communicate with users based on the information it generates.

[0773] "Approval process" refers to the process used to confirm a user's intention to purchase medical-related products.

[0774] The "transaction function" refers to a system feature used by users to purchase medical-related products.

[0775] One embodiment of this invention is a comprehensive system in which a user, server, and terminal work together to provide a user with the ability to take medication appropriately and manage their health. The specific embodiment is described below.

[0776] Users log in to the system using a smartphone, which is the end device, and its facial recognition function. The smartphone's camera and sensors for biometric information sensing are used for facial recognition. This device acquires the user's identification information and physical information, such as heart rate and stress level.

[0777] The server operates as an information processing unit using Python and the Flask framework, storing and analyzing real-time acquired user physical information. Based on this information, it generates medication notifications and sends reminders to the user. Furthermore, the server utilizes a generative AI model as an intelligent engine to generate feedback and motivational messages regarding the user's health status.

[0778] For example, if a user has not yet taken their medication today, the server analyzes the user's health data and past medication patterns, and in response to the question, "Is it okay that I took my medication late today?", it advises, "Let's adjust your future medication times so that your progress is not affected."

[0779] Example prompt: "User A took medication today. Their stress level is high; please generate specific advice for relaxation."

[0780] In this way, users can take their medication at the appropriate time and easily purchase medical products with the support of an intelligent engine.

[0781] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0782] Step 1:

[0783] The device activates the facial recognition function when the user operates their smartphone. The input is camera data from the smartphone, and the facial recognition algorithm is used to obtain the user's identification information. This allows the device to log the user in.

[0784] Step 2:

[0785] The device acquires the user's biometric information using sensors. The input data includes heart rate and stress levels, which are measured and temporarily stored within the device. The acquired biometric information is used to assess the user's physical condition.

[0786] Step 3:

[0787] The server receives the user's biometric and identification information from the terminal. The input is data transmitted from the terminal, which is stored in a database and analyzed by an information processing device. The output of the analysis is a health assessment of the user.

[0788] Step 4:

[0789] The server generates medication reminders for the user based on a health assessment. The input is the analyzed health status, and a scheduling algorithm is used to determine and notify the user of the next dose timing. The output is a reminder sent to the user's smartphone.

[0790] Step 5:

[0791] The server activates an intelligent engine and uses a generative AI model to generate feedback in response to the user's question, "I was late taking my medication today, is that okay?". The input is the user's past medication data and health assessment, and the output is personalized advice.

[0792] Step 6:

[0793] The device displays feedback and notifications from the server to the user. The input consists of reminders and advice received from the server, which are displayed in an easy-to-understand format, allowing the user to make decisions based on that information.

[0794] Step 7:

[0795] Users purchase necessary medical products using an in-app approval mechanism. Inputs include recommended products from the server and user approval; output is a purchase completion notification. The terminal records purchase history for future recommendations.

[0796] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0797] This invention enhances support for continuing sublingual immunotherapy by using a system that integrates the user, terminal, server, and emotion engine. The server manages information from the user and generates medication alerts and accumulates the user's emotional data based on that data. Through this, the aim is to maintain the user's physical and mental health.

[0798] The device is responsible for collecting user health data and uses facial recognition technology and biometric data measurement functions to understand the user's current state. Furthermore, by incorporating an emotion engine, it recognizes the user's emotions in real time from facial analysis and biometric data. For example, if the user shows an anxious expression, the emotion engine detects this state and sends that information to the server.

[0799] Based on the received emotional data, the server dynamically modifies the content of advice and motivational messages provided to the user via a generating AI. For example, if the emotional engine determines that the user is feeling down, the server can generate a positive and encouraging message and display it to the user through their device. In this way, personalized support that takes the user's emotions into account is provided, aiming to motivate them to continue treatment.

[0800] By using a device equipped with an emotion engine, users can visualize their own emotions and health status. The application provides feedback to the user and helps maintain a balance between health and emotions as part of their daily treatment. For example, when a user is feeling stressed, the application can suggest relaxation methods and simple exercises, making the treatment process more palatable.

[0801] This system provides emotional support to users undergoing sublingual immunotherapy, creating an environment that maximizes the effectiveness of the treatment. Thus, the configuration disclosed herein not only enhances conventional treatment support systems but also aims for more comprehensive user support through a new approach that utilizes emotion recognition.

[0802] The following describes the processing flow.

[0803] Step 1:

[0804] The user launches the smartphone app and prepares to check their health status. They log in and access the app's main screen.

[0805] Step 2:

[0806] The device activates its camera and takes a picture of the user's face. Using facial recognition technology, it identifies the user's face and acquires the image data.

[0807] Step 3:

[0808] The device analyzes facial features using the acquired facial image, and the emotion engine recognizes the user's emotions from their facial expressions. In this process, it detects facial expression patterns such as smiles and anxiety.

[0809] Step 4:

[0810] The device uses sensors to measure biometric information such as the user's heart rate and skin potential. An emotion engine analyzes this data to supplement the user's emotional state.

[0811] Step 5:

[0812] The device sends the user's emotional state, as recognized by the emotion engine, to the server. This information includes data on the type and intensity of the emotion.

[0813] Step 6:

[0814] The server integrates received emotional and health data to assess the user's overall health status. Based on the analysis results, a generative AI generates optimal feedback.

[0815] Step 7:

[0816] The server sends the generated feedback to the terminal and displays it to the user. This includes advice on the user's current health status and motivational messages tailored to their emotions.

[0817] Step 8:

[0818] Users review the displayed feedback and, if necessary, perform the recommended relaxation methods or exercises.

[0819] Step 9:

[0820] Users maintain motivation to continue treatment by continuing to manage their health based on daily treatment and feedback.

[0821] (Example 2)

[0822] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0823] A challenge in continuing sublingual immunotherapy is the lack of a system that provides support that considers not only the user's physical health but also their emotional health. Conventional systems have been unable to adequately grasp the user's emotional state, limiting their ability to provide personalized support.

[0824] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0825] In this invention, the server includes terminal means for collecting the user's biometric data and performing emotion analysis, means for managing the user's emotion data and biometric information and dynamically generating messages, and artificial intelligence means for interacting with the user based on the generated messages. This makes it possible to analyze the user's emotional state in real time and provide personalized health advice and motivational messages based on that analysis.

[0826] "User biometric data" is a general term for data that indicates the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0827] "Emotional analysis" is a process that analyzes and determines a user's emotional state based on their facial expressions and biometric data.

[0828] "Terminal means" refers to electronic devices used to collect the user's biometric data and analyze their emotional state.

[0829] A "server system" is a central processing unit that receives data transmitted from terminals and performs data analysis and information generation.

[0830] "Dynamically generating messages" means creating messages with content that is optimal for the user's current emotional state at any given time.

[0831] "Artificial intelligence means" refers to a program or device that uses generated messages to conduct automated conversations with users.

[0832] "Personalized health advice" refers to individualized health management guidance provided based on the user's emotional state and health data.

[0833] A "motivational message" is a message intended to psychologically motivate or encourage a user.

[0834] This invention is a system that comprehensively supports the physical and emotional health of users undergoing sublingual immunotherapy. The system consists of a user, a terminal, a server, an emotion analysis engine, and a generative AI model.

[0835] The device uses built-in sensors to collect the user's biometric data in real time. This provides physiological indicators such as heart rate and blood pressure. Additionally, facial recognition technology using a camera analyzes the user's facial expressions and determines their emotional state. All of this data is input into the device's emotion analysis engine.

[0836] The server receives emotional data and biometric information transmitted from the terminal and stores it in a database. Based on this information, a generative AI model operates to dynamically generate personalized advice and encouraging messages for the user. For example, a prompt such as "The user appears to be stressed. Please suggest ways to relax." is input to the generative AI model. This process allows the server to generate the most appropriate message based on the user's state.

[0837] Users receive these messages through their devices, visualizing their emotional and physical state. Providing daily feedback to the system enables more precise personalization. For example, if a user is tagged as "anxious," the device can display specific suggestions such as, "We recommend taking a deep breath and going for a short walk."

[0838] In this way, the present invention provides a novel approach to effectively support users' sublingual immunotherapy, enabling the maintenance of overall health.

[0839] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0840] Step 1:

[0841] The device collects the user's biometric information and facial expression data. As input, it collects the user's heart rate, blood pressure, and facial image data. This information is acquired through sensors and a camera. As output, a dataset containing biometric information and image data is generated. Specifically, when the user stands in front of the device, the sensors automatically activate and data acquisition begins.

[0842] Step 2:

[0843] The device inputs collected data into an emotion analysis engine to analyze the user's emotional state. The input consists of collected biometric information and facial expression data, which the emotion analysis engine processes. Specifically, a facial recognition algorithm determines emotions from facial expressions. The output is an analysis result indicating the user's emotional state. The analysis result is expressed as an emotion label such as anxiety, joy, or stress.

[0844] Step 3:

[0845] The terminal sends analyzed emotion data to the server. The input is analyzed data indicating the user's emotional state. This data is processed and then sent to the server to accumulate data. The output is the emotion data stored on the server. Specifically, data is sent from the terminal to the server periodically.

[0846] Step 4:

[0847] The server uses a generative AI model based on received emotional data to generate messages for the user. The input is data on the user's emotional state. By inputting "The user is feeling anxious. Please suggest ways to relax." as a prompt into the generative AI model, a message containing advice is obtained as output. The generated message is customized based on the user's individual profile.

[0848] Step 5:

[0849] The terminal delivers messages sent from the server to the user. The input is the generated message from the server. The output is the message displayed on the terminal's screen. Specifically, the terminal notifies the user of the message and provides an interface for the user to confirm it. The user confirms the message, and feedback is sent back to the server as needed.

[0850] (Application Example 2)

[0851] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0852] In sublingual immunotherapy, comprehensive management of the user's physical and mental health is crucial to supporting their continued treatment. However, conventional systems lack support that takes emotional states into account, making it difficult to address the individual needs of users. Therefore, there is a need for methods that provide personalized support based on the user's emotions and health status.

[0853] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0854] In this invention, the server includes information processing means for collecting user health data and generating medication alerts; device means for recognizing the user's face and measuring biometric information; intelligent engine means for interacting with the user based on the generated information; and presentation means for analyzing the user's emotional state and suggesting appropriate products or services. This enables personalized support and product suggestions based on the user's emotional state.

[0855] "Information processing means" refers to a device or system that has the function of generating appropriate medication alerts by collecting and analyzing user health data.

[0856] A "device means" is a device that recognizes the user's face and measures biometric information to understand the user's latest health status.

[0857] An "intelligent engine" is an artificial intelligence system that interacts with users based on collected information and provides appropriate advice and feedback.

[0858] A "presentation method" is a system that analyzes the user's emotional state and, based on the results, proposes the most suitable products and services in real time.

[0859] To implement this invention, the system is primarily configured based on the interaction between servers, terminals, and users.

[0860] First, the server acts as an information processing device, storing and managing health data collected from users and generating appropriate medication alerts. The server uses a generative AI model to perform necessary data analysis based on the user's health status and emotions. High-speed server equipment is used for this process.

[0861] The device recognizes the user's face and measures biometric information. Computer vision technologies such as OpenCV using a camera are employed for face recognition, and a TensorFlow-based emotion recognition model is incorporated to analyze the user's emotional state. This enables real-time assessment of the user's health status.

[0862] Furthermore, the server, acting as an intelligent engine, generates optimal advice based on the recognized user's emotions and health data. This generation is based on a generative AI model and enables dynamic messaging using prompts. For example, if a user is showing signs of stress, inputting a prompt such as, "The customer appears anxious. Please create a suggestion for products that will have a relaxing effect," will allow the generative AI to provide an appropriate message.

[0863] As a means of presentation, the system makes suggestions on the device side based on the user's emotional state. The display is done using devices such as smart glasses or digital displays, allowing for real-time presentation of necessary products and services. Therefore, by using this system, users can receive personalized support, enabling continuous support for sublingual immunotherapy and effective product recommendations.

[0864] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0865] Step 1:

[0866] The device collects the user's health data. As input, it acquires the user's biometric information and facial images through its camera and sensors. The device uses a facial recognition algorithm to identify the user, compares and analyzes the obtained biometric data against pre-set baseline values, and sends the results to a server.

[0867] Step 2:

[0868] The server analyzes the received data to assess the user's health and emotional state. Using health data sent from the terminal as input, it recognizes the user's emotions through a generative AI model. The server utilizes text analysis and machine learning algorithms to quantify, for example, stress levels, and passes the results to the generative AI model.

[0869] Step 3:

[0870] The server uses a generative AI model to generate optimal advice and product suggestions for the user. The input is the results of the health status and emotion assessment mentioned earlier. The generative AI model utilizes preset prompts to generate suggestions for products that have a relaxing effect when the user shows signs of anxiety. For example, it uses the prompt, "The customer is showing signs of anxiety. Please create suggestions for products that have a relaxing effect."

[0871] Step 4:

[0872] The server sends generated advice and suggestions to the terminal, which then presents them to the user. It receives generated text messages as input and displays them on the terminal's display or smart glasses. The terminal provides information visually to the user through its user interface, allowing the user to decide on actions based on the suggestions.

[0873] Step 5:

[0874] The user reviews the presented products or advice and takes action as needed. Based on the information received, the user checks products with relaxing effects and makes purchases in stores or orders online. The user's response is fed back into the system via the terminal and used to improve the accuracy of future suggestions.

[0875] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0876] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0877] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0878] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0879] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0880] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0881] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0882] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0883] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0884] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0885] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0886] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0887] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0888] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0889] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0890] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0891] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0892] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0893] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0894] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0895] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0896] The following is further disclosed regarding the embodiments described above.

[0897] (Claim 1)

[0898] A server means for collecting user health data and generating medication alerts,

[0899] A terminal means for measuring the user's facial recognition and biometric information,

[0900] An artificial intelligence system that interacts with users based on the information it generates,

[0901] A system that includes this.

[0902] (Claim 2)

[0903] The system according to claim 1, which generates and notifies an alert based on the timing of taking prescribed medication.

[0904] (Claim 3)

[0905] The system according to claim 1, which analyzes a user's facial image to detect stress signs and evaluate their health status.

[0906] "Example 1"

[0907] (Claim 1)

[0908] An information management system that collects user medical information and generates medication warnings based on a highly accurate scheduling algorithm,

[0909] An input means that performs facial recognition of the user and measures biosignals,

[0910] A generative AI means that interacts with the user based on the analyzed information,

[0911] A system that includes this.

[0912] (Claim 2)

[0913] The system according to claim 1, which generates personalized messages to support the user in continuing treatment.

[0914] (Claim 3)

[0915] The system according to claim 1, which analyzes a user's facial image, detects stress indicators including a relaxed state, and evaluates their health status.

[0916] "Application Example 1"

[0917] (Claim 1)

[0918] An information processing device that collects user physical information and generates medication notification,

[0919] A terminal device that acquires user identification information and biometric information,

[0920] An intelligent engine means that communicates with the user based on the generated information,

[0921] A transaction function that allows users to purchase medical-related products using a means of user approval,

[0922] A system that includes this.

[0923] (Claim 2)

[0924] The system according to claim 1, which generates and provides notifications based on the prescribed medication administration schedule.

[0925] (Claim 3)

[0926] The system according to claim 1, which analyzes a user's facial image to detect psychological burden and evaluate their health status.

[0927] "Example 2 of combining an emotion engine"

[0928] (Claim 1)

[0929] A terminal device that collects user biometric data and performs emotion analysis,

[0930] A server means for managing user emotional data and biometric information and dynamically generating messages,

[0931] An artificial intelligence means that interacts with the user based on the generated message,

[0932] A means of visualizing the user's health status through a device and collecting feedback,

[0933] A system that includes this.

[0934] (Claim 2)

[0935] The system according to claim 1, which generates and provides motivational messages based on the user's emotional state.

[0936] (Claim 3)

[0937] The system according to claim 1, which analyzes the user's emotional patterns from collected data and generates personalized health advice.

[0938] "Application example 2 when combining with an emotional engine"

[0939] (Claim 1)

[0940] Information processing means for collecting user health data and generating medication alerts,

[0941] A device means for measuring user facial recognition and biometric information,

[0942] An intelligent engine means that interacts with the user based on the generated information,

[0943] A means of presenting appropriate products or services by analyzing the user's emotional state,

[0944] A system that includes this.

[0945] (Claim 2)

[0946] The system according to claim 1, which generates and notifies an alert based on the timing of taking prescribed medication.

[0947] (Claim 3)

[0948] The system according to claim 1, which analyzes a user's facial image to detect stress signs and evaluate their health status. [Explanation of Symbols]

[0949] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A server means for collecting user health data and generating medication alerts, A terminal means for measuring the user's facial recognition and biometric information, An artificial intelligence system that interacts with users based on the information it generates, A system that includes this.

2. The system according to claim 1, which generates and notifies an alert based on the timing of taking prescribed medication.

3. The system according to claim 1, which analyzes a user's facial image to detect stress signs and evaluate their health status.

Images