System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring biological information of a user; means for analyzing the biological information; means for making a diagnosis based on a result of the analysis and past medical information; means for making a prescription based on a result of the diagnosis; means for delivering the prescribed medicine; and means for observing a course of treatment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, with the advancement of busy lifestyles and an aging society, it has become increasingly difficult for ordinary people to easily manage their daily health and receive prompt and efficient medical consultations when necessary. In particular, when medical institutions are overwhelmed due to factors such as the spread of infectious diseases, there is a demand for remote health management and medical treatment. However, conventional systems have had difficulty providing a comprehensive range of services, from acquiring biometric information to diagnosis, prescriptions, and follow-up observation. Therefore, the present invention aims to provide a system that allows anyone to easily manage their daily health and, when necessary, receive remote medical consultations. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by the following means. Specifically, a system is provided that includes a means for acquiring a user's biometric information, a means for analyzing the biometric information, a means for making a diagnosis based on the analysis results and past medical information, a means for prescribing based on the diagnosis results, a means for delivering the prescribed medication, and a means for monitoring the progress of treatment. In particular, by acquiring the acquired biometric information using a camera on a smart device, including a means for interactively interviewing the user based on the analysis results, and including a means for analyzing the effects of treatment using a machine learning model, efficient and highly accurate health management and medical care can be achieved. Furthermore, since the prescribed medication is delivered to an address specified by the user, the user can receive the necessary medical care from the comfort of their own home.

[0006] A "user" is an individual who uses the system to receive health care and treatment.

[0007] "Biometric information" is data that indicates the user's physical condition, and includes measurements such as heart rate, respiratory rate, and body temperature.

[0008] "Means of acquisition" refers to the means of measuring biometric information using the camera or sensor of a smart device and collecting it as data.

[0009] The "means for analyzing" refers to a means for analyzing the acquired biometric information using machine learning algorithms or other data analysis techniques to assess the health condition.

[0010] The "means for performing diagnosis" is a means for evaluating the health condition of the user based on the analyzed biological information and past medical information, and providing a diagnosis result.

[0011] "Means of prescribing" refers to the means of selecting the necessary medicines based on the diagnostic results and issuing a prescription.

[0012] "Delivery means" refers to the means for sending the prescribed medicine to the address specified by the user.

[0013] "Means for monitoring progress" refers to means for continuously monitoring additional vital signs and progress of treatment provided by the user, and for assessing the effectiveness of treatment.

[0014] "Smart devices" are electronic devices equipped with internet connectivity and various sensors, such as mobile phones, tablets, and wearable devices that users use on a daily basis.

[0015] A "machine learning model" is an algorithm that learns from past data and predicts and classifies future data.

[0016] A "chatbot" is an artificial intelligence (AI) system that communicates with users in an interactive format.

[0017] "Past medical information" refers to medical-related data that has been accumulated up to now, such as the user's medical history, past medical examination results, and medication history. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] System Overview

[0040] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biological information, analyze data, make diagnoses, prescribe medicines, and monitor progress. Specific embodiments of each part of the system are described below.

[0041] Acquisition of biometric information

[0042] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[0043] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into the required format, and then sent to the server.

[0044] Data analysis

[0045] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model. The server evaluates the user's current health condition based on past accumulated data and the trained model. The analysis results are generated as a simple diagnosis.

[0046] diagnosis

[0047] Server: The simple diagnosis results are provided to the chatbot, which then collects additional information through dialogue with the user and conducts a medical interview. The data obtained through this interactive interview is then sent back to the server and used for the final diagnosis.

[0048] Terminal: The chatbot asks the user questions and collects their answers to gain more detailed information, which improves the accuracy of the diagnosis.

[0049] Prescription of medication

[0050] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[0051] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[0052] Users: Users can receive their medication at home, allowing them to receive the medical care they need without leaving their homes.

[0053] Observation

[0054] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[0055] Server: The server receives the biometric data again and analyzes it with a machine learning model to evaluate the effectiveness of treatment. The analysis results are notified to the user and doctor, and necessary measures are taken.

[0056] Specific examples

[0057] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment in the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies them of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera. The user follows the on-screen instructions to hold their face or hand in front of the camera to collect biometric information such as heart rate and respiratory rate.

[0061] Step 2:

[0062] Terminal: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the terminal and converted into an appropriate format. The data is then sent to the server.

[0063] Step 3:

[0064] Server: The server receives the biometric information sent from the device, stores the data in a database, analyzes the biometric information using a machine learning model, and generates a simple diagnosis result as the analysis result.

[0065] Step 4:

[0066] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates questions for the user and begins the medical interview.

[0067] Step 5:

[0068] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. These answers are then sent back to the server.

[0069] Step 6:

[0070] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes the final diagnosis and generates the results.

[0071] Step 7:

[0072] Server: Based on the final diagnosis, the doctor prescribes the necessary medication. The prescription is then electronically sent to the partner pharmacy.

[0073] Step 8:

[0074] Pharmacy: The pharmacy prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[0075] Step 9:

[0076] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[0077] Step 10:

[0078] Server: Receives progress information and additional biometric data entered by the user and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified and given appropriate instructions.

[0079] In this way, the system provides consistent daily health management and medical treatment through collaboration between users, terminals, and servers.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Conventional health management systems have had issues with the user having to spend a lot of time and effort to obtain biometric information, and the diagnosis and treatment process is fragmented, making it difficult to provide comprehensive medical care.In addition, it is difficult to perform real-time diagnosis or interactive interviews, making it difficult to quickly and accurately grasp the user's health condition.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for acquiring biometric information of a user, means for analyzing the biometric information, means for making a diagnosis based on the analysis result and past medical information, means for prescribing medicine based on the diagnosis result, means for delivering the prescribed medicine, means for monitoring the progress of treatment, means for activating a camera of a smart device to measure biometric information based on a health management option selected by the user, means for converting the biometric information into a predetermined format and transmitting it to the server, means for analyzing the biometric information using a machine learning model, and means for conducting an interview in an interactive format with the user using a chatbot, thereby enabling the user to receive consistent health management and medical services.

[0085] "User" refers to an individual who uses the system for a particular purpose.

[0086] "Biometric information" refers to various health-related data obtained from the human body (e.g., heart rate, respiratory rate, body temperature, etc.).

[0087] "Means" refers to a specific device, method, or part of a system used to achieve a particular purpose.

[0088] "Smart device" refers to a portable electronic device with advanced computing capabilities (e.g., smartphone, tablet, etc.).

[0089] "Camera" refers to a digital imaging device for capturing images.

[0090] "Analysis" refers to the process of examining the data obtained in detail and drawing conclusions or information.

[0091] "Server" refers to a high-performance computer system that manages, processes, and provides data over a network.

[0092] A "machine learning model" refers to a collection of algorithms and statistical models that learn from data and make predictions and decisions.

[0093] A "chatbot" refers to a program or artificial intelligence that mimics a conversation with a user.

[0094] "Format" refers to the particular structure or form of data.

[0095] "Medical interview" refers to the process by which a doctor gathers information from a patient about their symptoms and medical history.

[0096] "Diagnosis" refers to the process by which a medical professional identifies an illness or condition based on a patient's symptoms and test results.

[0097] "Prescription" refers to a doctor prescribing an appropriate medication to a patient.

[0098] "Pharmacy" means an establishment that prepares and dispenses medicines based on prescriptions.

[0099] "Delivery" refers to the process of getting an item to a designated location.

[0100] "Treatment" refers to medical procedures aimed at improving or recovering from an illness or disability.

[0101] "Follow-up" refers to the process of continually checking the progress of treatment and making any necessary adjustments.

[0102] The present invention is a system for supporting a user's health management, and is implemented using specific procedures, hardware, and software. This system acquires daily biological information and performs data analysis, diagnosis, drug prescription, and follow-up. Specific embodiments of each part are described below.

[0103] Acquisition of biometric information

[0104] User: The user launches the LINE app on their smart device and selects the "Health Management" option. The app activates the camera and displays an interface for measuring heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure their biometric information.

[0105] Device: The camera on the smart device captures and analyzes the user's video data, collecting biometric information such as heart rate, respiratory rate, and body temperature. This data is temporarily stored on the device, converted into a specific format, and then sent to a server.

[0106] Data analysis

[0107] Server: The server receives the transmitted biometric information and analyzes it using a machine learning model. The model used for the analysis is based on past accumulated data and training data. From the analysis results, the server evaluates the user's current health condition and generates a simple diagnosis.

[0108] diagnosis

[0109] Server: The simple diagnosis results are provided to the chatbot, which then asks the user additional questions. The chatbot then interacts with the user to gather more information and sends that data back to the server. The final diagnosis is based on this data.

[0110] Terminal: The terminal sends additional questions to the user through the chatbot and collects the user's answers, thereby improving the accuracy of the diagnosis.

[0111] Prescription of medication

[0112] Server: Based on the final diagnosis, the server generates instructions for the doctor to prescribe the necessary medication. The prescription is sent to a partner pharmacy, which prepares the medication and arranges for delivery.

[0113] User: The user receives the medication at home and takes it as directed.

[0114] Observation

[0115] User: The user can periodically enter the progress of treatment in the LINE app, and can obtain vital data again and send it to the server.

[0116] Server: The server receives the new biometric data and re-analyzes it using machine learning models. The results are then communicated to the user and doctor, and further action is taken if necessary.

[0117] (Example)

[0118] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data and evaluates the effectiveness of the treatment. This allows users to receive comprehensive health management and medical services from the comfort of their own home.

[0119] (Example of a prompt)

[0120] "Measure your heart rate and breathing rate."

[0121] "Please select the health management option and enter your data."

[0122] "I'll do a quick checkup. Please assess your health."

[0123] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0124] Step 1: Obtaining biometric information

[0125] User: The user launches the LINE app and enters health management mode by selecting the "Health Management" option. User action (selecting an option) is required as input.

[0126] Terminal: The terminal activates the smart device's camera and displays the interface. When the user holds their face or hand over the camera, the camera captures video data. Video data is provided as input data, and biometric information such as heart rate, respiratory rate, and body temperature is generated as output.

[0127] How it works: The camera module captures video data, which is then analyzed by internal software to measure biometric information, which is then temporarily stored in the device's memory.

[0128] Step 2: Sending biometric information

[0129] Terminal: The terminal converts the collected biometric data into a predetermined format. This conversion process includes noise removal and data normalization. As input, it requires raw biometric data, and as output, it obtains formatted data.

[0130] The formatted data is then sent to the server using the terminal's network module.

[0131] What it does: Specific algorithms and techniques are used to transform the data into a different format. The optimized data is then sent to the server using a secure protocol.

[0132] Step 3: Analyze the data

[0133] Server: The server receives the transmitted biometric data. It requires formatted biometric data transmitted from the device as input data. This data is then fed into a machine learning model and analyzed to assess the health status.

[0134] How it works: The machine learning model analyzes the data and detects abnormalities, such as heart rate fluctuation patterns or breathing rate. The analysis results are temporarily stored in a database on the server. The analysis results and a simple diagnosis are generated as output.

[0135] Step 4: Simple diagnosis and interview

[0136] Server: The server generates a simple diagnosis based on the analysis results. The input data is the analysis results of the machine learning model. This result is provided to the chatbot, which then begins interviewing the user.

[0137] Terminal: The terminal sends follow-up questions to the user through the chatbot and collects answers from the user. The input data includes the user's answers, and the output provides additional health information.

[0138] Specific actions: For example, specific prompts such as "Please tell us about your recent fatigue and appetite changes" are displayed to the user. These answers are sent to the server in real time.

[0139] Step 5: Sending diagnostic results and prescribing medication

[0140] Server: The server combines the medical interview data and biometric information to make the final diagnosis. The analysis results and medical interview data are required as input data. Based on this, the doctor prescribes the necessary medication and generates a prescription.

[0141] Specific operation: Prescription data is automatically generated and electronically transmitted to affiliated pharmacies. As an output, information on the completed prescription is generated.

[0142] Step 6: Follow-up and evaluation

[0143] User: The user periodically updates the progress of their treatment on the LINE app, and their vital data is acquired again and sent to the server. The input data includes new vital information and the progress of treatment.

[0144] Server: The server reanalyzes the updated information and uses machine learning models to evaluate the effectiveness of the treatment. The input data is the updated biometric information, and the output is the evaluation result of the treatment effectiveness.

[0145] Specific operation: Based on the analysis results, the server provides feedback to the user, such as "Treatment is progressing smoothly" or "Additional tests are required." The results are then notified to the user and their doctor.

[0146] (Application example 1)

[0147] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0148] The working environment for employees at logistics centers is harsh, making health management important. However, it is difficult for employees to properly monitor their own health status and detect abnormalities early amid their busy work schedules. Furthermore, there are limited means to respond quickly when abnormalities are discovered. Given this background, there is a need for an efficient and effective method of managing the health of employees working at logistics centers.

[0149] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0150] In this invention, the server includes means for acquiring user biometric information, means for analyzing the biometric information, means for monitoring the biometric information in real time if the user is an employee working at a logistics center, means for making a diagnosis based on the analysis results and past medical information, means for notifying a manager when abnormal biometric information is detected, means for prescribing medication based on the diagnosis results, means for delivering the prescribed medication, and means for monitoring the progress of treatment. This allows for efficient and effective health management of employees at the logistics center and enables rapid response to abnormalities.

[0151] "Users" refer to employees working at the logistics center.

[0152] "Biometric information" is data that indicates the user's health condition, such as heart rate, respiratory rate, and body temperature.

[0153] A "smart device" is an information device such as a smartphone or tablet that is carried by a user.

[0154] A "camera" is an image capture device built into a smart device.

[0155] "Analysis" refers to the process of analyzing collected biometric information using methods such as machine learning models to extract useful information.

[0156] "Diagnosis" refers to the act of evaluating the user's health condition based on the analysis results of biological information and past medical information.

[0157] "Prescription" is the act of a doctor prescribing the necessary medication based on diagnostic results.

[0158] "Delivery" refers to the act of delivering prescribed medicine to a location designated by the user.

[0159] "Follow-up" is the act of monitoring the progress of treatment and continually assessing health status.

[0160] "Administrator" refers to the person in charge of managing the health of employees at the logistics center.

[0161] The present invention is a system for supporting the health management of employees working at a logistics center. This system works in conjunction with smart devices to consistently acquire biometric information, analyze data, diagnose, notify abnormalities, prescribe medication, and monitor treatment progress. Specific embodiments of each part of the system are described below.

[0162] Acquisition of biometric information

[0163] User: The user launches the application on their smart device and holds their face or hand over the camera to measure their biometric information, including heart rate, respiratory rate, and body temperature.

[0164] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[0165] Data analysis

[0166] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model (for example, a model built with Keras). The server evaluates the user's current health status based on past accumulated data and the trained model. If an abnormality is detected, the analysis results will notify the administrator.

[0167] Diagnostics and Notifications

[0168] Server: Based on the analysis results and past medical information, the server diagnoses the user's health condition. If abnormal vital signs are detected, a notification is sent to the administrator, prompting prompt action.

[0169] Managers: Managers can monitor the health status of employees within the distribution center and take appropriate action if necessary.

[0170] Medication and follow-up

[0171] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[0172] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[0173] Users: Users can receive medication at home, allowing them to receive the medical care they need without leaving their homes. Furthermore, they can enter their treatment progress through an application on their smart device.

[0174] Server: The server receives the retransmitted vital signs and analyzes them with a machine learning model to evaluate the effectiveness of treatment. The analysis results are then notified to the user and doctor, and necessary measures are taken.

[0175] Specific examples

[0176] For example, if an employee working at a logistics center feels unwell, they first launch an application on their smart device and hold their face up to the camera to measure their heart rate and respiratory rate. The data is then sent to a server where it is analyzed using a machine learning model. If an abnormality is detected based on the analysis results, an alert is sent to an administrator. The administrator can then take prompt action, requesting a diagnosis from a doctor if necessary and prescribing medication. The user then receives the medication at home and records the progress of their treatment in the application, and the server analyzes the data to continue monitoring their condition.

[0177] Prompt Sentence Examples

[0178] For example, the following prompts could be fed into a generative AI model:

[0179] "We want to build a system in our logistics center that allows employees to monitor their daily health using their smartphones. Using smart devices, we can acquire vital signs (heart rate, respiratory rate, body temperature) and analyze them using machine learning models. If any abnormal values ​​are detected, a notification will be sent to the manager."

[0180] In this way, a system can be realized that comprehensively manages the health of employees working at logistics centers.

[0181] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0182] Step 1:

[0183] User: Launches the smart device application and holds their face or hand in front of the camera. The input is video data captured by the smart device's camera. The output is biometric information such as heart rate, respiratory rate, and body temperature.

[0184] Step 2:

[0185] Terminal: Analyzes video data captured by the smart device's camera to collect biometric information. Video data provided by the user is used as input. Analyzed biometric information is obtained as output. Specifically, the smart device's image processing function is used to estimate heart rate and respiratory rate. The data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[0186] Step 3:

[0187] Server: Receives the transmitted biometric information and analyzes it using a machine learning model (built using Keras). The biometric information sent from the device is used as input. The output is an assessment of the user's current health status. Specifically, the server compares the biometric information with an existing database to check for any abnormalities.

[0188] Step 4:

[0189] Server: Diagnoses the user's health condition based on the analysis results and past medical information. Analysis results and past medical information are used as input. Diagnosis results are obtained as output. Specifically, if an abnormality is detected based on the analysis results, the next step is notification processing.

[0190] Step 5:

[0191] Server: If abnormal biometric information is detected, it sends a notification to the administrator. The diagnosis results are used as input. The notification data to be sent to the administrator is obtained as output. Specifically, when an abnormality is detected, an alert is sent to the administrator's smart device using the LINE API.

[0192] Step 6:

[0193] Administrator: Receives notifications, monitors employee health status in real time, and takes appropriate action. The input is the notification sent from the server. The output is the response procedure to be taken by the administrator. Specifically, the administrator who receives the notification will request a diagnosis from a doctor if necessary, and ensure the safety of the employee.

[0194] Step 7:

[0195] Server: Based on the diagnosis results, the doctor prescribes the necessary medication. The diagnosis results are used as input. A prescription is generated as output. Specifically, the server creates an electronic prescription based on instructions from the doctor and sends it to a partner pharmacy.

[0196] Step 8:

[0197] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescription is used as input. The output is the medicine to be sent to the user's delivery address. The specific operation is that the pharmacy prepares the prescription and arranges for delivery to the specified address.

[0198] Step 9:

[0199] User: Receives medication at home and performs treatment. The input is the delivered medication. The output is data on the progress of treatment. The specific operation is that the user receives the medication and records the progress of treatment through the application.

[0200] Step 10:

[0201] Server: Receives the progress of treatment and analyzes the vital signs again. The input is the treatment progress data sent by the user. The output is the evaluation result of the treatment effect. Specifically, the server analyzes the new vital signs and checks whether the treatment is effective. The results are notified to the user and the doctor.

[0202] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0203] System Overview

[0204] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biometric information, analyze data, make diagnoses, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of the system are described below.

[0205] Acquisition of biometric information

[0206] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[0207] Device: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to the server.

[0208] Acquiring emotional information

[0209] Device: The smart device's camera and microphone are used to capture the user's facial expressions and vocal changes and send them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[0210] Data analysis

[0211] Server: The transmitted biometric and emotional information is received by the server and stored in a database. The server analyzes the biometric information using a machine learning model and also incorporates emotional information from the emotion engine into the analysis. This allows for a more accurate assessment of the user's health condition.

[0212] diagnosis

[0213] Server: The analysis results are generated as a simple diagnosis. This simple diagnosis result is provided to the LINE chatbot, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[0214] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. The chatbot asks questions taking into account the user's emotional state, allowing the user to respond more naturally.

[0215] Prescription of medication

[0216] Server: Additional information collected through the interview is sent to the server and reanalyzed for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes a final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[0217] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prepared medicine is then sent to the address specified by the user via a delivery service.

[0218] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[0219] Observation

[0220] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[0221] Server: The server receives the biometric and emotional information again and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified of the results and given appropriate instructions.

[0222] Specific examples

[0223] For example, suppose a user feels unwell. In this case, the user first activates the camera through the LINE app and measures their heart rate and breathing rate. This data is sent to the server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the analysis results, the chatbot conducts a medical interview with the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies the user of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[0224] The processing flow will be explained below.

[0225] Step 1:

[0226] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate and breathing rate. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[0227] Step 2:

[0228] Device: The smart device's camera captures biometric information such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the device and converted into an appropriate format. This data is then sent to the server.

[0229] Step 3:

[0230] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The video and audio data is sent to the emotion engine, which analyzes them in real time and recognizes the user's emotional state (e.g., joy, sadness, anger, surprise).

[0231] Step 4:

[0232] Server: Receives biometric and emotional information sent from the device and stores it in a database. The server uses a machine learning model to analyze the biometric information and also incorporates emotional information into the analysis. This allows the server to evaluate the user's current health condition and generate a simple diagnosis.

[0233] Step 5:

[0234] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates individual questions for the user and begins the medical interview.

[0235] Step 6:

[0236] On the device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. During this process, the chatbot dynamically adjusts the questions to take into account the user's emotional state.

[0237] Step 7:

[0238] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and past medical information, the doctor makes the final diagnosis and generates a diagnosis result.

[0239] Step 8:

[0240] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and electronically sends it to a partner pharmacy.

[0241] Step 9:

[0242] Pharmacy: Prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[0243] Step 10:

[0244] User: The user receives the medication at home. After taking it, they enter information about their treatment progress and side effects through the LINE app. Vital data may also be collected again.

[0245] Step 11:

[0246] Server: Receives the biometric and emotional information entered by the user again and analyzes it using a machine learning model. The analysis results are sent to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The results are also sent to the user, who is given appropriate instructions.

[0247] This system allows users to receive comprehensive health management and medical services from the comfort of their own home. By combining it with an emotion engine, it is possible to provide more accurate diagnoses and personalized medical services.

[0248] Example 2

[0249] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0250] Conventional health management systems lack the ability to incorporate visual and audio emotional information from users, making it difficult to comprehensively assess a user's accurate health and emotional state. This can lead to poor diagnostic accuracy and a poor user experience. Furthermore, the lack of machine learning models to analyze biometric and emotional information limits the accuracy of data analysis. It is desirable to address these issues and realize comprehensive user health management.

[0251] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0252] In this invention, the server includes a means for acquiring general information about the user, a means for analyzing the general information, and a means for making a diagnosis based on the analysis results and past health information. This enables comprehensive health management for the user. Furthermore, a system is provided that includes a means for acquiring the user's emotions using a camera and microphone of a smart device, a means for analyzing the emotional information, a means for reflecting the analyzed emotional information in the diagnosis results, a means for analyzing the general information and emotional information using a machine learning model, and a means for conducting an interview with the user in an interactive format based on the analysis results. This allows for a more accurate assessment of the user's health status, an appropriate diagnosis and prescription, and more personalized medical services.

[0253] A "user" is an individual who uses the system to manage their own health.

[0254] "Biometric information" refers to various data obtained from the user's body, such as heart rate, respiratory rate, body temperature, and blood pressure.

[0255] "General information" is a blanket term that refers to a variety of data, including biometric and emotional information about the user.

[0256] "Analysis" is the process of evaluating and making judgments on collected data using machine learning models and algorithms.

[0257] "Determination" refers to evaluating the user's health condition based on the analysis results and past health information.

[0258] A "prescription" is an act in which a doctor prescribes necessary medication based on the user's health condition.

[0259] "Emotion information" is data that indicates the psychological state of the user, obtained from facial expressions, voice, etc.

[0260] A "smart device" is an electronic device such as a mobile phone or tablet that includes sensors such as a camera or microphone.

[0261] The "emotion engine" is a system that analyzes captured facial expressions and voice data to recognize the user's emotions.

[0262] A "machine learning model" is an algorithm or computational model that identifies patterns based on large amounts of data and makes predictions and judgments.

[0263] "Dialogue format" refers to a format in which communication with users is interactive, such as through chatbots.

[0264] This invention is a system that supports users' health management, and by using a common messaging app, it is possible to consistently acquire daily biometric information, analyze data, make judgments, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of this system will be described.

[0265] Acquisition of biometric information

[0266] A user launches a messaging app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[0267] The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to a server.

[0268] Acquiring emotional information

[0269] The device uses the smart device's camera and microphone to capture the user's facial expressions and vocal changes and transmit them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[0270] Data analysis

[0271] The server receives the transmitted biometric and emotional information and stores it in a database. The server then analyzes the biometric information using a machine learning model and incorporates the emotional information from the emotion engine into the analysis, thereby enabling a more accurate assessment of the user's health condition.

[0272] judgement

[0273] The server generates a simple diagnostic result based on the analysis results. This simple diagnostic result is provided to the chatbot of the messaging app, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[0274] The chatbot's questions are displayed on the device's chat screen in the messaging app, and the user can respond to them and provide additional health information. The chatbot takes the user's emotional state into account when posing questions, allowing the user to respond more naturally.

[0275] Prescription of medication

[0276] The server analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The doctor makes the final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[0277] The pharmacy prepares the medicine based on the prescription and arranges for delivery, which is then sent to the address specified by the user via a delivery service.

[0278] Users receive and take their medication at home, and can enter progress information, including treatment progress and side effects, through a messaging app.

[0279] Observation

[0280] The user periodically enters the progress of treatment through the messaging app, and it is also possible to collect vital data again and send it to the server.

[0281] The server receives the retransmitted biometric and emotional information and reanalyzes it using a machine learning model. The analysis results are then sent to the doctor to evaluate the effectiveness of the treatment, and necessary measures are taken. The user is also notified of the results and given appropriate instructions.

[0282] Specific examples

[0283] For example, if a user feels unwell, they first activate the camera through a messaging app to measure their heart rate and breathing rate. This data is sent to a server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the results of this analysis, the chatbot conducts a medical interview with the user, and additional information is collected accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is then delivered to the user's home from an affiliated pharmacy. The user then enters the progress of their treatment through the messaging app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notify the user of the results. This enables users to receive comprehensive health management and medical care from the comfort of their own home.

[0284] Prompt sentence for generative AI model

[0285] "Please explain in natural language how a user can manage their health using a messaging app. Please show the flow of the system, including specific actions, from obtaining biometric and emotional information, to making a judgment, prescribing medication, and monitoring progress."

[0286] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0287] Step 1: Launch your messaging app

[0288] User: Launches a messaging app and selects the "Health Management" option. The input is a user action, and the output is the app switching to "Health Management" mode. Specific actions include the user operating the smartphone screen and tapping the health management option within the app.

[0289] Step 2: Measuring biometric data

[0290] Terminal: The smart device's camera activates and displays an interface for measuring heart rate, respiratory rate, and body temperature. The input is the user's face or hand, and the output is biometric data. Specific operations include the device's camera using facial recognition technology to capture the user's face or hand and measure biometric information.

[0291] Step 3: Storing and sending data

[0292] Terminal: The acquired biometric data is temporarily stored on the terminal, converted into an appropriate format, and sent to the server. The input is the measured biometric data, and the output is the format-converted data sent to the server. Specific operations include the terminal converting the measured data into JSON format, etc., and sending the data to the server via the HTTPS protocol.

[0293] Step 4: Capturing emotional information

[0294] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The input is the user's facial expressions and voice, and the output is the captured emotion data. Specifically, the device's camera analyzes the user's facial expressions using a facial expression recognition algorithm, and the microphone records voice data.

[0295] Step 5: Sending emotional information

[0296] Device: Sends captured data to the emotion engine. The input is the captured emotion data, and the output is the data sent to the emotion engine. Specific operations include the device sending facial expression and voice data to the emotion engine's analysis API.

[0297] Step 6: Receiving and storing data

[0298] Server: Receives the transmitted biometric and emotional information and stores it in a database. The input is the data transmitted from the device, and the output is the data stored in the database. Specifically, the server receives the data at the API endpoint where it receives data and stores it in an SQL database.

[0299] Step 7: Analysis using machine learning models

[0300] Server: Analyzes biometric and emotional information using a machine learning model. The input is the stored biometric and emotional information, and the output is the analysis results. Specific operations include the server analyzing the data using Python's Scikit-learn or TensorFlow to evaluate the user's health status.

[0301] Step 8: Generate quick diagnostic results

[0302] Server: Generates a simple diagnostic result based on the analysis results. The input is the analysis result from the machine learning model, and the output is a simple diagnostic result. Specifically, the server uses a specialized rule-based algorithm to output the diagnostic result from the analysis results.

[0303] Step 9: Chatbot consultation

[0304] Server: Provides the generated simple diagnosis results to the chatbot of the messaging app, which then generates additional questions for the user. The input is the simple diagnosis results, and the output is the additional questions sent to the user. Specifically, the server sends data to the chatbot using the messaging API, and the chatbot then sends dynamically generated questions to the user.

[0305] Terminal: Questions from the chatbot are displayed on the chat screen of the messaging app. The input is the question from the chatbot, and the output is the user's answer. Specific actions include the user opening the chat screen on their smartphone and answering the displayed question by typing.

[0306] Step 10: Generate a final diagnosis and prescription

[0307] Server: Analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The input is the medical interview results and past health information, and the output is the final diagnosis and prescription data. Specific operations include a remote doctor on the server using the electronic diagnostic system to confirm the final diagnosis and create an electronic prescription.

[0308] Step 11: Submit your prescription

[0309] Server: Electronically transmits prescriptions to affiliated pharmacies. The input is prescription data, and the output is the prescription sent to the pharmacy. Specifically, the server transmits data to the pharmacy's electronic prescription system.

[0310] Step 12: Prepare and deliver medication

[0311] Pharmacy: Prepares medicine based on prescription and arranges for delivery. The input is the received prescription data, and the output is the medicine arranged for delivery. Specifically, the pharmacy prepares the medicine, provides the information to the delivery service, and delivers it to the user.

[0312] User: Receives and takes medicine. The input is the delivered medicine, and the output is progress information after taking it. The specific operation is for the user to receive the delivered medicine and use it according to the dosage instructions.

[0313] Step 13: Enter progress information

[0314] User: Enters treatment progress and side effects into a messaging app. The input is treatment progress information, and the output is the progress information sent to the server. Specifically, the user enters progress information using a form within the app and presses the send button.

[0315] Step 14: Reanalyze the data

[0316] Server: Analyzes the retransmitted biometric and emotional information. The input is the newly transmitted biometric and emotional information, and the output is the reanalysis results. Specifically, the server receives the new data and analyzes it again using the machine learning model.

[0317] Step 15: Notification of results

[0318] Server: Notifies the doctor of the analysis results and takes necessary measures. The input is the reanalysis results, and the output is notification data for the doctor and a result notification for the user. Specifically, the server sends the analysis results to the doctor via email or a notification system, and notifies the user of the results via a messaging app.

[0319] (Application example 2)

[0320] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] Conventional health management systems analyze biometric information acquired through the camera and microphone of a user's smart device and can provide a diagnosis and prescription based on the results. However, these systems do not support the use of biometric measurement results to recommend health-related products, deliver those products, or monitor their effectiveness. As a result, they do not adequately provide a means for users to use appropriate health-related products based on their own health status and continuously improve their health. Furthermore, it is difficult to track the effectiveness of products and make optimal recommendations for individual users.

[0322] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0323] In this invention, the server includes a means for acquiring biometric information of a user, a means for analyzing the biometric information, a means for making a diagnosis based on the analysis results and past medical information, a means for prescribing a prescription based on the diagnosis results, a means for presenting the prescribed health-related product, a means for delivering the health-related product, and a means for monitoring the effects. This makes it possible to analyze the user's biometric information and emotional information, and based on this, to propose and deliver the most appropriate health-related product, and further monitor the effects over time.

[0324] "User" refers to an individual who uses this system.

[0325] "Biometric information" refers to data about a user's basic bodily functions, such as temperature, heart rate, and breathing rate.

[0326] "Analysis" refers to data processing to evaluate and diagnose health status based on acquired biometric information.

[0327] "Diagnosis" refers to assessing the user's health condition and determining necessary treatment and measures based on analysis results and past medical information.

[0328] "Prescription" refers to the act of providing needed medicines or health-related products based on diagnostic results.

[0329] "Health-related products" refers to supplements, exercise equipment, health foods, etc. used to maintain or improve the user's health.

[0330] "Delivery" refers to the delivery of health-related products selected or prescribed by the user to the address specified by the user.

[0331] "Follow-up" refers to the ongoing monitoring and evaluation of a user's health status after using a prescribed product.

[0332] This system acquires a user's biometric and emotional information, and based on that information, diagnoses their health, recommends and delivers health-related products, and monitors their effectiveness. This system is implemented using smart devices, a server, a machine learning model, and an emotion analysis engine.

[0333] System Overview

[0334] The system can be broadly divided into three parts: a part that acquires the user's biometric information, a part that analyzes the biometric and emotional information, a part that suggests and delivers health-related products based on the analysis results, and finally a part that monitors the effects over time.

[0335] Acquiring user's biometric information

[0336] The user launches the smartphone application and uses the camera and microphone to measure their heart rate, respiratory rate, body temperature, facial expressions, and voice. Specifically, image processing software such as OpenCV is used to extract biometric information from the camera footage, and emotional information is analyzed from the voice and facial expression data using software such as EmotionAPI.

[0337] Data analysis

[0338] The acquired biometric and emotional information is sent from the device to a server, which then analyzes the data using machine learning models such as TensorFlow. The analysis results are then compared with past medical information to assess the user's health status.

[0339] Proposal and delivery of health-related products

[0340] Based on the diagnosis results, appropriate health-related products are recommended to the user. These products can be purchased directly through the application and delivered to the specified address. Product recommendations are made in conjunction with a health database that is collated with the analysis results.

[0341] Follow-up of effects

[0342] Even after a user has purchased or taken a prescribed product, biometric and emotional information is periodically collected again to monitor progress. The collected data is then sent to the server for further analysis. Based on the results, new recommendations or adjustments are made.

[0343] Specific examples

[0344] For example, a user opens the app every morning and checks their health using the camera and microphone. Based on the results, the app diagnoses a vitamin D deficiency and recommends vitamin supplements. The user purchases the supplements and checks again a week later. The app also collects this data and uses it to make future recommendations.

[0345] Prompt Sentence Examples

[0346] To assess the user's health, perform the following steps:

[0347] 1. Turn on the camera to measure your heart rate, breathing rate, and temperature.

[0348] 2. Next, the microphone captures the voice and recognizes facial expressions.

[0349] 3. Based on the analysis results, we recommend appropriate health-related products.

[0350] For example, if you have a vitamin D deficiency, you might see a push notification like this:

[0351] "I suggest a vitamin D supplement. And how about some exercise equipment along with that?"

[0352] In this way, the present invention can provide comprehensive support for the user's health management.

[0353] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0354] Step 1: User biometric information acquisition

[0355] The user launches a smartphone application, which uses the camera and microphone to measure their heart rate, respiration rate, body temperature, facial expressions, and voice. Specifically, the smartphone camera takes a picture of their face, and the heart rate and respiration rate are analyzed using image processing software such as OpenCV. The microphone also captures their voice, and emotional information is analyzed using EmotionAPI or similar. The input at this stage is the user's face image and voice data, and the output is the analyzed heart rate, respiration rate, body temperature, and emotional information.

[0356] Step 2: Sending data to the server

[0357] The device sends the acquired and analyzed biometric and emotional information to the server. Specifically, the data is formatted in JSON format or similar and sent to the server using a secure protocol (e.g., HTTPS). The input is the analyzed biometric and emotional information, and the output is the data sent to the server.

[0358] Step 3: Data analysis by the server

[0359] The server analyzes the received biometric and emotional information using machine learning models such as TensorFlow. Past medical information is also collated during the analysis. Specifically, past medical information is retrieved from a database, and the user's health status is evaluated as the analysis result. The input is the transmitted biometric, emotional, and past medical information, and the output is the diagnosis result.

[0360] Step 4: Proposing health-related products

[0361] The server proposes appropriate health-related products based on the analysis and diagnosis results. It compares the analyzed data with a database of health-related products to select the most suitable products for the user. The proposals are notified to the user through the application. The input is the diagnosis results and a database of health-related products, and the output is a list of proposed products.

[0362] Step 5: Purchase and shipping

[0363] The user selects the desired product from the list of suggested products and completes the purchase process through the application. The server receives the purchase information and initiates the delivery process. The input is the user's purchase selection and delivery address information, and the output is product delivery instructions. The product delivery instructions are sent to the delivery company.

[0364] Step 6: Monitor the results

[0365] The user uses the purchased health-related product and periodically re-acquires biometric and emotional information. The device again transmits the newly acquired data to the server. The server again analyzes the data using a machine learning model and evaluates the effectiveness. If necessary, it provides the user with appropriate instructions or the next suggestion. The input is the newly acquired biometric and emotional information, and the output is the evaluation of the effectiveness and new suggestions.

[0366] Through the above steps, a system that comprehensively supports the user's health management is realized.

[0367] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0368] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0369] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0370] [Second embodiment]

[0371] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0372] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0373] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0374] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0375] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0376] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0377] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0378] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0379] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0380] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0381] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0382] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0383] System Overview

[0384] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biological information, analyze data, make diagnoses, prescribe medicines, and monitor progress. Specific embodiments of each part of the system are described below.

[0385] Acquisition of biometric information

[0386] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[0387] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into the required format, and then sent to the server.

[0388] Data analysis

[0389] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model. The server evaluates the user's current health condition based on past accumulated data and the trained model. The analysis results are generated as a simple diagnosis.

[0390] diagnosis

[0391] Server: The simple diagnosis results are provided to the chatbot, which then collects additional information through dialogue with the user and conducts a medical interview. The data obtained through this interactive interview is then sent back to the server and used for the final diagnosis.

[0392] Terminal: The chatbot asks the user questions and collects their answers to gain more detailed information, which improves the accuracy of the diagnosis.

[0393] Prescription of medication

[0394] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[0395] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[0396] Users: Users can receive their medication at home, allowing them to receive the medical care they need without leaving their homes.

[0397] Observation

[0398] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[0399] Server: The server receives the biometric data again and analyzes it with a machine learning model to evaluate the effectiveness of treatment. The analysis results are notified to the user and doctor, and necessary measures are taken.

[0400] Specific examples

[0401] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment in the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies them of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[0402] The processing flow will be explained below.

[0403] Step 1:

[0404] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera. The user follows the on-screen instructions to hold their face or hand in front of the camera to collect biometric information such as heart rate and respiratory rate.

[0405] Step 2:

[0406] Terminal: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the terminal and converted into an appropriate format. The data is then sent to the server.

[0407] Step 3:

[0408] Server: The server receives the biometric information sent from the device, stores the data in a database, analyzes the biometric information using a machine learning model, and generates a simple diagnosis result as the analysis result.

[0409] Step 4:

[0410] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates questions for the user and begins the medical interview.

[0411] Step 5:

[0412] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. These answers are then sent back to the server.

[0413] Step 6:

[0414] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes the final diagnosis and generates the results.

[0415] Step 7:

[0416] Server: Based on the final diagnosis, the doctor prescribes the necessary medication. The prescription is then electronically sent to the partner pharmacy.

[0417] Step 8:

[0418] Pharmacy: The pharmacy prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[0419] Step 9:

[0420] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[0421] Step 10:

[0422] Server: Receives progress information and additional biometric data entered by the user and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified and given appropriate instructions.

[0423] In this way, the system provides consistent daily health management and medical treatment through collaboration between users, terminals, and servers.

[0424] Example 1

[0425] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0426] Conventional health management systems have had issues with the user having to spend a lot of time and effort to obtain biometric information, and the diagnosis and treatment process is fragmented, making it difficult to provide comprehensive medical care.In addition, it is difficult to perform real-time diagnosis or interactive interviews, making it difficult to quickly and accurately grasp the user's health condition.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0428] In this invention, the server includes means for acquiring biometric information of a user, means for analyzing the biometric information, means for making a diagnosis based on the analysis result and past medical information, means for prescribing medicine based on the diagnosis result, means for delivering the prescribed medicine, means for monitoring the progress of treatment, means for activating a camera of a smart device to measure biometric information based on a health management option selected by the user, means for converting the biometric information into a predetermined format and transmitting it to the server, means for analyzing the biometric information using a machine learning model, and means for conducting an interview in an interactive format with the user using a chatbot, thereby enabling the user to receive consistent health management and medical services.

[0429] "User" refers to an individual who uses the system for a particular purpose.

[0430] "Biometric information" refers to various health-related data obtained from the human body (e.g., heart rate, respiratory rate, body temperature, etc.).

[0431] "Means" refers to a specific device, method, or part of a system used to achieve a particular purpose.

[0432] "Smart device" refers to a portable electronic device with advanced computing capabilities (e.g., smartphone, tablet, etc.).

[0433] "Camera" refers to a digital imaging device for capturing images.

[0434] "Analysis" refers to the process of examining the data obtained in detail and drawing conclusions or information.

[0435] "Server" refers to a high-performance computer system that manages, processes, and provides data over a network.

[0436] A "machine learning model" refers to a collection of algorithms and statistical models that learn from data and make predictions and decisions.

[0437] A "chatbot" refers to a program or artificial intelligence that mimics a conversation with a user.

[0438] "Format" refers to the particular structure or form of data.

[0439] "Medical interview" refers to the process by which a doctor gathers information from a patient about their symptoms and medical history.

[0440] "Diagnosis" refers to the process by which a medical professional identifies an illness or condition based on a patient's symptoms and test results.

[0441] "Prescription" refers to a doctor prescribing an appropriate medication to a patient.

[0442] "Pharmacy" means an establishment that prepares and dispenses medicines based on prescriptions.

[0443] "Delivery" refers to the process of getting an item to a designated location.

[0444] "Treatment" refers to medical procedures aimed at improving or recovering from an illness or disability.

[0445] "Follow-up" refers to the process of continually checking the progress of treatment and making any necessary adjustments.

[0446] The present invention is a system for supporting a user's health management, and is implemented using specific procedures, hardware, and software. This system acquires daily biological information and performs data analysis, diagnosis, drug prescription, and follow-up. Specific embodiments of each part are described below.

[0447] Acquisition of biometric information

[0448] User: The user launches the LINE app on their smart device and selects the "Health Management" option. The app activates the camera and displays an interface for measuring heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure their biometric information.

[0449] Device: The camera on the smart device captures and analyzes the user's video data, collecting biometric information such as heart rate, respiratory rate, and body temperature. This data is temporarily stored on the device, converted into a specific format, and then sent to a server.

[0450] Data analysis

[0451] Server: The server receives the transmitted biometric information and analyzes it using a machine learning model. The model used for the analysis is based on past accumulated data and training data. From the analysis results, the server evaluates the user's current health condition and generates a simple diagnosis.

[0452] diagnosis

[0453] Server: The simple diagnosis results are provided to the chatbot, which then asks the user additional questions. The chatbot then interacts with the user to gather more information and sends that data back to the server. The final diagnosis is based on this data.

[0454] Terminal: The terminal sends additional questions to the user through the chatbot and collects the user's answers, thereby improving the accuracy of the diagnosis.

[0455] Prescription of medication

[0456] Server: Based on the final diagnosis, the server generates instructions for the doctor to prescribe the necessary medication. The prescription is sent to a partner pharmacy, which prepares the medication and arranges for delivery.

[0457] User: The user receives the medication at home and takes it as directed.

[0458] Observation

[0459] User: The user can periodically enter the progress of treatment in the LINE app, and can obtain vital data again and send it to the server.

[0460] Server: The server receives the new biometric data and re-analyzes it using machine learning models. The results are then communicated to the user and doctor, and further action is taken if necessary.

[0461] (Example)

[0462] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data and evaluates the effectiveness of the treatment. This allows users to receive comprehensive health management and medical services from the comfort of their own home.

[0463] (Example of a prompt)

[0464] "Measure your heart rate and breathing rate."

[0465] "Please select the health management option and enter your data."

[0466] "I'll do a quick checkup. Please assess your health."

[0467] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0468] Step 1: Obtaining biometric information

[0469] User: The user launches the LINE app and enters health management mode by selecting the "Health Management" option. User action (selecting an option) is required as input.

[0470] Terminal: The terminal activates the smart device's camera and displays the interface. When the user holds their face or hand over the camera, the camera captures video data. Video data is provided as input data, and biometric information such as heart rate, respiratory rate, and body temperature is generated as output.

[0471] How it works: The camera module captures video data, which is then analyzed by internal software to measure biometric information, which is then temporarily stored in the device's memory.

[0472] Step 2: Sending biometric information

[0473] Terminal: The terminal converts the collected biometric data into a predetermined format. This conversion process includes noise removal and data normalization. As input, it requires raw biometric data, and as output, it obtains formatted data.

[0474] The formatted data is then sent to the server using the terminal's network module.

[0475] What it does: Specific algorithms and techniques are used to transform the data into a different format. The optimized data is then sent to the server using a secure protocol.

[0476] Step 3: Analyze the data

[0477] Server: The server receives the transmitted biometric data. It requires formatted biometric data transmitted from the device as input data. This data is then fed into a machine learning model and analyzed to assess the health status.

[0478] How it works: The machine learning model analyzes the data and detects abnormalities, such as heart rate fluctuation patterns or breathing rate. The analysis results are temporarily stored in a database on the server. The analysis results and a simple diagnosis are generated as output.

[0479] Step 4: Simple diagnosis and interview

[0480] Server: The server generates a simple diagnosis based on the analysis results. The input data is the analysis results of the machine learning model. This result is provided to the chatbot, which then begins interviewing the user.

[0481] Terminal: The terminal sends follow-up questions to the user through the chatbot and collects answers from the user. The input data includes the user's answers, and the output provides additional health information.

[0482] Specific actions: For example, specific prompts such as "Please tell us about your recent fatigue and appetite changes" are displayed to the user. These answers are sent to the server in real time.

[0483] Step 5: Sending diagnostic results and prescribing medication

[0484] Server: The server combines the medical interview data and biometric information to make the final diagnosis. The analysis results and medical interview data are required as input data. Based on this, the doctor prescribes the necessary medication and generates a prescription.

[0485] Specific operation: Prescription data is automatically generated and electronically transmitted to affiliated pharmacies. As an output, information on the completed prescription is generated.

[0486] Step 6: Follow-up and evaluation

[0487] User: The user periodically updates the progress of their treatment on the LINE app, and their vital data is acquired again and sent to the server. The input data includes new vital information and the progress of treatment.

[0488] Server: The server reanalyzes the updated information and uses machine learning models to evaluate the effectiveness of the treatment. The input data is the updated biometric information, and the output is the evaluation result of the treatment effectiveness.

[0489] Specific operation: Based on the analysis results, the server provides feedback to the user, such as "Treatment is progressing smoothly" or "Additional tests are required." The results are then notified to the user and their doctor.

[0490] (Application example 1)

[0491] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0492] The working environment for employees at logistics centers is harsh, making health management important. However, it is difficult for employees to properly monitor their own health status and detect abnormalities early amid their busy work schedules. Furthermore, there are limited means to respond quickly when abnormalities are discovered. Given this background, there is a need for an efficient and effective method of managing the health of employees working at logistics centers.

[0493] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0494] In this invention, the server includes means for acquiring user biometric information, means for analyzing the biometric information, means for monitoring the biometric information in real time if the user is an employee working at a logistics center, means for making a diagnosis based on the analysis results and past medical information, means for notifying a manager when abnormal biometric information is detected, means for prescribing medication based on the diagnosis results, means for delivering the prescribed medication, and means for monitoring the progress of treatment. This allows for efficient and effective health management of employees at the logistics center and enables rapid response to abnormalities.

[0495] "Users" refer to employees working at the logistics center.

[0496] "Biometric information" is data that indicates the user's health condition, such as heart rate, respiratory rate, and body temperature.

[0497] A "smart device" is an information device such as a smartphone or tablet that is carried by a user.

[0498] A "camera" is an image capture device built into a smart device.

[0499] "Analysis" refers to the process of analyzing collected biometric information using methods such as machine learning models to extract useful information.

[0500] "Diagnosis" refers to the act of evaluating the user's health condition based on the analysis results of biological information and past medical information.

[0501] "Prescription" is the act of a doctor prescribing the necessary medication based on diagnostic results.

[0502] "Delivery" refers to the act of delivering prescribed medicine to a location designated by the user.

[0503] "Follow-up" is the act of monitoring the progress of treatment and continually assessing health status.

[0504] "Administrator" refers to the person in charge of managing the health of employees at the logistics center.

[0505] The present invention is a system for supporting the health management of employees working at a logistics center. This system works in conjunction with smart devices to consistently acquire biometric information, analyze data, diagnose, notify abnormalities, prescribe medication, and monitor treatment progress. Specific embodiments of each part of the system are described below.

[0506] Acquisition of biometric information

[0507] User: The user launches the application on their smart device and holds their face or hand over the camera to measure their biometric information, including heart rate, respiratory rate, and body temperature.

[0508] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[0509] Data analysis

[0510] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model (for example, a model built with Keras). The server evaluates the user's current health status based on past accumulated data and the trained model. If an abnormality is detected, the analysis results will notify the administrator.

[0511] Diagnostics and Notifications

[0512] Server: Based on the analysis results and past medical information, the server diagnoses the user's health condition. If abnormal vital signs are detected, a notification is sent to the administrator, prompting prompt action.

[0513] Managers: Managers can monitor the health status of employees within the distribution center and take appropriate action if necessary.

[0514] Medication and follow-up

[0515] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[0516] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[0517] Users: Users can receive medication at home, allowing them to receive the medical care they need without leaving their homes. Furthermore, they can enter their treatment progress through an application on their smart device.

[0518] Server: The server receives the retransmitted vital signs and analyzes them with a machine learning model to evaluate the effectiveness of treatment. The analysis results are then notified to the user and doctor, and necessary measures are taken.

[0519] Specific examples

[0520] For example, if an employee working at a logistics center feels unwell, they first launch an application on their smart device and hold their face up to the camera to measure their heart rate and respiratory rate. The data is then sent to a server where it is analyzed using a machine learning model. If an abnormality is detected based on the analysis results, an alert is sent to an administrator. The administrator can then take prompt action, requesting a diagnosis from a doctor if necessary and prescribing medication. The user then receives the medication at home and records the progress of their treatment in the application, and the server analyzes the data to continue monitoring their condition.

[0521] Prompt Sentence Examples

[0522] For example, the following prompts could be fed into a generative AI model:

[0523] "We want to build a system in our logistics center that allows employees to monitor their daily health using their smartphones. Using smart devices, we can acquire vital signs (heart rate, respiratory rate, body temperature) and analyze them using machine learning models. If any abnormal values ​​are detected, a notification will be sent to the manager."

[0524] In this way, a system can be realized that comprehensively manages the health of employees working at logistics centers.

[0525] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0526] Step 1:

[0527] User: Launches the smart device application and holds their face or hand in front of the camera. The input is video data captured by the smart device's camera. The output is biometric information such as heart rate, respiratory rate, and body temperature.

[0528] Step 2:

[0529] Terminal: Analyzes video data captured by the smart device's camera to collect biometric information. Video data provided by the user is used as input. Analyzed biometric information is obtained as output. Specifically, the smart device's image processing function is used to estimate heart rate and respiratory rate. The data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[0530] Step 3:

[0531] Server: Receives the transmitted biometric information and analyzes it using a machine learning model (built using Keras). The biometric information sent from the device is used as input. The output is an assessment of the user's current health status. Specifically, the server compares the biometric information with an existing database to check for any abnormalities.

[0532] Step 4:

[0533] Server: Diagnoses the user's health condition based on the analysis results and past medical information. Analysis results and past medical information are used as input. Diagnosis results are obtained as output. Specifically, if an abnormality is detected based on the analysis results, the next step is notification processing.

[0534] Step 5:

[0535] Server: If abnormal biometric information is detected, it sends a notification to the administrator. The diagnosis results are used as input. The notification data to be sent to the administrator is obtained as output. Specifically, when an abnormality is detected, an alert is sent to the administrator's smart device using the LINE API.

[0536] Step 6:

[0537] Administrator: Receives notifications, monitors employee health status in real time, and takes appropriate action. The input is the notification sent from the server. The output is the response procedure to be taken by the administrator. Specifically, the administrator who receives the notification will request a diagnosis from a doctor if necessary, and ensure the safety of the employee.

[0538] Step 7:

[0539] Server: Based on the diagnosis results, the doctor prescribes the necessary medication. The diagnosis results are used as input. A prescription is generated as output. Specifically, the server creates an electronic prescription based on instructions from the doctor and sends it to a partner pharmacy.

[0540] Step 8:

[0541] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescription is used as input. The output is the medicine to be sent to the user's delivery address. The specific operation is that the pharmacy prepares the prescription and arranges for delivery to the specified address.

[0542] Step 9:

[0543] User: Receives medication at home and performs treatment. The input is the delivered medication. The output is data on the progress of treatment. The specific operation is that the user receives the medication and records the progress of treatment through the application.

[0544] Step 10:

[0545] Server: Receives the progress of treatment and analyzes the vital signs again. The input is the treatment progress data sent by the user. The output is the evaluation result of the treatment effect. Specifically, the server analyzes the new vital signs and checks whether the treatment is effective. The results are notified to the user and the doctor.

[0546] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0547] System Overview

[0548] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biometric information, analyze data, make diagnoses, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of the system are described below.

[0549] Acquisition of biometric information

[0550] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[0551] Device: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to the server.

[0552] Acquiring emotional information

[0553] Device: The smart device's camera and microphone are used to capture the user's facial expressions and vocal changes and send them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[0554] Data analysis

[0555] Server: The transmitted biometric and emotional information is received by the server and stored in a database. The server analyzes the biometric information using a machine learning model and also incorporates emotional information from the emotion engine into the analysis. This allows for a more accurate assessment of the user's health condition.

[0556] diagnosis

[0557] Server: The analysis results are generated as a simple diagnosis. This simple diagnosis result is provided to the LINE chatbot, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[0558] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. The chatbot asks questions taking into account the user's emotional state, allowing the user to respond more naturally.

[0559] Prescription of medication

[0560] Server: Additional information collected through the interview is sent to the server and reanalyzed for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes a final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[0561] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prepared medicine is then sent to the address specified by the user via a delivery service.

[0562] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[0563] Observation

[0564] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[0565] Server: The server receives the biometric and emotional information again and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified of the results and given appropriate instructions.

[0566] Specific examples

[0567] For example, suppose a user feels unwell. In this case, the user first activates the camera through the LINE app and measures their heart rate and breathing rate. This data is sent to the server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the analysis results, the chatbot conducts a medical interview with the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies the user of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[0568] The processing flow will be explained below.

[0569] Step 1:

[0570] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate and breathing rate. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[0571] Step 2:

[0572] Device: The smart device's camera captures biometric information such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the device and converted into an appropriate format. This data is then sent to the server.

[0573] Step 3:

[0574] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The video and audio data is sent to the emotion engine, which analyzes them in real time and recognizes the user's emotional state (e.g., joy, sadness, anger, surprise).

[0575] Step 4:

[0576] Server: Receives biometric and emotional information sent from the device and stores it in a database. The server uses a machine learning model to analyze the biometric information and also incorporates emotional information into the analysis. This allows the server to evaluate the user's current health condition and generate a simple diagnosis.

[0577] Step 5:

[0578] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates individual questions for the user and begins the medical interview.

[0579] Step 6:

[0580] On the device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. During this process, the chatbot dynamically adjusts the questions to take into account the user's emotional state.

[0581] Step 7:

[0582] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and past medical information, the doctor makes the final diagnosis and generates a diagnosis result.

[0583] Step 8:

[0584] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and electronically sends it to a partner pharmacy.

[0585] Step 9:

[0586] Pharmacy: Prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[0587] Step 10:

[0588] User: The user receives the medication at home. After taking it, they enter information about their treatment progress and side effects through the LINE app. Vital data may also be collected again.

[0589] Step 11:

[0590] Server: Receives the biometric and emotional information entered by the user again and analyzes it using a machine learning model. The analysis results are sent to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The results are also sent to the user, who is given appropriate instructions.

[0591] This system allows users to receive comprehensive health management and medical services from the comfort of their own home. By combining it with an emotion engine, it is possible to provide more accurate diagnoses and personalized medical services.

[0592] Example 2

[0593] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0594] Conventional health management systems lack the ability to incorporate visual and audio emotional information from users, making it difficult to comprehensively assess a user's accurate health and emotional state. This can lead to poor diagnostic accuracy and a poor user experience. Furthermore, the lack of machine learning models to analyze biometric and emotional information limits the accuracy of data analysis. It is desirable to address these issues and realize comprehensive user health management.

[0595] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0596] In this invention, the server includes a means for acquiring general information about the user, a means for analyzing the general information, and a means for making a diagnosis based on the analysis results and past health information. This enables comprehensive health management for the user. Furthermore, a system is provided that includes a means for acquiring the user's emotions using a camera and microphone of a smart device, a means for analyzing the emotional information, a means for reflecting the analyzed emotional information in the diagnosis results, a means for analyzing the general information and emotional information using a machine learning model, and a means for conducting an interview with the user in an interactive format based on the analysis results. This allows for a more accurate assessment of the user's health status, an appropriate diagnosis and prescription, and more personalized medical services.

[0597] A "user" is an individual who uses the system to manage their own health.

[0598] "Biometric information" refers to various data obtained from the user's body, such as heart rate, respiratory rate, body temperature, and blood pressure.

[0599] "General information" is a blanket term that refers to a variety of data, including biometric and emotional information about the user.

[0600] "Analysis" is the process of evaluating and making judgments on collected data using machine learning models and algorithms.

[0601] "Determination" refers to evaluating the user's health condition based on the analysis results and past health information.

[0602] A "prescription" is an act in which a doctor prescribes necessary medication based on the user's health condition.

[0603] "Emotion information" is data that indicates the psychological state of the user, obtained from facial expressions, voice, etc.

[0604] A "smart device" is an electronic device such as a mobile phone or tablet that includes sensors such as a camera or microphone.

[0605] The "emotion engine" is a system that analyzes captured facial expressions and voice data to recognize the user's emotions.

[0606] A "machine learning model" is an algorithm or computational model that identifies patterns based on large amounts of data and makes predictions and judgments.

[0607] "Dialogue format" refers to a format in which communication with users is interactive, such as through chatbots.

[0608] This invention is a system that supports users' health management, and by using a common messaging app, it is possible to consistently acquire daily biometric information, analyze data, make judgments, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of this system will be described.

[0609] Acquisition of biometric information

[0610] A user launches a messaging app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[0611] The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to a server.

[0612] Acquiring emotional information

[0613] The device uses the smart device's camera and microphone to capture the user's facial expressions and vocal changes and transmit them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[0614] Data analysis

[0615] The server receives the transmitted biometric and emotional information and stores it in a database. The server then analyzes the biometric information using a machine learning model and incorporates the emotional information from the emotion engine into the analysis, thereby enabling a more accurate assessment of the user's health condition.

[0616] judgement

[0617] The server generates a simple diagnostic result based on the analysis results. This simple diagnostic result is provided to the chatbot of the messaging app, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[0618] The chatbot's questions are displayed on the device's chat screen in the messaging app, and the user can respond to them and provide additional health information. The chatbot takes the user's emotional state into account when posing questions, allowing the user to respond more naturally.

[0619] Prescription of medication

[0620] The server analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The doctor makes the final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[0621] The pharmacy prepares the medicine based on the prescription and arranges for delivery, which is then sent to the address specified by the user via a delivery service.

[0622] Users receive and take their medication at home, and can enter progress information, including treatment progress and side effects, through a messaging app.

[0623] Observation

[0624] The user periodically enters the progress of treatment through the messaging app, and it is also possible to collect vital data again and send it to the server.

[0625] The server receives the retransmitted biometric and emotional information and reanalyzes it using a machine learning model. The analysis results are then sent to the doctor to evaluate the effectiveness of the treatment, and necessary measures are taken. The user is also notified of the results and given appropriate instructions.

[0626] Specific examples

[0627] For example, if a user feels unwell, they first activate the camera through a messaging app to measure their heart rate and breathing rate. This data is sent to a server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the results of this analysis, the chatbot conducts a medical interview with the user, and additional information is collected accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is then delivered to the user's home from an affiliated pharmacy. The user then enters the progress of their treatment through the messaging app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notify the user of the results. This enables users to receive comprehensive health management and medical care from the comfort of their own home.

[0628] Prompt sentence for generative AI model

[0629] "Please explain in natural language how a user can manage their health using a messaging app. Please show the flow of the system, including specific actions, from obtaining biometric and emotional information, to making a judgment, prescribing medication, and monitoring progress."

[0630] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0631] Step 1: Launch your messaging app

[0632] User: Launches a messaging app and selects the "Health Management" option. The input is a user action, and the output is the app switching to "Health Management" mode. Specific actions include the user operating the smartphone screen and tapping the health management option within the app.

[0633] Step 2: Measuring biometric data

[0634] Terminal: The smart device's camera activates and displays an interface for measuring heart rate, respiratory rate, and body temperature. The input is the user's face or hand, and the output is biometric data. Specific operations include the device's camera using facial recognition technology to capture the user's face or hand and measure biometric information.

[0635] Step 3: Storing and sending data

[0636] Terminal: The acquired biometric data is temporarily stored on the terminal, converted into an appropriate format, and sent to the server. The input is the measured biometric data, and the output is the format-converted data sent to the server. Specific operations include the terminal converting the measured data into JSON format, etc., and sending the data to the server via the HTTPS protocol.

[0637] Step 4: Capturing emotional information

[0638] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The input is the user's facial expressions and voice, and the output is the captured emotion data. Specifically, the device's camera analyzes the user's facial expressions using a facial expression recognition algorithm, and the microphone records voice data.

[0639] Step 5: Sending emotional information

[0640] Device: Sends captured data to the emotion engine. The input is the captured emotion data, and the output is the data sent to the emotion engine. Specific operations include the device sending facial expression and voice data to the emotion engine's analysis API.

[0641] Step 6: Receiving and storing data

[0642] Server: Receives the transmitted biometric and emotional information and stores it in a database. The input is the data transmitted from the device, and the output is the data stored in the database. Specifically, the server receives the data at the API endpoint where it receives data and stores it in an SQL database.

[0643] Step 7: Analysis using machine learning models

[0644] Server: Analyzes biometric and emotional information using a machine learning model. The input is the stored biometric and emotional information, and the output is the analysis results. Specific operations include the server analyzing the data using Python's Scikit-learn or TensorFlow to evaluate the user's health status.

[0645] Step 8: Generate quick diagnostic results

[0646] Server: Generates a simple diagnostic result based on the analysis results. The input is the analysis result from the machine learning model, and the output is a simple diagnostic result. Specifically, the server uses a specialized rule-based algorithm to output the diagnostic result from the analysis results.

[0647] Step 9: Chatbot consultation

[0648] Server: Provides the generated simple diagnosis results to the chatbot of the messaging app, which then generates additional questions for the user. The input is the simple diagnosis results, and the output is the additional questions sent to the user. Specifically, the server sends data to the chatbot using the messaging API, and the chatbot then sends dynamically generated questions to the user.

[0649] Terminal: Questions from the chatbot are displayed on the chat screen of the messaging app. The input is the question from the chatbot, and the output is the user's answer. Specific actions include the user opening the chat screen on their smartphone and answering the displayed question by typing.

[0650] Step 10: Generate a final diagnosis and prescription

[0651] Server: Analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The input is the medical interview results and past health information, and the output is the final diagnosis and prescription data. Specific operations include a remote doctor on the server using the electronic diagnostic system to confirm the final diagnosis and create an electronic prescription.

[0652] Step 11: Submit your prescription

[0653] Server: Electronically transmits prescriptions to affiliated pharmacies. The input is prescription data, and the output is the prescription sent to the pharmacy. Specifically, the server transmits data to the pharmacy's electronic prescription system.

[0654] Step 12: Prepare and deliver medication

[0655] Pharmacy: Prepares medicine based on prescription and arranges for delivery. The input is the received prescription data, and the output is the medicine arranged for delivery. Specifically, the pharmacy prepares the medicine, provides the information to the delivery service, and delivers it to the user.

[0656] User: Receives and takes medicine. The input is the delivered medicine, and the output is progress information after taking it. The specific operation is for the user to receive the delivered medicine and use it according to the dosage instructions.

[0657] Step 13: Enter progress information

[0658] User: Enters treatment progress and side effects into a messaging app. The input is treatment progress information, and the output is the progress information sent to the server. Specifically, the user enters progress information using a form within the app and presses the send button.

[0659] Step 14: Reanalyze the data

[0660] Server: Analyzes the retransmitted biometric and emotional information. The input is the newly transmitted biometric and emotional information, and the output is the reanalysis results. Specifically, the server receives the new data and analyzes it again using the machine learning model.

[0661] Step 15: Notification of results

[0662] Server: Notifies the doctor of the analysis results and takes necessary measures. The input is the reanalysis results, and the output is notification data for the doctor and a result notification for the user. Specifically, the server sends the analysis results to the doctor via email or a notification system, and notifies the user of the results via a messaging app.

[0663] (Application example 2)

[0664] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0665] Conventional health management systems analyze biometric information acquired through the camera and microphone of a user's smart device and can provide a diagnosis and prescription based on the results. However, these systems do not support the use of biometric measurement results to recommend health-related products, deliver those products, or monitor their effectiveness. As a result, they do not adequately provide a means for users to use appropriate health-related products based on their own health status and continuously improve their health. Furthermore, it is difficult to track the effectiveness of products and make optimal recommendations for individual users.

[0666] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0667] In this invention, the server includes a means for acquiring biometric information of a user, a means for analyzing the biometric information, a means for making a diagnosis based on the analysis results and past medical information, a means for prescribing a prescription based on the diagnosis results, a means for presenting the prescribed health-related product, a means for delivering the health-related product, and a means for monitoring the effects. This makes it possible to analyze the user's biometric information and emotional information, and based on this, to propose and deliver the most appropriate health-related product, and further monitor the effects over time.

[0668] "User" refers to an individual who uses this system.

[0669] "Biometric information" refers to data about a user's basic bodily functions, such as temperature, heart rate, and breathing rate.

[0670] "Analysis" refers to data processing to evaluate and diagnose health status based on acquired biometric information.

[0671] "Diagnosis" refers to assessing the user's health condition and determining necessary treatment and measures based on analysis results and past medical information.

[0672] "Prescription" refers to the act of providing needed medicines or health-related products based on diagnostic results.

[0673] "Health-related products" refers to supplements, exercise equipment, health foods, etc. used to maintain or improve the user's health.

[0674] "Delivery" refers to the delivery of health-related products selected or prescribed by the user to the address specified by the user.

[0675] "Follow-up" refers to the ongoing monitoring and evaluation of a user's health status after using a prescribed product.

[0676] This system acquires a user's biometric and emotional information, and based on that information, diagnoses their health, recommends and delivers health-related products, and monitors their effectiveness. This system is implemented using smart devices, a server, a machine learning model, and an emotion analysis engine.

[0677] System Overview

[0678] The system can be broadly divided into three parts: a part that acquires the user's biometric information, a part that analyzes the biometric and emotional information, a part that suggests and delivers health-related products based on the analysis results, and finally a part that monitors the effects over time.

[0679] Acquiring user's biometric information

[0680] The user launches the smartphone application and uses the camera and microphone to measure their heart rate, respiratory rate, body temperature, facial expressions, and voice. Specifically, image processing software such as OpenCV is used to extract biometric information from the camera footage, and emotional information is analyzed from the voice and facial expression data using software such as EmotionAPI.

[0681] Data analysis

[0682] The acquired biometric and emotional information is sent from the device to a server, which then analyzes the data using machine learning models such as TensorFlow. The analysis results are then compared with past medical information to assess the user's health status.

[0683] Proposal and delivery of health-related products

[0684] Based on the diagnosis results, appropriate health-related products are recommended to the user. These products can be purchased directly through the application and delivered to the specified address. Product recommendations are made in conjunction with a health database that is collated with the analysis results.

[0685] Follow-up of effects

[0686] Even after a user has purchased or taken a prescribed product, biometric and emotional information is periodically collected again to monitor progress. The collected data is then sent to the server for further analysis. Based on the results, new recommendations or adjustments are made.

[0687] Specific examples

[0688] For example, a user opens the app every morning and checks their health using the camera and microphone. Based on the results, the app diagnoses a vitamin D deficiency and recommends vitamin supplements. The user purchases the supplements and checks again a week later. The app also collects this data and uses it to make future recommendations.

[0689] Prompt Sentence Examples

[0690] To assess the user's health, perform the following steps:

[0691] 1. Turn on the camera to measure your heart rate, breathing rate, and temperature.

[0692] 2. Next, the microphone captures the voice and recognizes facial expressions.

[0693] 3. Based on the analysis results, we recommend appropriate health-related products.

[0694] For example, if you have a vitamin D deficiency, you might see a push notification like this:

[0695] "I suggest a vitamin D supplement. And how about some exercise equipment along with that?"

[0696] In this way, the present invention can provide comprehensive support for the user's health management.

[0697] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0698] Step 1: User biometric information acquisition

[0699] The user launches a smartphone application, which uses the camera and microphone to measure their heart rate, respiration rate, body temperature, facial expressions, and voice. Specifically, the smartphone camera takes a picture of their face, and the heart rate and respiration rate are analyzed using image processing software such as OpenCV. The microphone also captures their voice, and emotional information is analyzed using EmotionAPI or similar. The input at this stage is the user's face image and voice data, and the output is the analyzed heart rate, respiration rate, body temperature, and emotional information.

[0700] Step 2: Sending data to the server

[0701] The device sends the acquired and analyzed biometric and emotional information to the server. Specifically, the data is formatted in JSON format or similar and sent to the server using a secure protocol (e.g., HTTPS). The input is the analyzed biometric and emotional information, and the output is the data sent to the server.

[0702] Step 3: Data analysis by the server

[0703] The server analyzes the received biometric and emotional information using machine learning models such as TensorFlow. Past medical information is also collated during the analysis. Specifically, past medical information is retrieved from a database, and the user's health status is evaluated as the analysis result. The input is the transmitted biometric, emotional, and past medical information, and the output is the diagnosis result.

[0704] Step 4: Proposing health-related products

[0705] The server proposes appropriate health-related products based on the analysis and diagnosis results. It compares the analyzed data with a database of health-related products to select the most suitable products for the user. The proposals are notified to the user through the application. The input is the diagnosis results and a database of health-related products, and the output is a list of proposed products.

[0706] Step 5: Purchase and shipping

[0707] The user selects the desired product from the list of suggested products and completes the purchase process through the application. The server receives the purchase information and initiates the delivery process. The input is the user's purchase selection and delivery address information, and the output is product delivery instructions. The product delivery instructions are sent to the delivery company.

[0708] Step 6: Monitor the results

[0709] The user uses the purchased health-related product and periodically re-acquires biometric and emotional information. The device again transmits the newly acquired data to the server. The server again analyzes the data using a machine learning model and evaluates the effectiveness. If necessary, it provides the user with appropriate instructions or the next suggestion. The input is the newly acquired biometric and emotional information, and the output is the evaluation of the effectiveness and new suggestions.

[0710] Through the above steps, a system that comprehensively supports the user's health management is realized.

[0711] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0712] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0713] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0714] [Third embodiment]

[0715] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0716] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0717] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0718] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0719] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0720] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0721] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0722] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0723] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0724] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0725] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0726] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0727] System Overview

[0728] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biological information, analyze data, make diagnoses, prescribe medicines, and monitor progress. Specific embodiments of each part of the system are described below.

[0729] Acquisition of biometric information

[0730] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[0731] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into the required format, and then sent to the server.

[0732] Data analysis

[0733] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model. The server evaluates the user's current health condition based on past accumulated data and the trained model. The analysis results are generated as a simple diagnosis.

[0734] diagnosis

[0735] Server: The simple diagnosis results are provided to the chatbot, which then collects additional information through dialogue with the user and conducts a medical interview. The data obtained through this interactive interview is then sent back to the server and used for the final diagnosis.

[0736] Terminal: The chatbot asks the user questions and collects their answers to gain more detailed information, which improves the accuracy of the diagnosis.

[0737] Prescription of medication

[0738] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[0739] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[0740] Users: Users can receive their medication at home, allowing them to receive the medical care they need without leaving their homes.

[0741] Observation

[0742] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[0743] Server: The server receives the biometric data again and analyzes it with a machine learning model to evaluate the effectiveness of treatment. The analysis results are notified to the user and doctor, and necessary measures are taken.

[0744] Specific examples

[0745] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment in the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies them of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[0746] The processing flow will be explained below.

[0747] Step 1:

[0748] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera. The user follows the on-screen instructions to hold their face or hand in front of the camera to collect biometric information such as heart rate and respiratory rate.

[0749] Step 2:

[0750] Terminal: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the terminal and converted into an appropriate format. The data is then sent to the server.

[0751] Step 3:

[0752] Server: The server receives the biometric information sent from the device, stores the data in a database, analyzes the biometric information using a machine learning model, and generates a simple diagnosis result as the analysis result.

[0753] Step 4:

[0754] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates questions for the user and begins the medical interview.

[0755] Step 5:

[0756] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. These answers are then sent back to the server.

[0757] Step 6:

[0758] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes the final diagnosis and generates the results.

[0759] Step 7:

[0760] Server: Based on the final diagnosis, the doctor prescribes the necessary medication. The prescription is then electronically sent to the partner pharmacy.

[0761] Step 8:

[0762] Pharmacy: The pharmacy prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[0763] Step 9:

[0764] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[0765] Step 10:

[0766] Server: Receives progress information and additional biometric data entered by the user and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified and given appropriate instructions.

[0767] In this way, the system provides consistent daily health management and medical treatment through collaboration between users, terminals, and servers.

[0768] Example 1

[0769] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0770] Conventional health management systems have had issues with the user having to spend a lot of time and effort to obtain biometric information, and the diagnosis and treatment process is fragmented, making it difficult to provide comprehensive medical care.In addition, it is difficult to perform real-time diagnosis or interactive interviews, making it difficult to quickly and accurately grasp the user's health condition.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0772] In this invention, the server includes means for acquiring biometric information of a user, means for analyzing the biometric information, means for making a diagnosis based on the analysis result and past medical information, means for prescribing medicine based on the diagnosis result, means for delivering the prescribed medicine, means for monitoring the progress of treatment, means for activating a camera of a smart device to measure biometric information based on a health management option selected by the user, means for converting the biometric information into a predetermined format and transmitting it to the server, means for analyzing the biometric information using a machine learning model, and means for conducting an interview in an interactive format with the user using a chatbot, thereby enabling the user to receive consistent health management and medical services.

[0773] "User" refers to an individual who uses the system for a particular purpose.

[0774] "Biometric information" refers to various health-related data obtained from the human body (e.g., heart rate, respiratory rate, body temperature, etc.).

[0775] "Means" refers to a specific device, method, or part of a system used to achieve a particular purpose.

[0776] "Smart device" refers to a portable electronic device with advanced computing capabilities (e.g., smartphone, tablet, etc.).

[0777] "Camera" refers to a digital imaging device for capturing images.

[0778] "Analysis" refers to the process of examining the data obtained in detail and drawing conclusions or information.

[0779] "Server" refers to a high-performance computer system that manages, processes, and provides data over a network.

[0780] A "machine learning model" refers to a collection of algorithms and statistical models that learn from data and make predictions and decisions.

[0781] A "chatbot" refers to a program or artificial intelligence that mimics a conversation with a user.

[0782] "Format" refers to the particular structure or form of data.

[0783] "Medical interview" refers to the process by which a doctor gathers information from a patient about their symptoms and medical history.

[0784] "Diagnosis" refers to the process by which a medical professional identifies an illness or condition based on a patient's symptoms and test results.

[0785] "Prescription" refers to a doctor prescribing an appropriate medication to a patient.

[0786] "Pharmacy" means an establishment that prepares and dispenses medicines based on prescriptions.

[0787] "Delivery" refers to the process of getting an item to a designated location.

[0788] "Treatment" refers to medical procedures aimed at improving or recovering from an illness or disability.

[0789] "Follow-up" refers to the process of continually checking the progress of treatment and making any necessary adjustments.

[0790] The present invention is a system for supporting a user's health management, and is implemented using specific procedures, hardware, and software. This system acquires daily biological information and performs data analysis, diagnosis, drug prescription, and follow-up. Specific embodiments of each part are described below.

[0791] Acquisition of biometric information

[0792] User: The user launches the LINE app on their smart device and selects the "Health Management" option. The app activates the camera and displays an interface for measuring heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure their biometric information.

[0793] Device: The camera on the smart device captures and analyzes the user's video data, collecting biometric information such as heart rate, respiratory rate, and body temperature. This data is temporarily stored on the device, converted into a specific format, and then sent to a server.

[0794] Data analysis

[0795] Server: The server receives the transmitted biometric information and analyzes it using a machine learning model. The model used for the analysis is based on past accumulated data and training data. From the analysis results, the server evaluates the user's current health condition and generates a simple diagnosis.

[0796] diagnosis

[0797] Server: The simple diagnosis results are provided to the chatbot, which then asks the user additional questions. The chatbot then interacts with the user to gather more information and sends that data back to the server. The final diagnosis is based on this data.

[0798] Terminal: The terminal sends additional questions to the user through the chatbot and collects the user's answers, thereby improving the accuracy of the diagnosis.

[0799] Prescription of medication

[0800] Server: Based on the final diagnosis, the server generates instructions for the doctor to prescribe the necessary medication. The prescription is sent to a partner pharmacy, which prepares the medication and arranges for delivery.

[0801] User: The user receives the medication at home and takes it as directed.

[0802] Observation

[0803] User: The user can periodically enter the progress of treatment in the LINE app, and can obtain vital data again and send it to the server.

[0804] Server: The server receives the new biometric data and re-analyzes it using machine learning models. The results are then communicated to the user and doctor, and further action is taken if necessary.

[0805] (Example)

[0806] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data and evaluates the effectiveness of the treatment. This allows users to receive comprehensive health management and medical services from the comfort of their own home.

[0807] (Example of a prompt)

[0808] "Measure your heart rate and breathing rate."

[0809] "Please select the health management option and enter your data."

[0810] "I'll do a quick checkup. Please assess your health."

[0811] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0812] Step 1: Obtaining biometric information

[0813] User: The user launches the LINE app and enters health management mode by selecting the "Health Management" option. User action (selecting an option) is required as input.

[0814] Terminal: The terminal activates the smart device's camera and displays the interface. When the user holds their face or hand over the camera, the camera captures video data. Video data is provided as input data, and biometric information such as heart rate, respiratory rate, and body temperature is generated as output.

[0815] How it works: The camera module captures video data, which is then analyzed by internal software to measure biometric information, which is then temporarily stored in the device's memory.

[0816] Step 2: Sending biometric information

[0817] Terminal: The terminal converts the collected biometric data into a predetermined format. This conversion process includes noise removal and data normalization. As input, it requires raw biometric data, and as output, it obtains formatted data.

[0818] The formatted data is then sent to the server using the terminal's network module.

[0819] What it does: Specific algorithms and techniques are used to transform the data into a different format. The optimized data is then sent to the server using a secure protocol.

[0820] Step 3: Analyze the data

[0821] Server: The server receives the transmitted biometric data. It requires formatted biometric data transmitted from the device as input data. This data is then fed into a machine learning model and analyzed to assess the health status.

[0822] How it works: The machine learning model analyzes the data and detects abnormalities, such as heart rate fluctuation patterns or breathing rate. The analysis results are temporarily stored in a database on the server. The analysis results and a simple diagnosis are generated as output.

[0823] Step 4: Simple diagnosis and interview

[0824] Server: The server generates a simple diagnosis based on the analysis results. The input data is the analysis results of the machine learning model. This result is provided to the chatbot, which then begins interviewing the user.

[0825] Terminal: The terminal sends follow-up questions to the user through the chatbot and collects answers from the user. The input data includes the user's answers, and the output provides additional health information.

[0826] Specific actions: For example, specific prompts such as "Please tell us about your recent fatigue and appetite changes" are displayed to the user. These answers are sent to the server in real time.

[0827] Step 5: Sending diagnostic results and prescribing medication

[0828] Server: The server combines the medical interview data and biometric information to make the final diagnosis. The analysis results and medical interview data are required as input data. Based on this, the doctor prescribes the necessary medication and generates a prescription.

[0829] Specific operation: Prescription data is automatically generated and electronically transmitted to affiliated pharmacies. As an output, information on the completed prescription is generated.

[0830] Step 6: Follow-up and evaluation

[0831] User: The user periodically updates the progress of their treatment on the LINE app, and their vital data is acquired again and sent to the server. The input data includes new vital information and the progress of treatment.

[0832] Server: The server reanalyzes the updated information and uses machine learning models to evaluate the effectiveness of the treatment. The input data is the updated biometric information, and the output is the evaluation result of the treatment effectiveness.

[0833] Specific operation: Based on the analysis results, the server provides feedback to the user, such as "Treatment is progressing smoothly" or "Additional tests are required." The results are then notified to the user and their doctor.

[0834] (Application example 1)

[0835] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0836] The working environment for employees at logistics centers is harsh, making health management important. However, it is difficult for employees to properly monitor their own health status and detect abnormalities early amid their busy work schedules. Furthermore, there are limited means to respond quickly when abnormalities are discovered. Given this background, there is a need for an efficient and effective method of managing the health of employees working at logistics centers.

[0837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0838] In this invention, the server includes means for acquiring user biometric information, means for analyzing the biometric information, means for monitoring the biometric information in real time if the user is an employee working at a logistics center, means for making a diagnosis based on the analysis results and past medical information, means for notifying a manager when abnormal biometric information is detected, means for prescribing medication based on the diagnosis results, means for delivering the prescribed medication, and means for monitoring the progress of treatment. This allows for efficient and effective health management of employees at the logistics center and enables rapid response to abnormalities.

[0839] "Users" refer to employees working at the logistics center.

[0840] "Biometric information" is data that indicates the user's health condition, such as heart rate, respiratory rate, and body temperature.

[0841] A "smart device" is an information device such as a smartphone or tablet that is carried by a user.

[0842] A "camera" is an image capture device built into a smart device.

[0843] "Analysis" refers to the process of analyzing collected biometric information using methods such as machine learning models to extract useful information.

[0844] "Diagnosis" refers to the act of evaluating the user's health condition based on the analysis results of biological information and past medical information.

[0845] "Prescription" is the act of a doctor prescribing the necessary medication based on diagnostic results.

[0846] "Delivery" refers to the act of delivering prescribed medicine to a location designated by the user.

[0847] "Follow-up" is the act of monitoring the progress of treatment and continually assessing health status.

[0848] "Administrator" refers to the person in charge of managing the health of employees at the logistics center.

[0849] The present invention is a system for supporting the health management of employees working at a logistics center. This system works in conjunction with smart devices to consistently acquire biometric information, analyze data, diagnose, notify abnormalities, prescribe medication, and monitor treatment progress. Specific embodiments of each part of the system are described below.

[0850] Acquisition of biometric information

[0851] User: The user launches the application on their smart device and holds their face or hand over the camera to measure their biometric information, including heart rate, respiratory rate, and body temperature.

[0852] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[0853] Data analysis

[0854] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model (for example, a model built with Keras). The server evaluates the user's current health status based on past accumulated data and the trained model. If an abnormality is detected, the analysis results will notify the administrator.

[0855] Diagnostics and Notifications

[0856] Server: Based on the analysis results and past medical information, the server diagnoses the user's health condition. If abnormal vital signs are detected, a notification is sent to the administrator, prompting prompt action.

[0857] Managers: Managers can monitor the health status of employees within the distribution center and take appropriate action if necessary.

[0858] Medication and follow-up

[0859] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[0860] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[0861] Users: Users can receive medication at home, allowing them to receive the medical care they need without leaving their homes. Furthermore, they can enter their treatment progress through an application on their smart device.

[0862] Server: The server receives the retransmitted vital signs and analyzes them with a machine learning model to evaluate the effectiveness of treatment. The analysis results are then notified to the user and doctor, and necessary measures are taken.

[0863] Specific examples

[0864] For example, if an employee working at a logistics center feels unwell, they first launch an application on their smart device and hold their face up to the camera to measure their heart rate and respiratory rate. The data is then sent to a server where it is analyzed using a machine learning model. If an abnormality is detected based on the analysis results, an alert is sent to an administrator. The administrator can then take prompt action, requesting a diagnosis from a doctor if necessary and prescribing medication. The user then receives the medication at home and records the progress of their treatment in the application, and the server analyzes the data to continue monitoring their condition.

[0865] Prompt Sentence Examples

[0866] For example, the following prompts could be fed into a generative AI model:

[0867] "We want to build a system in our logistics center that allows employees to monitor their daily health using their smartphones. Using smart devices, we can acquire vital signs (heart rate, respiratory rate, body temperature) and analyze them using machine learning models. If any abnormal values ​​are detected, a notification will be sent to the manager."

[0868] In this way, a system can be realized that comprehensively manages the health of employees working at logistics centers.

[0869] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0870] Step 1:

[0871] User: Launches the smart device application and holds their face or hand in front of the camera. The input is video data captured by the smart device's camera. The output is biometric information such as heart rate, respiratory rate, and body temperature.

[0872] Step 2:

[0873] Terminal: Analyzes video data captured by the smart device's camera to collect biometric information. Video data provided by the user is used as input. Analyzed biometric information is obtained as output. Specifically, the smart device's image processing function is used to estimate heart rate and respiratory rate. The data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[0874] Step 3:

[0875] Server: Receives the transmitted biometric information and analyzes it using a machine learning model (built using Keras). The biometric information sent from the device is used as input. The output is an assessment of the user's current health status. Specifically, the server compares the biometric information with an existing database to check for any abnormalities.

[0876] Step 4:

[0877] Server: Diagnoses the user's health condition based on the analysis results and past medical information. Analysis results and past medical information are used as input. Diagnosis results are obtained as output. Specifically, if an abnormality is detected based on the analysis results, the next step is notification processing.

[0878] Step 5:

[0879] Server: If abnormal biometric information is detected, it sends a notification to the administrator. The diagnosis results are used as input. The notification data to be sent to the administrator is obtained as output. Specifically, when an abnormality is detected, an alert is sent to the administrator's smart device using the LINE API.

[0880] Step 6:

[0881] Administrator: Receives notifications, monitors employee health status in real time, and takes appropriate action. The input is the notification sent from the server. The output is the response procedure to be taken by the administrator. Specifically, the administrator who receives the notification will request a diagnosis from a doctor if necessary, and ensure the safety of the employee.

[0882] Step 7:

[0883] Server: Based on the diagnosis results, the doctor prescribes the necessary medication. The diagnosis results are used as input. A prescription is generated as output. Specifically, the server creates an electronic prescription based on instructions from the doctor and sends it to a partner pharmacy.

[0884] Step 8:

[0885] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescription is used as input. The output is the medicine to be sent to the user's delivery address. The specific operation is that the pharmacy prepares the prescription and arranges for delivery to the specified address.

[0886] Step 9:

[0887] User: Receives medication at home and performs treatment. The input is the delivered medication. The output is data on the progress of treatment. The specific operation is that the user receives the medication and records the progress of treatment through the application.

[0888] Step 10:

[0889] Server: Receives the progress of treatment and analyzes the vital signs again. The input is the treatment progress data sent by the user. The output is the evaluation result of the treatment effect. Specifically, the server analyzes the new vital signs and checks whether the treatment is effective. The results are notified to the user and the doctor.

[0890] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0891] System Overview

[0892] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biometric information, analyze data, make diagnoses, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of the system are described below.

[0893] Acquisition of biometric information

[0894] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[0895] Device: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to the server.

[0896] Acquiring emotional information

[0897] Device: The smart device's camera and microphone are used to capture the user's facial expressions and vocal changes and send them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[0898] Data analysis

[0899] Server: The transmitted biometric and emotional information is received by the server and stored in a database. The server analyzes the biometric information using a machine learning model and also incorporates emotional information from the emotion engine into the analysis. This allows for a more accurate assessment of the user's health condition.

[0900] diagnosis

[0901] Server: The analysis results are generated as a simple diagnosis. This simple diagnosis result is provided to the LINE chatbot, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[0902] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. The chatbot asks questions taking into account the user's emotional state, allowing the user to respond more naturally.

[0903] Prescription of medication

[0904] Server: Additional information collected through the interview is sent to the server and reanalyzed for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes a final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[0905] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prepared medicine is then sent to the address specified by the user via a delivery service.

[0906] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[0907] Observation

[0908] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[0909] Server: The server receives the biometric and emotional information again and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified of the results and given appropriate instructions.

[0910] Specific examples

[0911] For example, suppose a user feels unwell. In this case, the user first activates the camera through the LINE app and measures their heart rate and breathing rate. This data is sent to the server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the analysis results, the chatbot conducts a medical interview with the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies the user of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[0912] The processing flow will be explained below.

[0913] Step 1:

[0914] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate and breathing rate. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[0915] Step 2:

[0916] Device: The smart device's camera captures biometric information such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the device and converted into an appropriate format. This data is then sent to the server.

[0917] Step 3:

[0918] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The video and audio data is sent to the emotion engine, which analyzes them in real time and recognizes the user's emotional state (e.g., joy, sadness, anger, surprise).

[0919] Step 4:

[0920] Server: Receives biometric and emotional information sent from the device and stores it in a database. The server uses a machine learning model to analyze the biometric information and also incorporates emotional information into the analysis. This allows the server to evaluate the user's current health condition and generate a simple diagnosis.

[0921] Step 5:

[0922] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates individual questions for the user and begins the medical interview.

[0923] Step 6:

[0924] On the device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. During this process, the chatbot dynamically adjusts the questions to take into account the user's emotional state.

[0925] Step 7:

[0926] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and past medical information, the doctor makes the final diagnosis and generates a diagnosis result.

[0927] Step 8:

[0928] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and electronically sends it to a partner pharmacy.

[0929] Step 9:

[0930] Pharmacy: Prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[0931] Step 10:

[0932] User: The user receives the medication at home. After taking it, they enter information about their treatment progress and side effects through the LINE app. Vital data may also be collected again.

[0933] Step 11:

[0934] Server: Receives the biometric and emotional information entered by the user again and analyzes it using a machine learning model. The analysis results are sent to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The results are also sent to the user, who is given appropriate instructions.

[0935] This system allows users to receive comprehensive health management and medical services from the comfort of their own home. By combining it with an emotion engine, it is possible to provide more accurate diagnoses and personalized medical services.

[0936] Example 2

[0937] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0938] Conventional health management systems lack the ability to incorporate visual and audio emotional information from users, making it difficult to comprehensively assess a user's accurate health and emotional state. This can lead to poor diagnostic accuracy and a poor user experience. Furthermore, the lack of machine learning models to analyze biometric and emotional information limits the accuracy of data analysis. It is desirable to address these issues and realize comprehensive user health management.

[0939] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0940] In this invention, the server includes a means for acquiring general information about the user, a means for analyzing the general information, and a means for making a diagnosis based on the analysis results and past health information. This enables comprehensive health management for the user. Furthermore, a system is provided that includes a means for acquiring the user's emotions using a camera and microphone of a smart device, a means for analyzing the emotional information, a means for reflecting the analyzed emotional information in the diagnosis results, a means for analyzing the general information and emotional information using a machine learning model, and a means for conducting an interview with the user in an interactive format based on the analysis results. This allows for a more accurate assessment of the user's health status, an appropriate diagnosis and prescription, and more personalized medical services.

[0941] A "user" is an individual who uses the system to manage their own health.

[0942] "Biometric information" refers to various data obtained from the user's body, such as heart rate, respiratory rate, body temperature, and blood pressure.

[0943] "General information" is a blanket term that refers to a variety of data, including biometric and emotional information about the user.

[0944] "Analysis" is the process of evaluating and making judgments on collected data using machine learning models and algorithms.

[0945] "Determination" refers to evaluating the user's health condition based on the analysis results and past health information.

[0946] A "prescription" is an act in which a doctor prescribes necessary medication based on the user's health condition.

[0947] "Emotion information" is data that indicates the psychological state of the user, obtained from facial expressions, voice, etc.

[0948] A "smart device" is an electronic device such as a mobile phone or tablet that includes sensors such as a camera or microphone.

[0949] The "emotion engine" is a system that analyzes captured facial expressions and voice data to recognize the user's emotions.

[0950] A "machine learning model" is an algorithm or computational model that identifies patterns based on large amounts of data and makes predictions and judgments.

[0951] "Dialogue format" refers to a format in which communication with users is interactive, such as through chatbots.

[0952] This invention is a system that supports users' health management, and by using a common messaging app, it is possible to consistently acquire daily biometric information, analyze data, make judgments, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of this system will be described.

[0953] Acquisition of biometric information

[0954] A user launches a messaging app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[0955] The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to a server.

[0956] Acquiring emotional information

[0957] The device uses the smart device's camera and microphone to capture the user's facial expressions and vocal changes and transmit them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[0958] Data analysis

[0959] The server receives the transmitted biometric and emotional information and stores it in a database. The server then analyzes the biometric information using a machine learning model and incorporates the emotional information from the emotion engine into the analysis, thereby enabling a more accurate assessment of the user's health condition.

[0960] judgement

[0961] The server generates a simple diagnostic result based on the analysis results. This simple diagnostic result is provided to the chatbot of the messaging app, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[0962] The chatbot's questions are displayed on the device's chat screen in the messaging app, and the user can respond to them and provide additional health information. The chatbot takes the user's emotional state into account when posing questions, allowing the user to respond more naturally.

[0963] Prescription of medication

[0964] The server analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The doctor makes the final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[0965] The pharmacy prepares the medicine based on the prescription and arranges for delivery, which is then sent to the address specified by the user via a delivery service.

[0966] Users receive and take their medication at home, and can enter progress information, including treatment progress and side effects, through a messaging app.

[0967] Observation

[0968] The user periodically enters the progress of treatment through the messaging app, and it is also possible to collect vital data again and send it to the server.

[0969] The server receives the retransmitted biometric and emotional information and reanalyzes it using a machine learning model. The analysis results are then sent to the doctor to evaluate the effectiveness of the treatment, and necessary measures are taken. The user is also notified of the results and given appropriate instructions.

[0970] Specific examples

[0971] For example, if a user feels unwell, they first activate the camera through a messaging app to measure their heart rate and breathing rate. This data is sent to a server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the results of this analysis, the chatbot conducts a medical interview with the user, and additional information is collected accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is then delivered to the user's home from an affiliated pharmacy. The user then enters the progress of their treatment through the messaging app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notify the user of the results. This enables users to receive comprehensive health management and medical care from the comfort of their own home.

[0972] Prompt sentence for generative AI model

[0973] "Please explain in natural language how a user can manage their health using a messaging app. Please show the flow of the system, including specific actions, from obtaining biometric and emotional information, to making a judgment, prescribing medication, and monitoring progress."

[0974] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0975] Step 1: Launch your messaging app

[0976] User: Launches a messaging app and selects the "Health Management" option. The input is a user action, and the output is the app switching to "Health Management" mode. Specific actions include the user operating the smartphone screen and tapping the health management option within the app.

[0977] Step 2: Measuring biometric data

[0978] Terminal: The smart device's camera activates and displays an interface for measuring heart rate, respiratory rate, and body temperature. The input is the user's face or hand, and the output is biometric data. Specific operations include the device's camera using facial recognition technology to capture the user's face or hand and measure biometric information.

[0979] Step 3: Storing and sending data

[0980] Terminal: The acquired biometric data is temporarily stored on the terminal, converted into an appropriate format, and sent to the server. The input is the measured biometric data, and the output is the format-converted data sent to the server. Specific operations include the terminal converting the measured data into JSON format, etc., and sending the data to the server via the HTTPS protocol.

[0981] Step 4: Capturing emotional information

[0982] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The input is the user's facial expressions and voice, and the output is the captured emotion data. Specifically, the device's camera analyzes the user's facial expressions using a facial expression recognition algorithm, and the microphone records voice data.

[0983] Step 5: Sending emotional information

[0984] Device: Sends captured data to the emotion engine. The input is the captured emotion data, and the output is the data sent to the emotion engine. Specific operations include the device sending facial expression and voice data to the emotion engine's analysis API.

[0985] Step 6: Receiving and storing data

[0986] Server: Receives the transmitted biometric and emotional information and stores it in a database. The input is the data transmitted from the device, and the output is the data stored in the database. Specifically, the server receives the data at the API endpoint where it receives data and stores it in an SQL database.

[0987] Step 7: Analysis using machine learning models

[0988] Server: Analyzes biometric and emotional information using a machine learning model. The input is the stored biometric and emotional information, and the output is the analysis results. Specific operations include the server analyzing the data using Python's Scikit-learn or TensorFlow to evaluate the user's health status.

[0989] Step 8: Generate quick diagnostic results

[0990] Server: Generates a simple diagnostic result based on the analysis results. The input is the analysis result from the machine learning model, and the output is a simple diagnostic result. Specifically, the server uses a specialized rule-based algorithm to output the diagnostic result from the analysis results.

[0991] Step 9: Chatbot consultation

[0992] Server: Provides the generated simple diagnosis results to the chatbot of the messaging app, which then generates additional questions for the user. The input is the simple diagnosis results, and the output is the additional questions sent to the user. Specifically, the server sends data to the chatbot using the messaging API, and the chatbot then sends dynamically generated questions to the user.

[0993] Terminal: Questions from the chatbot are displayed on the chat screen of the messaging app. The input is the question from the chatbot, and the output is the user's answer. Specific actions include the user opening the chat screen on their smartphone and answering the displayed question by typing.

[0994] Step 10: Generate a final diagnosis and prescription

[0995] Server: Analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The input is the medical interview results and past health information, and the output is the final diagnosis and prescription data. Specific operations include a remote doctor on the server using the electronic diagnostic system to confirm the final diagnosis and create an electronic prescription.

[0996] Step 11: Submit your prescription

[0997] Server: Electronically transmits prescriptions to affiliated pharmacies. The input is prescription data, and the output is the prescription sent to the pharmacy. Specifically, the server transmits data to the pharmacy's electronic prescription system.

[0998] Step 12: Prepare and deliver medication

[0999] Pharmacy: Prepares medicine based on prescription and arranges for delivery. The input is the received prescription data, and the output is the medicine arranged for delivery. Specifically, the pharmacy prepares the medicine, provides the information to the delivery service, and delivers it to the user.

[1000] User: Receives and takes medicine. The input is the delivered medicine, and the output is progress information after taking it. The specific operation is for the user to receive the delivered medicine and use it according to the dosage instructions.

[1001] Step 13: Enter progress information

[1002] User: Enters treatment progress and side effects into a messaging app. The input is treatment progress information, and the output is the progress information sent to the server. Specifically, the user enters progress information using a form within the app and presses the send button.

[1003] Step 14: Reanalyze the data

[1004] Server: Analyzes the retransmitted biometric and emotional information. The input is the newly transmitted biometric and emotional information, and the output is the reanalysis results. Specifically, the server receives the new data and analyzes it again using the machine learning model.

[1005] Step 15: Notification of results

[1006] Server: Notifies the doctor of the analysis results and takes necessary measures. The input is the reanalysis results, and the output is notification data for the doctor and a result notification for the user. Specifically, the server sends the analysis results to the doctor via email or a notification system, and notifies the user of the results via a messaging app.

[1007] (Application example 2)

[1008] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1009] Conventional health management systems analyze biometric information acquired through the camera and microphone of a user's smart device and can provide a diagnosis and prescription based on the results. However, these systems do not support the use of biometric measurement results to recommend health-related products, deliver those products, or monitor their effectiveness. As a result, they do not adequately provide a means for users to use appropriate health-related products based on their own health status and continuously improve their health. Furthermore, it is difficult to track the effectiveness of products and make optimal recommendations for individual users.

[1010] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1011] In this invention, the server includes a means for acquiring biometric information of a user, a means for analyzing the biometric information, a means for making a diagnosis based on the analysis results and past medical information, a means for prescribing a prescription based on the diagnosis results, a means for presenting the prescribed health-related product, a means for delivering the health-related product, and a means for monitoring the effects. This makes it possible to analyze the user's biometric information and emotional information, and based on this, to propose and deliver the most appropriate health-related product, and further monitor the effects over time.

[1012] "User" refers to an individual who uses this system.

[1013] "Biometric information" refers to data about a user's basic bodily functions, such as temperature, heart rate, and breathing rate.

[1014] "Analysis" refers to data processing to evaluate and diagnose health status based on acquired biometric information.

[1015] "Diagnosis" refers to assessing the user's health condition and determining necessary treatment and measures based on analysis results and past medical information.

[1016] "Prescription" refers to the act of providing needed medicines or health-related products based on diagnostic results.

[1017] "Health-related products" refers to supplements, exercise equipment, health foods, etc. used to maintain or improve the user's health.

[1018] "Delivery" refers to the delivery of health-related products selected or prescribed by the user to the address specified by the user.

[1019] "Follow-up" refers to the ongoing monitoring and evaluation of a user's health status after using a prescribed product.

[1020] This system acquires a user's biometric and emotional information, and based on that information, diagnoses their health, recommends and delivers health-related products, and monitors their effectiveness. This system is implemented using smart devices, a server, a machine learning model, and an emotion analysis engine.

[1021] System Overview

[1022] The system can be broadly divided into three parts: a part that acquires the user's biometric information, a part that analyzes the biometric and emotional information, a part that suggests and delivers health-related products based on the analysis results, and finally a part that monitors the effects over time.

[1023] Acquiring user's biometric information

[1024] The user launches the smartphone application and uses the camera and microphone to measure their heart rate, respiratory rate, body temperature, facial expressions, and voice. Specifically, image processing software such as OpenCV is used to extract biometric information from the camera footage, and emotional information is analyzed from the voice and facial expression data using software such as EmotionAPI.

[1025] Data analysis

[1026] The acquired biometric and emotional information is sent from the device to a server, which then analyzes the data using machine learning models such as TensorFlow. The analysis results are then compared with past medical information to assess the user's health status.

[1027] Proposal and delivery of health-related products

[1028] Based on the diagnosis results, appropriate health-related products are recommended to the user. These products can be purchased directly through the application and delivered to the specified address. Product recommendations are made in conjunction with a health database that is collated with the analysis results.

[1029] Follow-up of effects

[1030] Even after a user has purchased or taken a prescribed product, biometric and emotional information is periodically collected again to monitor progress. The collected data is then sent to the server for further analysis. Based on the results, new recommendations or adjustments are made.

[1031] Specific examples

[1032] For example, a user opens the app every morning and checks their health using the camera and microphone. Based on the results, the app diagnoses a vitamin D deficiency and recommends vitamin supplements. The user purchases the supplements and checks again a week later. The app also collects this data and uses it to make future recommendations.

[1033] Prompt Sentence Examples

[1034] To assess the user's health, perform the following steps:

[1035] 1. Turn on the camera to measure your heart rate, breathing rate, and temperature.

[1036] 2. Next, the microphone captures the voice and recognizes facial expressions.

[1037] 3. Based on the analysis results, we recommend appropriate health-related products.

[1038] For example, if you have a vitamin D deficiency, you might see a push notification like this:

[1039] "I suggest a vitamin D supplement. And how about some exercise equipment along with that?"

[1040] In this way, the present invention can provide comprehensive support for the user's health management.

[1041] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1042] Step 1: User biometric information acquisition

[1043] The user launches a smartphone application, which uses the camera and microphone to measure their heart rate, respiration rate, body temperature, facial expressions, and voice. Specifically, the smartphone camera takes a picture of their face, and the heart rate and respiration rate are analyzed using image processing software such as OpenCV. The microphone also captures their voice, and emotional information is analyzed using EmotionAPI or similar. The input at this stage is the user's face image and voice data, and the output is the analyzed heart rate, respiration rate, body temperature, and emotional information.

[1044] Step 2: Sending data to the server

[1045] The device sends the acquired and analyzed biometric and emotional information to the server. Specifically, the data is formatted in JSON format or similar and sent to the server using a secure protocol (e.g., HTTPS). The input is the analyzed biometric and emotional information, and the output is the data sent to the server.

[1046] Step 3: Data analysis by the server

[1047] The server analyzes the received biometric and emotional information using machine learning models such as TensorFlow. Past medical information is also collated during the analysis. Specifically, past medical information is retrieved from a database, and the user's health status is evaluated as the analysis result. The input is the transmitted biometric, emotional, and past medical information, and the output is the diagnosis result.

[1048] Step 4: Proposing health-related products

[1049] The server proposes appropriate health-related products based on the analysis and diagnosis results. It compares the analyzed data with a database of health-related products to select the most suitable products for the user. The proposals are notified to the user through the application. The input is the diagnosis results and a database of health-related products, and the output is a list of proposed products.

[1050] Step 5: Purchase and shipping

[1051] The user selects the desired product from the list of suggested products and completes the purchase process through the application. The server receives the purchase information and initiates the delivery process. The input is the user's purchase selection and delivery address information, and the output is product delivery instructions. The product delivery instructions are sent to the delivery company.

[1052] Step 6: Monitor the results

[1053] The user uses the purchased health-related product and periodically re-acquires biometric and emotional information. The device again transmits the newly acquired data to the server. The server again analyzes the data using a machine learning model and evaluates the effectiveness. If necessary, it provides the user with appropriate instructions or the next suggestion. The input is the newly acquired biometric and emotional information, and the output is the evaluation of the effectiveness and new suggestions.

[1054] Through the above steps, a system that comprehensively supports the user's health management is realized.

[1055] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1056] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1057] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1058] [Fourth embodiment]

[1059] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1060] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1061] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1062] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1063] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1064] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1065] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1066] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1067] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1068] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1069] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1070] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1071] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1072] System Overview

[1073] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biological information, analyze data, make diagnoses, prescribe medicines, and monitor progress. Specific embodiments of each part of the system are described below.

[1074] Acquisition of biometric information

[1075] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[1076] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into the required format, and then sent to the server.

[1077] Data analysis

[1078] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model. The server evaluates the user's current health condition based on past accumulated data and the trained model. The analysis results are generated as a simple diagnosis.

[1079] diagnosis

[1080] Server: The simple diagnosis results are provided to the chatbot, which then collects additional information through dialogue with the user and conducts a medical interview. The data obtained through this interactive interview is then sent back to the server and used for the final diagnosis.

[1081] Terminal: The chatbot asks the user questions and collects their answers to gain more detailed information, which improves the accuracy of the diagnosis.

[1082] Prescription of medication

[1083] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[1084] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[1085] Users: Users can receive their medication at home, allowing them to receive the medical care they need without leaving their homes.

[1086] Observation

[1087] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[1088] Server: The server receives the biometric data again and analyzes it with a machine learning model to evaluate the effectiveness of treatment. The analysis results are notified to the user and doctor, and necessary measures are taken.

[1089] Specific examples

[1090] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment in the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies them of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[1091] The processing flow will be explained below.

[1092] Step 1:

[1093] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera. The user follows the on-screen instructions to hold their face or hand in front of the camera to collect biometric information such as heart rate and respiratory rate.

[1094] Step 2:

[1095] Terminal: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the terminal and converted into an appropriate format. The data is then sent to the server.

[1096] Step 3:

[1097] Server: The server receives the biometric information sent from the device, stores the data in a database, analyzes the biometric information using a machine learning model, and generates a simple diagnosis result as the analysis result.

[1098] Step 4:

[1099] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates questions for the user and begins the medical interview.

[1100] Step 5:

[1101] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. These answers are then sent back to the server.

[1102] Step 6:

[1103] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes the final diagnosis and generates the results.

[1104] Step 7:

[1105] Server: Based on the final diagnosis, the doctor prescribes the necessary medication. The prescription is then electronically sent to the partner pharmacy.

[1106] Step 8:

[1107] Pharmacy: The pharmacy prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[1108] Step 9:

[1109] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[1110] Step 10:

[1111] Server: Receives progress information and additional biometric data entered by the user and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified and given appropriate instructions.

[1112] In this way, the system provides consistent daily health management and medical treatment through collaboration between users, terminals, and servers.

[1113] Example 1

[1114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1115] Conventional health management systems have had issues with the user having to spend a lot of time and effort to obtain biometric information, and the diagnosis and treatment process is fragmented, making it difficult to provide comprehensive medical care.In addition, it is difficult to perform real-time diagnosis or interactive interviews, making it difficult to quickly and accurately grasp the user's health condition.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1117] In this invention, the server includes means for acquiring biometric information of a user, means for analyzing the biometric information, means for making a diagnosis based on the analysis result and past medical information, means for prescribing medicine based on the diagnosis result, means for delivering the prescribed medicine, means for monitoring the progress of treatment, means for activating a camera of a smart device to measure biometric information based on a health management option selected by the user, means for converting the biometric information into a predetermined format and transmitting it to the server, means for analyzing the biometric information using a machine learning model, and means for conducting an interview in an interactive format with the user using a chatbot, thereby enabling the user to receive consistent health management and medical services.

[1118] "User" refers to an individual who uses the system for a particular purpose.

[1119] "Biometric information" refers to various health-related data obtained from the human body (e.g., heart rate, respiratory rate, body temperature, etc.).

[1120] "Means" refers to a specific device, method, or part of a system used to achieve a particular purpose.

[1121] "Smart device" refers to a portable electronic device with advanced computing capabilities (e.g., smartphone, tablet, etc.).

[1122] "Camera" refers to a digital imaging device for capturing images.

[1123] "Analysis" refers to the process of examining the data obtained in detail and drawing conclusions or information.

[1124] "Server" refers to a high-performance computer system that manages, processes, and provides data over a network.

[1125] A "machine learning model" refers to a collection of algorithms and statistical models that learn from data and make predictions and decisions.

[1126] A "chatbot" refers to a program or artificial intelligence that mimics a conversation with a user.

[1127] "Format" refers to the particular structure or form of data.

[1128] "Medical interview" refers to the process by which a doctor gathers information from a patient about their symptoms and medical history.

[1129] "Diagnosis" refers to the process by which a medical professional identifies an illness or condition based on a patient's symptoms and test results.

[1130] "Prescription" refers to a doctor prescribing an appropriate medication to a patient.

[1131] "Pharmacy" means an establishment that prepares and dispenses medicines based on prescriptions.

[1132] "Delivery" refers to the process of getting an item to a designated location.

[1133] "Treatment" refers to medical procedures aimed at improving or recovering from an illness or disability.

[1134] "Follow-up" refers to the process of continually checking the progress of treatment and making any necessary adjustments.

[1135] The present invention is a system for supporting a user's health management, and is implemented using specific procedures, hardware, and software. This system acquires daily biological information and performs data analysis, diagnosis, drug prescription, and follow-up. Specific embodiments of each part are described below.

[1136] Acquisition of biometric information

[1137] User: The user launches the LINE app on their smart device and selects the "Health Management" option. The app activates the camera and displays an interface for measuring heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure their biometric information.

[1138] Device: The camera on the smart device captures and analyzes the user's video data, collecting biometric information such as heart rate, respiratory rate, and body temperature. This data is temporarily stored on the device, converted into a specific format, and then sent to a server.

[1139] Data analysis

[1140] Server: The server receives the transmitted biometric information and analyzes it using a machine learning model. The model used for the analysis is based on past accumulated data and training data. From the analysis results, the server evaluates the user's current health condition and generates a simple diagnosis.

[1141] diagnosis

[1142] Server: The simple diagnosis results are provided to the chatbot, which then asks the user additional questions. The chatbot then interacts with the user to gather more information and sends that data back to the server. The final diagnosis is based on this data.

[1143] Terminal: The terminal sends additional questions to the user through the chatbot and collects the user's answers, thereby improving the accuracy of the diagnosis.

[1144] Prescription of medication

[1145] Server: Based on the final diagnosis, the server generates instructions for the doctor to prescribe the necessary medication. The prescription is sent to a partner pharmacy, which prepares the medication and arranges for delivery.

[1146] User: The user receives the medication at home and takes it as directed.

[1147] Observation

[1148] User: The user can periodically enter the progress of treatment in the LINE app, and can obtain vital data again and send it to the server.

[1149] Server: The server receives the new biometric data and re-analyzes it using machine learning models. The results are then communicated to the user and doctor, and further action is taken if necessary.

[1150] (Example)

[1151] For example, let's say a user feels unwell. In this case, they first activate the camera through the LINE app to measure their heart rate and respiratory rate. The data is sent to a server and analyzed using a machine learning model. Based on the analysis results, the chatbot interviews the user and collects additional information. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data and evaluates the effectiveness of the treatment. This allows users to receive comprehensive health management and medical services from the comfort of their own home.

[1152] (Example of a prompt)

[1153] "Measure your heart rate and breathing rate."

[1154] "Please select the health management option and enter your data."

[1155] "I'll do a quick checkup. Please assess your health."

[1156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1157] Step 1: Obtaining biometric information

[1158] User: The user launches the LINE app and enters health management mode by selecting the "Health Management" option. User action (selecting an option) is required as input.

[1159] Terminal: The terminal activates the smart device's camera and displays the interface. When the user holds their face or hand over the camera, the camera captures video data. Video data is provided as input data, and biometric information such as heart rate, respiratory rate, and body temperature is generated as output.

[1160] How it works: The camera module captures video data, which is then analyzed by internal software to measure biometric information, which is then temporarily stored in the device's memory.

[1161] Step 2: Sending biometric information

[1162] Terminal: The terminal converts the collected biometric data into a predetermined format. This conversion process includes noise removal and data normalization. As input, it requires raw biometric data, and as output, it obtains formatted data.

[1163] The formatted data is then sent to the server using the terminal's network module.

[1164] What it does: Specific algorithms and techniques are used to transform the data into a different format. The optimized data is then sent to the server using a secure protocol.

[1165] Step 3: Analyze the data

[1166] Server: The server receives the transmitted biometric data. It requires formatted biometric data transmitted from the device as input data. This data is then fed into a machine learning model and analyzed to assess the health status.

[1167] How it works: The machine learning model analyzes the data and detects abnormalities, such as heart rate fluctuation patterns or breathing rate. The analysis results are temporarily stored in a database on the server. The analysis results and a simple diagnosis are generated as output.

[1168] Step 4: Simple diagnosis and interview

[1169] Server: The server generates a simple diagnosis based on the analysis results. The input data is the analysis results of the machine learning model. This result is provided to the chatbot, which then begins interviewing the user.

[1170] Terminal: The terminal sends follow-up questions to the user through the chatbot and collects answers from the user. The input data includes the user's answers, and the output provides additional health information.

[1171] Specific actions: For example, specific prompts such as "Please tell us about your recent fatigue and appetite changes" are displayed to the user. These answers are sent to the server in real time.

[1172] Step 5: Sending diagnostic results and prescribing medication

[1173] Server: The server combines the medical interview data and biometric information to make the final diagnosis. The analysis results and medical interview data are required as input data. Based on this, the doctor prescribes the necessary medication and generates a prescription.

[1174] Specific operation: Prescription data is automatically generated and electronically transmitted to affiliated pharmacies. As an output, information on the completed prescription is generated.

[1175] Step 6: Follow-up and evaluation

[1176] User: The user periodically updates the progress of their treatment on the LINE app, and their vital data is acquired again and sent to the server. The input data includes new vital information and the progress of treatment.

[1177] Server: The server reanalyzes the updated information and uses machine learning models to evaluate the effectiveness of the treatment. The input data is the updated biometric information, and the output is the evaluation result of the treatment effectiveness.

[1178] Specific operation: Based on the analysis results, the server provides feedback to the user, such as "Treatment is progressing smoothly" or "Additional tests are required." The results are then notified to the user and their doctor.

[1179] (Application example 1)

[1180] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1181] The working environment for employees at logistics centers is harsh, making health management important. However, it is difficult for employees to properly monitor their own health status and detect abnormalities early amid their busy work schedules. Furthermore, there are limited means to respond quickly when abnormalities are discovered. Given this background, there is a need for an efficient and effective method of managing the health of employees working at logistics centers.

[1182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1183] In this invention, the server includes means for acquiring user biometric information, means for analyzing the biometric information, means for monitoring the biometric information in real time if the user is an employee working at a logistics center, means for making a diagnosis based on the analysis results and past medical information, means for notifying a manager when abnormal biometric information is detected, means for prescribing medication based on the diagnosis results, means for delivering the prescribed medication, and means for monitoring the progress of treatment. This allows for efficient and effective health management of employees at the logistics center and enables rapid response to abnormalities.

[1184] "Users" refer to employees working at the logistics center.

[1185] "Biometric information" is data that indicates the user's health condition, such as heart rate, respiratory rate, and body temperature.

[1186] A "smart device" is an information device such as a smartphone or tablet that is carried by a user.

[1187] A "camera" is an image capture device built into a smart device.

[1188] "Analysis" refers to the process of analyzing collected biometric information using methods such as machine learning models to extract useful information.

[1189] "Diagnosis" refers to the act of evaluating the user's health condition based on the analysis results of biological information and past medical information.

[1190] "Prescription" is the act of a doctor prescribing the necessary medication based on diagnostic results.

[1191] "Delivery" refers to the act of delivering prescribed medicine to a location designated by the user.

[1192] "Follow-up" is the act of monitoring the progress of treatment and continually assessing health status.

[1193] "Administrator" refers to the person in charge of managing the health of employees at the logistics center.

[1194] The present invention is a system for supporting the health management of employees working at a logistics center. This system works in conjunction with smart devices to consistently acquire biometric information, analyze data, diagnose, notify abnormalities, prescribe medication, and monitor treatment progress. Specific embodiments of each part of the system are described below.

[1195] Acquisition of biometric information

[1196] User: The user launches the application on their smart device and holds their face or hand over the camera to measure their biometric information, including heart rate, respiratory rate, and body temperature.

[1197] Terminal: The camera on the smart device analyzes the video data provided by the user and collects biometric information. The collected data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[1198] Data analysis

[1199] Server: The transmitted biometric information is received by the server and analyzed using a machine learning model (for example, a model built with Keras). The server evaluates the user's current health status based on past accumulated data and the trained model. If an abnormality is detected, the analysis results will notify the administrator.

[1200] Diagnostics and Notifications

[1201] Server: Based on the analysis results and past medical information, the server diagnoses the user's health condition. If abnormal vital signs are detected, a notification is sent to the administrator, prompting prompt action.

[1202] Managers: Managers can monitor the health status of employees within the distribution center and take appropriate action if necessary.

[1203] Medication and follow-up

[1204] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and sends it to a partner pharmacy.

[1205] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescribed medicine is delivered to the address specified by the user.

[1206] Users: Users can receive medication at home, allowing them to receive the medical care they need without leaving their homes. Furthermore, they can enter their treatment progress through an application on their smart device.

[1207] Server: The server receives the retransmitted vital signs and analyzes them with a machine learning model to evaluate the effectiveness of treatment. The analysis results are then notified to the user and doctor, and necessary measures are taken.

[1208] Specific examples

[1209] For example, if an employee working at a logistics center feels unwell, they first launch an application on their smart device and hold their face up to the camera to measure their heart rate and respiratory rate. The data is then sent to a server where it is analyzed using a machine learning model. If an abnormality is detected based on the analysis results, an alert is sent to an administrator. The administrator can then take prompt action, requesting a diagnosis from a doctor if necessary and prescribing medication. The user then receives the medication at home and records the progress of their treatment in the application, and the server analyzes the data to continue monitoring their condition.

[1210] Prompt Sentence Examples

[1211] For example, the following prompts could be fed into a generative AI model:

[1212] "We want to build a system in our logistics center that allows employees to monitor their daily health using their smartphones. Using smart devices, we can acquire vital signs (heart rate, respiratory rate, body temperature) and analyze them using machine learning models. If any abnormal values ​​are detected, a notification will be sent to the manager."

[1213] In this way, a system can be realized that comprehensively manages the health of employees working at logistics centers.

[1214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1215] Step 1:

[1216] User: Launches the smart device application and holds their face or hand in front of the camera. The input is video data captured by the smart device's camera. The output is biometric information such as heart rate, respiratory rate, and body temperature.

[1217] Step 2:

[1218] Terminal: Analyzes video data captured by the smart device's camera to collect biometric information. Video data provided by the user is used as input. Analyzed biometric information is obtained as output. Specifically, the smart device's image processing function is used to estimate heart rate and respiratory rate. The data is temporarily stored on the terminal, converted into an appropriate format, and then sent to the server.

[1219] Step 3:

[1220] Server: Receives the transmitted biometric information and analyzes it using a machine learning model (built using Keras). The biometric information sent from the device is used as input. The output is an assessment of the user's current health status. Specifically, the server compares the biometric information with an existing database to check for any abnormalities.

[1221] Step 4:

[1222] Server: Diagnoses the user's health condition based on the analysis results and past medical information. Analysis results and past medical information are used as input. Diagnosis results are obtained as output. Specifically, if an abnormality is detected based on the analysis results, the next step is notification processing.

[1223] Step 5:

[1224] Server: If abnormal biometric information is detected, it sends a notification to the administrator. The diagnosis results are used as input. The notification data to be sent to the administrator is obtained as output. Specifically, when an abnormality is detected, an alert is sent to the administrator's smart device using the LINE API.

[1225] Step 6:

[1226] Administrator: Receives notifications, monitors employee health status in real time, and takes appropriate action. The input is the notification sent from the server. The output is the response procedure to be taken by the administrator. Specifically, the administrator who receives the notification will request a diagnosis from a doctor if necessary, and ensure the safety of the employee.

[1227] Step 7:

[1228] Server: Based on the diagnosis results, the doctor prescribes the necessary medication. The diagnosis results are used as input. A prescription is generated as output. Specifically, the server creates an electronic prescription based on instructions from the doctor and sends it to a partner pharmacy.

[1229] Step 8:

[1230] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prescription is used as input. The output is the medicine to be sent to the user's delivery address. The specific operation is that the pharmacy prepares the prescription and arranges for delivery to the specified address.

[1231] Step 9:

[1232] User: Receives medication at home and performs treatment. The input is the delivered medication. The output is data on the progress of treatment. The specific operation is that the user receives the medication and records the progress of treatment through the application.

[1233] Step 10:

[1234] Server: Receives the progress of treatment and analyzes the vital signs again. The input is the treatment progress data sent by the user. The output is the evaluation result of the treatment effect. Specifically, the server analyzes the new vital signs and checks whether the treatment is effective. The results are notified to the user and the doctor.

[1235] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1236] System Overview

[1237] The present invention is a system that supports users' health management, and by using the LINE app, it is possible to consistently acquire daily biometric information, analyze data, make diagnoses, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of the system are described below.

[1238] Acquisition of biometric information

[1239] User: The user launches the LINE app and selects the "Health Management" option. This activates the camera on the smart device (such as a mobile phone) and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the instructions on the screen and holds their face or hand over the camera to measure the data.

[1240] Device: The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to the server.

[1241] Acquiring emotional information

[1242] Device: The smart device's camera and microphone are used to capture the user's facial expressions and vocal changes and send them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[1243] Data analysis

[1244] Server: The transmitted biometric and emotional information is received by the server and stored in a database. The server analyzes the biometric information using a machine learning model and also incorporates emotional information from the emotion engine into the analysis. This allows for a more accurate assessment of the user's health condition.

[1245] diagnosis

[1246] Server: The analysis results are generated as a simple diagnosis. This simple diagnosis result is provided to the LINE chatbot, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[1247] Device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. The chatbot asks questions taking into account the user's emotional state, allowing the user to respond more naturally.

[1248] Prescription of medication

[1249] Server: Additional information collected through the interview is sent to the server and reanalyzed for a final diagnosis. Based on the analysis results and previous medical information, the doctor makes a final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[1250] Server: The pharmacy prepares the medicine based on the prescription and arranges for delivery. The prepared medicine is then sent to the address specified by the user via a delivery service.

[1251] User: The user receives and takes the medicine at home. They enter information about their treatment, such as progress and side effects, through the LINE app.

[1252] Observation

[1253] User: The user periodically enters the progress of treatment through the LINE app. It is also possible to obtain vital data again and send it to the server.

[1254] Server: The server receives the biometric and emotional information again and analyzes it again using the machine learning model. The analysis results are notified to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The user is also notified of the results and given appropriate instructions.

[1255] Specific examples

[1256] For example, suppose a user feels unwell. In this case, the user first activates the camera through the LINE app and measures their heart rate and breathing rate. This data is sent to the server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the analysis results, the chatbot conducts a medical interview with the user and collects additional information accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is delivered to the user's home from an affiliated pharmacy. The user enters the progress of their treatment through the LINE app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notifies the user of the results. This allows users to receive comprehensive health management and medical care from the comfort of their own home.

[1257] The processing flow will be explained below.

[1258] Step 1:

[1259] User: Launches the LINE app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate and breathing rate. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[1260] Step 2:

[1261] Device: The smart device's camera captures biometric information such as heart rate, respiratory rate, and body temperature from the user's body surface. The captured data is temporarily stored on the device and converted into an appropriate format. This data is then sent to the server.

[1262] Step 3:

[1263] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The video and audio data is sent to the emotion engine, which analyzes them in real time and recognizes the user's emotional state (e.g., joy, sadness, anger, surprise).

[1264] Step 4:

[1265] Server: Receives biometric and emotional information sent from the device and stores it in a database. The server uses a machine learning model to analyze the biometric information and also incorporates emotional information into the analysis. This allows the server to evaluate the user's current health condition and generate a simple diagnosis.

[1266] Step 5:

[1267] Server: Provides the generated simple diagnostic results to the LINE chatbot, which then generates individual questions for the user and begins the medical interview.

[1268] Step 6:

[1269] On the device: Questions from the chatbot are displayed on the chat screen of the LINE app. The user answers the displayed questions and provides additional health information. During this process, the chatbot dynamically adjusts the questions to take into account the user's emotional state.

[1270] Step 7:

[1271] Server: Receives additional information sent from the chatbot and re-analyzes it for a final diagnosis. Based on the analysis results and past medical information, the doctor makes the final diagnosis and generates a diagnosis result.

[1272] Step 8:

[1273] Server: Based on the diagnosis, the doctor prescribes the necessary medication. The server generates a prescription based on the doctor's instructions and electronically sends it to a partner pharmacy.

[1274] Step 9:

[1275] Pharmacy: Prepares the medicine based on the prescription received from the server, and arranges for the medicine to be sent to the address specified by the user via a delivery service.

[1276] Step 10:

[1277] User: The user receives the medication at home. After taking it, they enter information about their treatment progress and side effects through the LINE app. Vital data may also be collected again.

[1278] Step 11:

[1279] Server: Receives the biometric and emotional information entered by the user again and analyzes it using a machine learning model. The analysis results are sent to the doctor to evaluate the effectiveness of the treatment and take necessary measures. The results are also sent to the user, who is given appropriate instructions.

[1280] This system allows users to receive comprehensive health management and medical services from the comfort of their own home. By combining it with an emotion engine, it is possible to provide more accurate diagnoses and personalized medical services.

[1281] Example 2

[1282] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1283] Conventional health management systems lack the ability to incorporate visual and audio emotional information from users, making it difficult to comprehensively assess a user's accurate health and emotional state. This can lead to poor diagnostic accuracy and a poor user experience. Furthermore, the lack of machine learning models to analyze biometric and emotional information limits the accuracy of data analysis. It is desirable to address these issues and realize comprehensive user health management.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1285] In this invention, the server includes a means for acquiring general information about the user, a means for analyzing the general information, and a means for making a diagnosis based on the analysis results and past health information. This enables comprehensive health management for the user. Furthermore, a system is provided that includes a means for acquiring the user's emotions using a camera and microphone of a smart device, a means for analyzing the emotional information, a means for reflecting the analyzed emotional information in the diagnosis results, a means for analyzing the general information and emotional information using a machine learning model, and a means for conducting an interview with the user in an interactive format based on the analysis results. This allows for a more accurate assessment of the user's health status, an appropriate diagnosis and prescription, and more personalized medical services.

[1286] A "user" is an individual who uses the system to manage their own health.

[1287] "Biometric information" refers to various data obtained from the user's body, such as heart rate, respiratory rate, body temperature, and blood pressure.

[1288] "General information" is a blanket term that refers to a variety of data, including biometric and emotional information about the user.

[1289] "Analysis" is the process of evaluating and making judgments on collected data using machine learning models and algorithms.

[1290] "Determination" refers to evaluating the user's health condition based on the analysis results and past health information.

[1291] A "prescription" is an act in which a doctor prescribes necessary medication based on the user's health condition.

[1292] "Emotion information" is data that indicates the psychological state of the user, obtained from facial expressions, voice, etc.

[1293] A "smart device" is an electronic device such as a mobile phone or tablet that includes sensors such as a camera or microphone.

[1294] The "emotion engine" is a system that analyzes captured facial expressions and voice data to recognize the user's emotions.

[1295] A "machine learning model" is an algorithm or computational model that identifies patterns based on large amounts of data and makes predictions and judgments.

[1296] "Dialogue format" refers to a format in which communication with users is interactive, such as through chatbots.

[1297] This invention is a system that supports users' health management, and by using a common messaging app, it is possible to consistently acquire daily biometric information, analyze data, make judgments, prescribe medication, and monitor progress. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to achieve more accurate health management and medical treatment. Specific embodiments of each part of this system will be described.

[1298] Acquisition of biometric information

[1299] A user launches a messaging app and selects the "Health Management" option. This activates the smart device's camera and displays an interface for measuring biometric information such as heart rate, respiratory rate, and body temperature. The user follows the on-screen instructions and holds their face or hand over the camera to measure the data.

[1300] The camera on the smart device captures data such as heart rate, respiratory rate, and body temperature from the user's body surface. The acquired data is temporarily stored on the device, converted into an appropriate format, and then sent to a server.

[1301] Acquiring emotional information

[1302] The device uses the smart device's camera and microphone to capture the user's facial expressions and vocal changes and transmit them to the emotion engine, which analyzes this data and recognizes the user's emotions (e.g., joy, sadness, anger, surprise, etc.).

[1303] Data analysis

[1304] The server receives the transmitted biometric and emotional information and stores it in a database. The server then analyzes the biometric information using a machine learning model and incorporates the emotional information from the emotion engine into the analysis, thereby enabling a more accurate assessment of the user's health condition.

[1305] judgement

[1306] The server generates a simple diagnostic result based on the analysis results. This simple diagnostic result is provided to the chatbot of the messaging app, which then generates additional questions for the user and conducts a medical interview. The interview, which is conducted in an interactive format with the user, dynamically changes its content based on emotional information.

[1307] The chatbot's questions are displayed on the device's chat screen in the messaging app, and the user can respond to them and provide additional health information. The chatbot takes the user's emotional state into account when posing questions, allowing the user to respond more naturally.

[1308] Prescription of medication

[1309] The server analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The doctor makes the final diagnosis and prescribes the necessary medication. The prescription is then electronically sent to a partner pharmacy.

[1310] The pharmacy prepares the medicine based on the prescription and arranges for delivery, which is then sent to the address specified by the user via a delivery service.

[1311] Users receive and take their medication at home, and can enter progress information, including treatment progress and side effects, through a messaging app.

[1312] Observation

[1313] The user periodically enters the progress of treatment through the messaging app, and it is also possible to collect vital data again and send it to the server.

[1314] The server receives the retransmitted biometric and emotional information and reanalyzes it using a machine learning model. The analysis results are then sent to the doctor to evaluate the effectiveness of the treatment, and necessary measures are taken. The user is also notified of the results and given appropriate instructions.

[1315] Specific examples

[1316] For example, if a user feels unwell, they first activate the camera through a messaging app to measure their heart rate and breathing rate. This data is sent to a server and analyzed using a machine learning model. At the same time, the user's facial expressions and voice are analyzed by an emotion engine, and emotional information is also evaluated. Based on the results of this analysis, the chatbot conducts a medical interview with the user, and additional information is collected accordingly. A doctor then makes a final diagnosis remotely and prescribes the necessary medication. The medication is then delivered to the user's home from an affiliated pharmacy. The user then enters the progress of their treatment through the messaging app, and the server analyzes the new data to evaluate the effectiveness of the treatment and notify the user of the results. This enables users to receive comprehensive health management and medical care from the comfort of their own home.

[1317] Prompt sentence for generative AI model

[1318] "Please explain in natural language how a user can manage their health using a messaging app. Please show the flow of the system, including specific actions, from obtaining biometric and emotional information, to making a judgment, prescribing medication, and monitoring progress."

[1319] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1320] Step 1: Launch your messaging app

[1321] User: Launches a messaging app and selects the "Health Management" option. The input is a user action, and the output is the app switching to "Health Management" mode. Specific actions include the user operating the smartphone screen and tapping the health management option within the app.

[1322] Step 2: Measuring biometric data

[1323] Terminal: The smart device's camera activates and displays an interface for measuring heart rate, respiratory rate, and body temperature. The input is the user's face or hand, and the output is biometric data. Specific operations include the device's camera using facial recognition technology to capture the user's face or hand and measure biometric information.

[1324] Step 3: Storing and sending data

[1325] Terminal: The acquired biometric data is temporarily stored on the terminal, converted into an appropriate format, and sent to the server. The input is the measured biometric data, and the output is the format-converted data sent to the server. Specific operations include the terminal converting the measured data into JSON format, etc., and sending the data to the server via the HTTPS protocol.

[1326] Step 4: Capturing emotional information

[1327] Device: The smart device's camera and microphone are used to capture the user's facial expressions and voice changes. The input is the user's facial expressions and voice, and the output is the captured emotion data. Specifically, the device's camera analyzes the user's facial expressions using a facial expression recognition algorithm, and the microphone records voice data.

[1328] Step 5: Sending emotional information

[1329] Device: Sends captured data to the emotion engine. The input is the captured emotion data, and the output is the data sent to the emotion engine. Specific operations include the device sending facial expression and voice data to the emotion engine's analysis API.

[1330] Step 6: Receiving and storing data

[1331] Server: Receives the transmitted biometric and emotional information and stores it in a database. The input is the data transmitted from the device, and the output is the data stored in the database. Specifically, the server receives the data at the API endpoint where it receives data and stores it in an SQL database.

[1332] Step 7: Analysis using machine learning models

[1333] Server: Analyzes biometric and emotional information using a machine learning model. The input is the stored biometric and emotional information, and the output is the analysis results. Specific operations include the server analyzing the data using Python's Scikit-learn or TensorFlow to evaluate the user's health status.

[1334] Step 8: Generate quick diagnostic results

[1335] Server: Generates a simple diagnostic result based on the analysis results. The input is the analysis result from the machine learning model, and the output is a simple diagnostic result. Specifically, the server uses a specialized rule-based algorithm to output the diagnostic result from the analysis results.

[1336] Step 9: Chatbot consultation

[1337] Server: Provides the generated simple diagnosis results to the chatbot of the messaging app, which then generates additional questions for the user. The input is the simple diagnosis results, and the output is the additional questions sent to the user. Specifically, the server sends data to the chatbot using the messaging API, and the chatbot then sends dynamically generated questions to the user.

[1338] Terminal: Questions from the chatbot are displayed on the chat screen of the messaging app. The input is the question from the chatbot, and the output is the user's answer. Specific actions include the user opening the chat screen on their smartphone and answering the displayed question by typing.

[1339] Step 10: Generate a final diagnosis and prescription

[1340] Server: Analyzes the additional information collected through the medical interview and makes a final diagnosis based on past health information. The input is the medical interview results and past health information, and the output is the final diagnosis and prescription data. Specific operations include a remote doctor on the server using the electronic diagnostic system to confirm the final diagnosis and create an electronic prescription.

[1341] Step 11: Submit your prescription

[1342] Server: Electronically transmits prescriptions to affiliated pharmacies. The input is prescription data, and the output is the prescription sent to the pharmacy. Specifically, the server transmits data to the pharmacy's electronic prescription system.

[1343] Step 12: Prepare and deliver medication

[1344] Pharmacy: Prepares medicine based on prescription and arranges for delivery. The input is the received prescription data, and the output is the medicine arranged for delivery. Specifically, the pharmacy prepares the medicine, provides the information to the delivery service, and delivers it to the user.

[1345] User: Receives and takes medicine. The input is the delivered medicine, and the output is progress information after taking it. The specific operation is for the user to receive the delivered medicine and use it according to the dosage instructions.

[1346] Step 13: Enter progress information

[1347] User: Enters treatment progress and side effects into a messaging app. The input is treatment progress information, and the output is the progress information sent to the server. Specifically, the user enters progress information using a form within the app and presses the send button.

[1348] Step 14: Reanalyze the data

[1349] Server: Analyzes the retransmitted biometric and emotional information. The input is the newly transmitted biometric and emotional information, and the output is the reanalysis results. Specifically, the server receives the new data and analyzes it again using the machine learning model.

[1350] Step 15: Notification of results

[1351] Server: Notifies the doctor of the analysis results and takes necessary measures. The input is the reanalysis results, and the output is notification data for the doctor and a result notification for the user. Specifically, the server sends the analysis results to the doctor via email or a notification system, and notifies the user of the results via a messaging app.

[1352] (Application example 2)

[1353] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1354] Conventional health management systems analyze biometric information acquired through the camera and microphone of a user's smart device and can provide a diagnosis and prescription based on the results. However, these systems do not support the use of biometric measurement results to recommend health-related products, deliver those products, or monitor their effectiveness. As a result, they do not adequately provide a means for users to use appropriate health-related products based on their own health status and continuously improve their health. Furthermore, it is difficult to track the effectiveness of products and make optimal recommendations for individual users.

[1355] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1356] In this invention, the server includes a means for acquiring biometric information of a user, a means for analyzing the biometric information, a means for making a diagnosis based on the analysis results and past medical information, a means for prescribing a prescription based on the diagnosis results, a means for presenting the prescribed health-related product, a means for delivering the health-related product, and a means for monitoring the effects. This makes it possible to analyze the user's biometric information and emotional information, and based on this, to propose and deliver the most appropriate health-related product, and further monitor the effects over time.

[1357] "User" refers to an individual who uses this system.

[1358] "Biometric information" refers to data about a user's basic bodily functions, such as temperature, heart rate, and breathing rate.

[1359] "Analysis" refers to data processing to evaluate and diagnose health status based on acquired biometric information.

[1360] "Diagnosis" refers to assessing the user's health condition and determining necessary treatment and measures based on analysis results and past medical information.

[1361] "Prescription" refers to the act of providing needed medicines or health-related products based on diagnostic results.

[1362] "Health-related products" refers to supplements, exercise equipment, health foods, etc. used to maintain or improve the user's health.

[1363] "Delivery" refers to the delivery of health-related products selected or prescribed by the user to the address specified by the user.

[1364] "Follow-up" refers to the ongoing monitoring and evaluation of a user's health status after using a prescribed product.

[1365] This system acquires a user's biometric and emotional information, and based on that information, diagnoses their health, recommends and delivers health-related products, and monitors their effectiveness. This system is implemented using smart devices, a server, a machine learning model, and an emotion analysis engine.

[1366] System Overview

[1367] The system can be broadly divided into three parts: a part that acquires the user's biometric information, a part that analyzes the biometric and emotional information, a part that suggests and delivers health-related products based on the analysis results, and finally a part that monitors the effects over time.

[1368] Acquiring user's biometric information

[1369] The user launches the smartphone application and uses the camera and microphone to measure their heart rate, respiratory rate, body temperature, facial expressions, and voice. Specifically, image processing software such as OpenCV is used to extract biometric information from the camera footage, and emotional information is analyzed from the voice and facial expression data using software such as EmotionAPI.

[1370] Data analysis

[1371] The acquired biometric and emotional information is sent from the device to a server, which then analyzes the data using machine learning models such as TensorFlow. The analysis results are then compared with past medical information to assess the user's health status.

[1372] Proposal and delivery of health-related products

[1373] Based on the diagnosis results, appropriate health-related products are recommended to the user. These products can be purchased directly through the application and delivered to the specified address. Product recommendations are made in conjunction with a health database that is collated with the analysis results.

[1374] Follow-up of effects

[1375] Even after a user has purchased or taken a prescribed product, biometric and emotional information is periodically collected again to monitor progress. The collected data is then sent to the server for further analysis. Based on the results, new recommendations or adjustments are made.

[1376] Specific examples

[1377] For example, a user opens the app every morning and checks their health using the camera and microphone. Based on the results, the app diagnoses a vitamin D deficiency and recommends vitamin supplements. The user purchases the supplements and checks again a week later. The app also collects this data and uses it to make future recommendations.

[1378] Prompt Sentence Examples

[1379] To assess the user's health, perform the following steps:

[1380] 1. Turn on the camera to measure your heart rate, breathing rate, and temperature.

[1381] 2. Next, the microphone captures the voice and recognizes facial expressions.

[1382] 3. Based on the analysis results, we recommend appropriate health-related products.

[1383] For example, if you have a vitamin D deficiency, you might see a push notification like this:

[1384] "I suggest a vitamin D supplement. And how about some exercise equipment along with that?"

[1385] In this way, the present invention can provide comprehensive support for the user's health management.

[1386] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1387] Step 1: User biometric information acquisition

[1388] The user launches a smartphone application, which uses the camera and microphone to measure their heart rate, respiration rate, body temperature, facial expressions, and voice. Specifically, the smartphone camera takes a picture of their face, and the heart rate and respiration rate are analyzed using image processing software such as OpenCV. The microphone also captures their voice, and emotional information is analyzed using EmotionAPI or similar. The input at this stage is the user's face image and voice data, and the output is the analyzed heart rate, respiration rate, body temperature, and emotional information.

[1389] Step 2: Sending data to the server

[1390] The device sends the acquired and analyzed biometric and emotional information to the server. Specifically, the data is formatted in JSON format or similar and sent to the server using a secure protocol (e.g., HTTPS). The input is the analyzed biometric and emotional information, and the output is the data sent to the server.

[1391] Step 3: Data analysis by the server

[1392] The server analyzes the received biometric and emotional information using machine learning models such as TensorFlow. Past medical information is also collated during the analysis. Specifically, past medical information is retrieved from a database, and the user's health status is evaluated as the analysis result. The input is the transmitted biometric, emotional, and past medical information, and the output is the diagnosis result.

[1393] Step 4: Proposing health-related products

[1394] The server proposes appropriate health-related products based on the analysis and diagnosis results. It compares the analyzed data with a database of health-related products to select the most suitable products for the user. The proposals are notified to the user through the application. The input is the diagnosis results and a database of health-related products, and the output is a list of proposed products.

[1395] Step 5: Purchase and shipping

[1396] The user selects the desired product from the list of suggested products and completes the purchase process through the application. The server receives the purchase information and initiates the delivery process. The input is the user's purchase selection and delivery address information, and the output is product delivery instructions. The product delivery instructions are sent to the delivery company.

[1397] Step 6: Monitor the results

[1398] The user uses the purchased health-related product and periodically re-acquires biometric and emotional information. The device again transmits the newly acquired data to the server. The server again analyzes the data using a machine learning model and evaluates the effectiveness. If necessary, it provides the user with appropriate instructions or the next suggestion. The input is the newly acquired biometric and emotional information, and the output is the evaluation of the effectiveness and new suggestions.

[1399] Through the above steps, a system that comprehensively supports the user's health management is realized.

[1400] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1401] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1402] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1403] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1404] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1405] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1406] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1407] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1408] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1409] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1410] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1411] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1412] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1413] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1414] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1415] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1416] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1417] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1418] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1419] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1420] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1421] The following is further disclosed regarding the above embodiment.

[1422] (Claim 1)

[1423] means for acquiring biometric information of a user;

[1424] means for analyzing the biological information;

[1425] A means for making a diagnosis based on the analysis results and past medical information;

[1426] A means for prescribing based on the diagnosis result;

[1427] A system comprising means for delivering the prescribed medication and means for monitoring the progress of treatment.

[1428] (Claim 2)

[1429] 10. The system of claim 1, wherein the acquired biometric information is acquired using a camera of a smart device.

[1430] (Claim 3)

[1431] 2. The system according to claim 1, further comprising means for interactively asking a question to a user based on the analysis result.

[1432] (Claim 4)

[1433] The system of claim 1, further comprising means for analyzing the effect of the treatment using a machine learning model.

[1434] (Claim 5)

[1435] 10. The system of claim 1, wherein the prescribed medication is delivered to an address specified by the user.

[1436] "Example 1"

[1437] (Claim 1)

[1438] means for acquiring biometric information of a user;

[1439] means for analyzing the biological information;

[1440] A means for making a diagnosis based on the analysis results and past medical information;

[1441] A means for prescribing based on the diagnosis result;

[1442] a means for delivering the prescribed medication;

[1443] a means of monitoring the progress of treatment;

[1444] a means for activating a camera of a smart device based on a health management option selected by the user, and measuring biological information;

[1445] means for converting the biometric information into a predetermined format and transmitting the same to a server;

[1446] a means for analyzing biometric information using a machine learning model;

[1447] A means for conducting a medical interview in an interactive format with a user using a chatbot;

[1448] A system including:

[1449] (Claim 2)

[1450] 2. The system according to claim 1, wherein the acquired biometric information is acquired using a camera of a smart device, converted into a predetermined format, and then transmitted to a server.

[1451] (Claim 3)

[1452] The system according to claim 1, further comprising means for evaluating the user's health condition using a machine learning model based on the analysis results, and for conducting an interview in an interactive format using a chatbot.

[1453] "Application Example 1"

[1454] (Claim 1)

[1455] means for acquiring biometric information of a user;

[1456] means for analyzing the biological information;

[1457] means for monitoring biometric information in real time when the user is an employee working at a logistics center;

[1458] A means for making a diagnosis based on the analysis results and past medical information;

[1459] a means for notifying an administrator when abnormal biometric information is detected;

[1460] A means for prescribing based on the diagnosis result;

[1461] A system comprising means for delivering the prescribed medication and means for monitoring the progress of treatment.

[1462] (Claim 2)

[1463] 10. The system of claim 1, wherein the acquired biometric informati...

Claims

1. means for acquiring biometric information of a user; means for analyzing the biological information; A means for making a diagnosis based on the analysis results and past medical information; A means for prescribing based on the diagnosis result; A system comprising means for delivering the prescribed medication and means for monitoring the progress of treatment.

2. The system of claim 1 , wherein the captured biometric information is captured using a camera on a smart device.

3. 2. The system according to claim 1, further comprising means for interactively asking a user questions based on the analysis results.

4. The system of claim 1 , further comprising means for analyzing the effect of the treatment using a machine learning model.

5. 10. The system of claim 1, wherein the prescribed medication is delivered to an address specified by the user.

Images