System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a question about a user's health; means for sending the question to a server hosting a generative artificial intelligence model; means for analyzing the question on the server and generating appropriate health information or advice based on the question; means for sending the generated information or advice back to the user's terminal; and means for displaying the information or advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] This invention relates to technology that addresses issues such as reducing hospital waiting times, shortages of medical staff resources, and patient self-care support in an era where the need for telemedicine and home medical care is increasing. Conventional methods have difficulty efficiently resolving these issues and impose a heavy burden on patients and medical staff. Specific issues include providing prompt and appropriate health information, improving the efficiency of medical device monitoring, and smoothly supporting home medical care. [Means for solving the problem]

[0005] The present invention solves the above problems using the following means. First, a system is provided that includes a means for inputting a user's health-related question, a means for sending the question to a server hosting a generative artificial intelligence model, a means for analyzing the question on the server and generating appropriate health information or advice, and a means for returning and displaying the generated information or advice to the user's terminal. Second, a system is provided that includes a means for collecting data from medical devices in real time and sending it to a server, and a means for analyzing the data using a generative artificial intelligence model on the server, detecting abnormalities, generating appropriate usage instructions or warning messages, and sending them to a user's or medical staff's terminal for display. Third, a system is provided that includes a means for a user to input a home medical care contact request, a means for sending the contact request to a server, and a means for analyzing the contact request and user information using a generative artificial intelligence model on the server, identifying appropriate medical staff, generating a link or method for establishing contact, and displaying the generated information on the user's terminal. This solves various problems that arise in the fields of telemedicine and home medical care, and enables the provision of efficient and effective medical services.

[0006] A "user" is an individual who utilizes the system to enter health-related questions and receive information and advice.

[0007] A "terminal" is a device used by a user (such as a smartphone, tablet, or PC) that inputs questions and displays received information.

[0008] A "server" is a computer system that runs a generative artificial intelligence model, analyzes data received from users, and generates appropriate information.

[0009] A "generative artificial intelligence model" is an algorithm or program that uses natural language processing technology to analyze input questions and data and generate appropriate answers and information.

[0010] "Health information" refers to general health advice and recommendations generated by the generative artificial intelligence model based on the analysis results.

[0011] A "medical device" is a device (e.g., heart rate monitor, blood pressure monitor, etc.) that collects patient health data and transmits it to a server in real time.

[0012] "Data" means health-related information collected by a medical device (e.g., heart rate, blood pressure, temperature, etc.).

[0013] "Analysis" is the process by which a generative artificial intelligence model evaluates the content of received questions and data through syntactic and semantic analysis to understand their meaning.

[0014] An "anomaly" is a value or pattern in the data collected by a medical device that falls outside the normal range, and is what the generative artificial intelligence model detects and generates an alert.

[0015] A "warning message" is a message that is generated by a generative artificial intelligence model when it detects an abnormality, and which serves to alert users and prompt them to take action.

[0016] "Home medical care" is a medical service in which patients receive medical care at home and can communicate with medical staff remotely.

[0017] A "contact request" is a request that a user receiving home medical care inputs into the system to the effect that they wish to contact medical staff.

[0018] "Medical staff" refers to professionals (e.g., doctors, nurses, etc.) who provide medical services to patients.

[0019] A "link" is a URL or contact method that a user uses to establish contact with medical staff. [Brief explanation of the drawings]

[0020] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system involves the user, terminal, and server working together to answer questions about the user's health, monitor medical equipment, and provide smooth support for home medical care.

[0042] Health consultation and information system

[0043] overview

[0044] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[0045] Specific examples

[0046] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[0047] Medical device monitoring system

[0048] overview

[0049] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[0050] Specific examples

[0051] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[0052] Home medical support system

[0053] overview

[0054] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff, and the server then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal.

[0055] Specific examples

[0056] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The terminal then displays this message to the user.

[0057] In this way, the present invention provides an efficient and effective medical support system that utilizes generative artificial intelligence models to solve various problems in remote medical care and home medical care. By having each module work in cooperation with each other, it is possible to improve user convenience and enhance the quality of medical care.

[0058] The processing flow will be explained below.

[0059] Health consultation and information system

[0060] Processing steps and specific operations

[0061] Step 1:

[0062] The user enters a health question into the terminal.

[0063] Specific behavior: The user types "I've been having trouble sleeping lately. Is there anything I can do?" into the device's input field.

[0064] Step 2:

[0065] The terminal sends the entered question to the server.

[0066] Specific operation: The device converts the entered question into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[0067] Step 3:

[0068] The server parses the received query.

[0069] Specific operation: The server inputs the question data into a generative artificial intelligence model, performs syntactic and semantic analysis, and clarifies the intent of the question.

[0070] Step 4:

[0071] The server generates appropriate health information or advice.

[0072] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[0073] Step 5:

[0074] The server generates a response and sends it back to the terminal.

[0075] Specific operation: The server returns the generated answer in JSON format to the terminal.

[0076] Step 6:

[0077] The terminal displays the received answer to the user.

[0078] Specific behavior: The device displays the answer it receives in the user interface, saying, "To create a relaxing routine, it's a good idea to create a good sleep environment."

[0079] Medical device monitoring system

[0080] Processing steps and specific operations

[0081] Step 1:

[0082] The medical device transmits data to the server in real time.

[0083] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[0084] Step 2:

[0085] The server parses the received data.

[0086] How it works: The server inputs data into a generative artificial intelligence model to detect abnormal values ​​and patterns.

[0087] Step 3:

[0088] The server evaluates the state based on the data.

[0089] Specific operation: The generative artificial intelligence model evaluates the patient's condition based on the analysis results and determines whether there are any abnormalities.

[0090] Step 4:

[0091] The server generates appropriate usage and warning messages.

[0092] Specific operation: If the server detects an abnormality, it generates a warning message such as, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[0093] Step 5:

[0094] The server sends the generated message to the terminal.

[0095] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[0096] Step 6:

[0097] The terminal displays the received message to the user.

[0098] Specific behavior: The device receives a message and displays it on the user interface, saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[0099] Home medical support system

[0100] Processing steps and specific operations

[0101] Step 1:

[0102] A user enters a home health care contact request.

[0103] Specific action: The user types into the device's input field, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[0104] Step 2:

[0105] The terminal sends a contact request to the server.

[0106] Specific operation: The terminal sends the entered contact request in JSON format to the server.

[0107] Step 3:

[0108] The server parses the contact request and the user information.

[0109] Specific operation: The server uses a generative artificial intelligence model to analyze the contact request and user information and evaluate the user's condition and symptoms.

[0110] Step 4:

[0111] The server identifies the appropriate medical staff.

[0112] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[0113] Step 5:

[0114] The server creates a link or method for establishing contact.

[0115] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[0116] Step 6:

[0117] The terminal displays the received contact information to the user.

[0118] Specific operation: The device displays the generated contact information in the user interface, displaying "Dr. Suzuki is available. Click here to contact him."

[0119] The above is a description of the specific processing steps and operations of each system.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] The current medical environment lacks a system that effectively supports telemedicine and home medical care. In particular, systems that can provide appropriate information and alerts in real time are needed for health consultations, medical device monitoring, and home medical care communications. This will improve convenience for patients and medical professionals and enable prompt and appropriate responses.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes a means for analyzing a user's health-related questions and generating appropriate health information or advice, a means for analyzing data from medical devices and detecting abnormalities, and a means for analyzing contact requests and user information and identifying appropriate medical professionals, thereby enabling a quick response to the user's health consultation, real-time detection of abnormalities in medical devices, and quick establishment of contact with appropriate medical professionals.

[0125] "User" refers to an individual or patient who uses the system to receive health consultations, monitor medical devices, or contact home medical care.

[0126] A "question" is a question that a user inputs about a health-related question or symptom and requests an answer.

[0127] A "generative artificial intelligence model" is an artificial intelligence system that analyzes input text and generates appropriate information and advice.

[0128] A "server" is a computer system that hosts a generative artificial intelligence model, analyzes questions sent by users and medical device data, and generates and returns appropriate information.

[0129] A "terminal" is a hardware device used by a user, such as a smartphone, tablet, or PC.

[0130] "Health information or advice" refers to information or recommended actions provided by a generative artificial intelligence model in response to a health consultation.

[0131] "Medical devices" are devices used to monitor a patient's health, such as heart rate monitors and blood pressure monitors.

[0132] "Data" refers to the patient's vital signs and measurement results collected in real time by medical devices.

[0133] An "abnormality" is a phenomenon that is determined as a measurement value or state outside the normal range as a result of the generative artificial intelligence model analyzing data.

[0134] A "contact request" is a request by a user to contact a medical professional regarding home medical care.

[0135] "Medical professionals" are professionals such as doctors and nurses who provide medical services and advice to users.

[0136] A "link or method" is a communication method or URL generated to allow a user to contact the appropriate healthcare professional.

[0137] A "warning message" is the warning message provided by the generative artificial intelligence model when an abnormality in a medical device is detected.

[0138] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system operates in cooperation with users, terminals, and servers to answer users' health-related questions, monitor medical equipment, and provide smooth support for home medical care.

[0139] Hardware and software used

[0140] Hardware: User devices (smartphones, tablets, PCs), medical devices (heart rate monitors, blood pressure monitors, etc.)

[0141] Software: Generative AI model on the server (e.g., GPT-3, BERT, etc.), communication protocol (HTTP / S)

[0142] Health consultation and information system

[0143] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[0144] Specific examples

[0145] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[0146] Medical device monitoring system

[0147] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[0148] Specific examples

[0149] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[0150] Home medical support system

[0151] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff. The server then generates links and instructions for establishing contact with the medical staff and displays them on the user's terminal.

[0152] Specific examples

[0153] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Medical Professional A is available. Please click here to contact him." The terminal then displays this message to the user.

[0154] Prompt Sentence Examples

[0155] "Please tell me what to do about not being able to sleep."

[0156] What should I do if my heart rate is high?

[0157] "I'm feeling nauseous, which doctor should I consult?"

[0158] As a result, the present invention utilizes generative artificial intelligence models to efficiently solve various issues in remote medical care and home medical care, providing an effective medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Health consultation and information system

[0161] Processing Steps

[0162] Step 1:

[0163] The user uses the terminal to input health-related questions.

[0164] Input: The text question entered by the user into the terminal

[0165] Output: The device is ready to send the question data to the server.

[0166] Step 2:

[0167] The terminal transmits the entered question to the server.

[0168] Input: Question data from the user

[0169] Output: Query data arrives at the server

[0170] Step 3:

[0171] The server passes the received question to a generative AI model and analyzes the content.

[0172] Input: Query data arriving at the server

[0173] Output: The results of the generative AI model's analysis of the question

[0174] Step 4:

[0175] The server generates appropriate health information and advice based on the analysis results returned by the generative AI model.

[0176] Input: Question content analyzed by the generative AI model

[0177] Output: Health information or advice

[0178] Step 5:

[0179] The server sends the generated response to the terminal.

[0180] Input: Generated health information or advice

[0181] Output: The device receives the data

[0182] Step 6:

[0183] The terminal displays the received response to the user.

[0184] Input: Health information or advice received from the server

[0185] Output: A screen display where users can view their answers

[0186] Medical device monitoring system

[0187] Processing Steps

[0188] Step 1:

[0189] Medical devices collect data in real time.

[0190] Input: Patient's vital signs (e.g., heart rate)

[0191] Output: Collected data

[0192] Step 2:

[0193] The medical device sends the collected data to a server.

[0194] Input: Data from medical devices

[0195] Output: Data arrives at the server

[0196] Step 3:

[0197] The server analyzes the received data using the generated AI model and checks for any abnormalities.

[0198] Input: Data arriving at the server

[0199] Output: Analysis results of the generative AI model (normal or abnormal)

[0200] Step 4:

[0201] If the server detects an abnormality, it will generate appropriate usage or warning messages.

[0202] Input: Analysis results of the generative AI model (in case of anomalies)

[0203] Output: Warning message

[0204] Step 5:

[0205] The server sends the generated warning message to the terminal of the user or medical staff.

[0206] Input: warning message

[0207] Output: The device receives the data

[0208] Step 6:

[0209] The terminal displays the received warning message to the user.

[0210] Input: The warning message received from the server

[0211] Output: A screen display where the user can view the warning

[0212] Home medical support system

[0213] Processing Steps

[0214] Step 1:

[0215] The user inputs a contact request for home medical care from a terminal.

[0216] Input: Text information entered by the user into the device

[0217] Output: The device is ready to send contact request data to the server

[0218] Step 2:

[0219] The terminal sends a contact request to the server.

[0220] Input: Contact request data from the user

[0221] Output: Contact request data arrives at the server

[0222] Step 3:

[0223] The server uses a generative AI model to analyze the contact request and user information to identify appropriate medical staff.

[0224] Input: Contact request data and user information received by the server

[0225] Output: Identification of appropriate medical staff

[0226] Step 4:

[0227] The server creates links and methods for establishing contact with medical staff.

[0228] Input: Identification of appropriate medical staff

[0229] Output: Means of contact (link or method)

[0230] Step 5:

[0231] The server sends the generated contact information to the user's terminal.

[0232] Input: Contact Method

[0233] Output: The device receives the data

[0234] Step 6:

[0235] The terminal displays the received contact information to the user.

[0236] Input: Contact information received from the server

[0237] Output: A screen display where the user can view contact information

[0238] Through these steps, the system can quickly respond to user questions and contact requests and monitor medical devices in real time.

[0239] (Application example 1)

[0240] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0241] The present invention relates to a system for more efficient and rapid health management using smart devices. Its purpose is to provide a means for users to quickly and easily ask health-related questions and receive appropriate advice while on the go. It also aims to provide a comprehensive telemedicine system that also includes medical equipment monitoring and home medical care support.

[0242] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0243] In this invention, the server includes means for receiving and analyzing voice input from the smart glasses, means for sending questions to a generative artificial intelligence model on the server and obtaining answers, and means for displaying the obtained answers on the display of the smart glasses, thereby enabling users to ask health-related questions in real time and receive appropriate advice even when they are out and about.

[0244] "User" means an individual or end user who uses the system.

[0245] "Health-related questions" refer to questions or concerns that a user may have about their own health condition or medical care.

[0246] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data from users and generates appropriate information and answers.

[0247] A "server" is a back-end system that hosts generative artificial intelligence models and processes data submitted by users.

[0248] "Terminal" refers to a device, such as smart glasses, a smartphone, a tablet, etc., that a user uses to send and receive input data.

[0249] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand the questions and data sent by the user and generate appropriate information.

[0250] "Smart glasses" are wearable devices that have a built-in display and can receive user input via voice or vision.

[0251] "Real-time" means that data is generated and analyzed almost simultaneously with user input.

[0252] "Medical device" refers to a hardware device used to monitor a user's physical condition, such as a heart rate monitor or blood pressure monitor.

[0253] A "display" is a display device for visually displaying generated information and answers.

[0254] A "link" is a means or method by which a user can easily contact designated medical personnel.

[0255] "Contact Request" means a notification or instruction for a User to seek home health care assistance.

[0256] The present invention provides a system that allows users to use smart glasses to ask health-related questions in real time and obtain appropriate health information and advice through a generative artificial intelligence model. Detailed embodiments for implementing this system are described below.

[0257] The system mainly consists of three main components: users, terminals, and servers.

[0258] 1. Users

[0259] As an end user of the system, a user wears smart glasses. They input health-related questions by voice, and the input is received by the microphone of the smart glasses. For example, they might ask, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?"

[0260] 2. Terminal (smart glasses)

[0261] The smart glasses have the following features:

[0262] Acquiring voice input: The smart glasses acquire the user's voice questions through a microphone.

[0263] Speech-to-text conversion: Convert the captured speech into text and send it to the server. This is done using a speech recognition library (for example, Python's speech_recognition library).

[0264] Communication with the server: The user's question is sent to the server and the answer is received from the server.

[0265] Display function: The answer received from the server is displayed on the smart glasses' display, and at the same time, feedback is given to the user via voice.

[0266] 3. Server

[0267] The server hosts the generative artificial intelligence model and has the following functions:

[0268] Question analysis: The text question sent by the user is analyzed and an appropriate answer is generated using a generative artificial intelligence model (e.g., GPT-3). The prompt sentence is the user's question, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?"

[0269] Answer generation: The generative artificial intelligence model generates appropriate health information and advice based on the analysis results and sends it back to the device.

[0270] Specific examples

[0271] In a specific usage scenario, a user wearing smart glasses asks a question while out and about: "If I consume caffeine before bed, I have trouble falling asleep. What should I do?" This question is picked up by the smart glasses' microphone and converted into text using a speech recognition library. The text question is sent to a server, where a generative artificial intelligence model generates an answer such as, "It's effective to avoid consuming caffeine before bed and to create a relaxing bedtime routine." The generated answer is sent back to the smart glasses and displayed on the user's display.

[0272] This allows users to solve health-related questions in real time and receive appropriate advice while on the go. Medical equipment monitoring and home medical care support are also analyzed on the server, and the necessary information is provided to users and medical staff.

[0273] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0274] Step 1:

[0275] The user puts on the smart glasses and speaks to ask a health-related question. The voice input is in the form of, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?" This voice data is picked up by the microphone device in the smart glasses.

[0276] Step 2:

[0277] The device (smart glasses) converts the acquired voice data into text data. This conversion is performed using a voice recognition library (for example, Python's speech_recognition library). The input is voice data, and the output is text data.

[0278] Step 3:

[0279] The terminal sends the user's question to the server in the form of text data. Here, the terminal sends the text data to the specified endpoint of the server using an HTTP POST request. The input is the text data, and the output is the request sent to the server.

[0280] Step 4:

[0281] The server passes the received text data to a generative artificial intelligence model and begins analyzing the question. This analysis is input to the model as a prompt sentence, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?" The input is text data (prompt sentence), and the output is answer data based on the analysis results.

[0282] Step 5:

[0283] A generative artificial intelligence model (e.g., GPT-3) analyzes the question and generates an appropriate answer. An example answer might be, "Headaches are a common side effect of medication, so we recommend consulting your doctor. Also, drink plenty of fluids and try to relax as much as possible." The input is the prompt, and the output is the generated answer data.

[0284] Step 6:

[0285] The server returns the answer data obtained from the generative AI model to the terminal. This process is performed as an HTTP response. The input is the generated answer data, and the output is the transmission of the response.

[0286] Step 7:

[0287] The terminal displays the received answer data on the display of the smart glasses. In addition, it also provides audio feedback to the user using a voice output library (e.g., Python's pyttsx3). The input is the answer data from the server, and the output is the display and audio output.

[0288] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0289] This invention relates to a telemedicine and home medical care support system that integrates a generative artificial intelligence model and an emotion engine. This system involves the user, terminal, and server working together to answer the user's health-related questions, monitor medical equipment, and provide seamless support for home medical care. This system also recognizes the user's emotional state and reflects this in its analysis and information provision, enabling more personalized medical support.

[0290] Health consultation and information system

[0291] overview

[0292] The user enters a health-related question into the device, and the question along with the user's emotional data detected by the emotion engine is sent to the server. A generative AI model on the server analyzes the question and emotional data and generates appropriate health information and advice. The generated information is sent back to the device and displayed to the user in a form adjusted based on the emotional data.

[0293] Specific examples

[0294] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects the user's level of stress from their facial expressions and tone of voice. The device then sends this question and emotional data to the server. The server uses a generative artificial intelligence model to analyze the question and emotional data and generate advice such as, "To create a good sleep environment, it is effective to limit your intake of caffeine and alcohol before bed and create a relaxing routine." The device then displays this advice to the user in a relaxed tone that takes into account the user's level of stress.

[0295] Medical device monitoring system

[0296] overview

[0297] Medical devices send data collected in real time to a server, where an emotion engine grasps the user's emotional state. A generative AI model on the server analyzes the data and emotion data, and generates appropriate usage instructions and warning messages if an abnormality is detected. This information is adjusted based on the emotion data and sent to and displayed on the user's or medical staff's device.

[0298] Specific examples

[0299] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. An emotion engine recognizes the user's current emotional state as stress. The server uses a generative artificial intelligence model to analyze the data and emotional information and generate a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is displayed to the user in a calm and gentle tone that takes into account the stressful state.

[0300] Home medical support system

[0301] overview

[0302] A user inputs a contact request for home medical care from a terminal, and the request and emotional data detected by the emotion engine are sent to the server. A generative artificial intelligence model on the server analyzes the contact request, emotional data, and user information to identify appropriate medical staff. It then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal in a format adjusted based on the emotional data.

[0303] Specific examples

[0304] The user types into the device, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to a doctor." The emotion engine detects anxiety from the user's facial expression. The device then sends this contact request and emotion data to the server. The server uses a generative artificial intelligence model to analyze the contact request and emotion information, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The device then displays this message to the user in a reassuring tone that takes into account the user's state of anxiety.

[0305] In this way, by integrating an emotion engine and a generative artificial intelligence model, this invention solves various issues in remote medical care and home medical care, and provides a personalized and efficient medical support system. By each module working together, it is possible to improve user convenience and enhance the quality of medical care.

[0306] The processing flow will be explained below.

[0307] Health consultation and information system

[0308] Processing steps and specific operations

[0309] Step 1:

[0310] The user enters a health question into the terminal.

[0311] Specific operation: The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?"

[0312] Step 2:

[0313] An emotion engine recognizes the user's emotional state.

[0314] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's stress level from their facial expressions and tone of voice.

[0315] Step 3:

[0316] The device sends the question and emotion data to the server.

[0317] Specific operation: The device converts the question and emotion data into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[0318] Step 4:

[0319] The server analyzes the question and sentiment data.

[0320] Specific operation: The server inputs the question and emotion data into the generative AI model, and performs syntactic and semantic analysis to clarify the intent and emotional state of the question.

[0321] Step 5:

[0322] The server generates appropriate health information or advice.

[0323] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[0324] Step 6:

[0325] The server generates a response and sends it back to the terminal.

[0326] Specific operation: The server returns the generated answer in JSON format to the terminal.

[0327] Step 7:

[0328] The terminal displays the received answer to the user.

[0329] What it does: The device adjusts the answer it receives based on the emotion data and displays it in the user interface, saying in a relaxed tone, "To ensure a good night's sleep, it's a good idea to create a relaxing routine."

[0330] Medical device monitoring system

[0331] Processing steps and specific operations

[0332] Step 1:

[0333] The medical device transmits data to the server in real time.

[0334] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[0335] Step 2:

[0336] An emotion engine recognizes the user's emotional state.

[0337] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of tension.

[0338] Step 3:

[0339] The server analyzes the received data and emotion data.

[0340] Specific operation: The server inputs data and emotional data into a generative artificial intelligence model to detect abnormal values, patterns, and emotional states.

[0341] Step 4:

[0342] The server evaluates the state based on the data.

[0343] Specific operation: Based on the analysis results, the generative artificial intelligence model evaluates the patient's condition and determines whether there are any abnormalities.

[0344] Step 5:

[0345] The server generates appropriate usage and warning messages.

[0346] Specific operation: If the server detects an abnormality, it generates a message saying, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[0347] Step 6:

[0348] The server sends the generated message to the terminal.

[0349] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[0350] Step 7:

[0351] The terminal displays the received message to the user.

[0352] Specific behavior: The device adjusts the warning message received based on the emotional data and displays it on the user interface, in a calm and gentle tone: "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[0353] Home medical support system

[0354] Processing steps and specific operations

[0355] Step 1:

[0356] A user enters a home health care contact request.

[0357] Specific operation: The user types into the terminal, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[0358] Step 2:

[0359] An emotion engine recognizes the user's emotional state.

[0360] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of anxiety.

[0361] Step 3:

[0362] The terminal transmits a contact request and emotion data to the server.

[0363] Specific operation: The device sends a contact request and emotion data in JSON format to the server.

[0364] Step 4:

[0365] The server analyzes the contact request and the emotion data.

[0366] Specific operation: The server inputs contact request and emotion data into the generative AI model, which evaluates the user's state and emotion.

[0367] Step 5:

[0368] The server identifies the appropriate medical staff.

[0369] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[0370] Step 6:

[0371] The server creates a link or method for establishing contact.

[0372] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[0373] Step 7:

[0374] The terminal displays the received contact information to the user.

[0375] What it does: The device adjusts the generated contact information based on the emotion data and displays it in the user interface, using a reassuring tone to say, "Dr. Suzuki is available. Click here to contact him."

[0376] The above is a description of the specific processing steps and operations of each system.

[0377] Example 2

[0378] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0379] In telemedicine and home medical care, there are issues with the efficient provision of information to users regarding their health, monitoring medical devices, and supporting home medical care. In particular, the lack of personalized information that takes into account the user's emotional state can lead to a decrease in user satisfaction.

[0380] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for sending a user's health-related question to a server hosting a generative artificial intelligence model; means for analyzing the question and emotion data on the server and generating appropriate health information or advice based on the question; means for adjusting the generated information or advice based on the emotion data and displaying it on the user's terminal; means for sending data from a medical device to the server; means for analyzing the data and emotion data and detecting abnormalities; means for generating appropriate usage instructions or warning messages adjusted based on the emotion data and sending and displaying the messages to the user's or medical staff's terminal; means for the user to send a home medical care contact request to the server; means for analyzing the contact request and user information using the generative artificial intelligence model and identifying appropriate medical staff; and means for generating a link or method for establishing contact with medical staff, adjusting the generated contact information based on the emotion data, and displaying it on the user's terminal. This enables personalized information provision that takes the user's emotional state into consideration.

[0381] "User" refers to an individual or organization that uses this system.

[0382] "Terminal" refers to a computing device used by a user, including a PC, smartphone, tablet, etc.

[0383] "Server" means the computer system hosting the generative artificial intelligence model and emotion engine, and is responsible for receiving, analyzing, and processing data sent by users.

[0384] A "generative artificial intelligence model" is a model that uses algorithms learned from large amounts of data to perform tasks such as natural language processing.

[0385] An "emotion engine" refers to a software or hardware system that analyzes a user's emotional state from facial expressions, tone of voice, etc., and acquires the data.

[0386] "Health information" refers to information related to the user's health condition, including preventative measures, improvement measures, and appropriate guidelines for action.

[0387] "Medical device" refers to a device for monitoring a user's health condition, including heart rate monitors, blood pressure monitors, and other vital signs monitoring devices.

[0388] "Emotion data" is data that indicates the emotional state of the user obtained by the emotion engine.

[0389] A "contact request" is a request that a user inputs into the system to the effect that they wish to be contacted by medical staff.

[0390] "Appropriate Medical Staff" refers to medical professionals selected to respond to a user's contact requests.

[0391] This invention relates to a telemedicine and home medical support system that integrates a generative artificial intelligence model and an emotion engine. This system works in conjunction with the user, terminal, and server elements to answer the user's health-related questions, monitor medical equipment, and provide smooth support for home medical care. This system also recognizes the user's emotional state and reflects it in the analysis and information provided, thereby achieving more personalized medical support.

[0392] Health consultation and information system

[0393] First, the user inputs a health-related question into the device's input screen. For example, "I haven't been able to sleep lately. Is there anything I can do about it?" At this stage, the device uses its built-in camera and microphone to detect the user's facial expressions and tone of voice and collect emotional data. The question and emotional data are then sent from the device to the server.

[0394] The server analyzes the received questions and emotional data using a generative artificial intelligence model (e.g., GPT-4) to generate appropriate health information and advice. For example, it might generate advice such as, "To ensure a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The generated information is returned to the device in a tone adjusted based on the emotional data, and the device displays it to the user.

[0395] Prompt Sentence Examples

[0396] The user types, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects stress and generates advice taking into account health information and stress levels, and displays it in a relaxed tone.

[0397] Medical device monitoring system

[0398] The medical device automatically collects real-time data such as heart rate and blood pressure. This data is sent from the medical device to a server, and the server uses an emotion engine to grasp the user's emotional state in real time. For example, if the user is in a state of tension, that information is also collected.

[0399] The server uses a generative artificial intelligence model to analyze the medical and emotional data to detect any abnormalities. If an abnormality is detected, the server generates a warning message such as, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is adjusted based on the emotional data and sent to the device in a calm and gentle tone, and displayed to the user.

[0400] Prompt Sentence Examples

[0401] A medical device collects heart rate data. An emotion engine detects the user's tension and generates and displays a calm message informing them of abnormal heart rates.

[0402] Home medical support system

[0403] The user inputs a request for contact regarding home medical care into the device. For example, "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." At this time, the device uses an emotion engine to detect the user's anxiety and collect emotional data.

[0404] The input contact request and emotional data are sent from the device to a server, which then uses a generative artificial intelligence model to analyze the contact request and emotional information and identify the appropriate medical staff. For example, a message such as "Dr. Suzuki is available. Please click here to contact him" is generated. This message is adjusted based on the emotional data and sent to the device and displayed to the user in a reassuring tone to ease anxiety.

[0405] Prompt Sentence Examples

[0406] A user types in "Nausea, thought to be a side effect of medication." Anxiety is detected. Contact instructions for a doctor are generated and displayed in a reassuring tone.

[0407] As described above, by integrating an emotion engine and a generative AI model, this invention can solve various issues in remote medical care and home medical care and provide a personalized and efficient medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[0408] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0409] Health consultation and information system

[0410] Step 1:

[0411] The user enters a question.

[0412] Specific operation: The user accesses the device's input screen and enters a health-related question such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[0413] Input: Question

[0414] Output: Question data saved on the device

[0415] Step 2:

[0416] The device collects emotional data.

[0417] Specific operation: Using the device's camera and microphone, the system detects the user's facial expressions and tone of voice, and obtains emotional data such as stress and anxiety.

[0418] Input: User's facial expressions and tone of voice

[0419] Output: Obtained emotion data

[0420] Step 3:

[0421] The question and emotion data are sent to the server.

[0422] Specific operation: The device sends the entered question and detected emotion data to the server via an HTTP request.

[0423] Input: Question data, emotion data

[0424] Output: Question data and emotion data stored on the server

[0425] Step 4:

[0426] The server analyzes the data.

[0427] Specific operation: The server inputs the received question content and emotional data into a generative AI model (e.g., GPT-4) to generate appropriate health information and advice.

[0428] Input: Question data, emotion data

[0429] Output: Generated health information and advice

[0430] Step 5:

[0431] The generated information is returned to the terminal.

[0432] Specific operation: The server returns the generated health information and advice to the device in a tone adjusted based on the emotional data.

[0433] Input: Generated health information, advice, and emotional data

[0434] Output: Tailored health information and advice sent.

[0435] Step 6:

[0436] The terminal displays the information to the user.

[0437] Specific behavior: The device displays the information it receives to the user in a relaxed tone.

[0438] Input: Tailored health information and advice

[0439] Output: What the user sees on the screen

[0440] Medical device monitoring system

[0441] Step 1:

[0442] Medical devices collect data.

[0443] What it does: Medical devices automatically collect information like heart rate and blood pressure in real time.

[0444] Input: User biometric data (heart rate, blood pressure, etc.)

[0445] Output: Collected biometric data

[0446] Step 2:

[0447] The medical device sends the data to the server.

[0448] Specific operation: Medical devices send collected data in real time to a server.

[0449] Input: Biometric data

[0450] Output: Biometric data stored on the server

[0451] Step 3:

[0452] The server captures the emotion data.

[0453] Specific operation: The server uses an emotion engine to analyze the received biometric data and the user's emotional state (e.g., tension) in real time.

[0454] Input: Biometric data, emotional state

[0455] Output: Emotion data

[0456] Step 4:

[0457] The server analyzes the data.

[0458] How it works: The server uses the generative AI model to analyze biometric and emotional data to detect any anomalies.

[0459] Input: Biometric data, emotional data

[0460] Output: Anomaly detection results

[0461] Step 5:

[0462] If an anomaly is detected, a message is generated.

[0463] Specific behavior: If the server detects an abnormality, it generates a warning message such as "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[0464] Input: Anomaly detection results, emotion data

[0465] Output: Warning message

[0466] Step 6:

[0467] Send the generated message.

[0468] Specific behavior: The server sends the generated message to the terminal in a calm and gentle tone.

[0469] Input: warning message

[0470] Output: Adjusted warning message sent

[0471] Step 7:

[0472] The terminal displays the message.

[0473] Specific behavior: Display the message received by the device to the user.

[0474] Input: Adjusted warning message

[0475] Output: The message the user sees on the screen

[0476] Home medical support system

[0477] Step 1:

[0478] A user enters a contact request.

[0479] Specific operation: The user inputs a contact request into the terminal, saying, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to speak to a doctor."

[0480] Input: Contact Request

[0481] Output: Contact request data stored on the device

[0482] Step 2:

[0483] The device collects emotional data.

[0484] Specific operation: The device uses an emotion engine to detect the user's anxiety and collect emotion data.

[0485] Input: User's facial expressions and tone of voice

[0486] Output: Collected emotion data

[0487] Step 3:

[0488] A contact request and emotion data are sent to the server.

[0489] Specific operation: The device sends a contact request and emotion data to the server.

[0490] Input: Contact request data, emotion data

[0491] Output: Contact request data and emotion data stored on the server

[0492] Step 4:

[0493] The server analyzes the data.

[0494] What it does: The server uses a generative AI model to analyze contact requests and emotional data to identify appropriate medical staff.

[0495] Input: Contact request data, emotion data

[0496] Output: Identified medical staff information

[0497] Step 5:

[0498] Generate contact methods.

[0499] Specific operation: The server generates a method of contacting the identified medical staff (e.g., a link where the medical staff is available).

[0500] Input: Medical staff information, emotion data

[0501] Output: Generated contact information

[0502] Step 6:

[0503] The generated information is transmitted.

[0504] What it does: The server sends the generated information to the device in a reassuring tone.

[0505] Input: Generated contact information

[0506] Output: Coordinated contact information sent

[0507] Step 7:

[0508] The terminal displays the information to the user.

[0509] Specific behavior: The device displays the information it receives to the user and provides links and methods for establishing contact with medical staff.

[0510] Input: Adjusted contact information

[0511] Output: What the user sees on the screen

[0512] (Application example 2)

[0513] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0514] In telemedicine and home medical care, there is a need to provide appropriate and personalized medical support that takes into account the user's emotional state. However, conventional systems can only generate uniform answers or warning messages in response to the user's health questions or data from medical devices, and lack feedback that reflects the user's emotional state. Therefore, a system is needed that can alleviate users' anxieties and doubts and provide more effective medical support.

[0515] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0516] In this invention, the server includes emotion recognition means for sensing and analyzing the user's emotional state, means for analyzing questions and data using a generative artificial intelligence model to generate appropriate information and advice, and means for adjusting the tone and content of the information and advice based on the emotion data, thereby enabling personalized medical information and advice according to the user's emotional state.

[0517] The "means for inputting questions about the user's health" is an interface that allows the user to input questions about their own health condition or symptoms into the terminal.

[0518] The "means for sending to a server hosting a generative artificial intelligence model" is a means having the function of transferring a question entered by a user to a server hosting a generative artificial intelligence model.

[0519] A "generative artificial intelligence model" is an artificial intelligence model that performs natural language processing and data analysis to generate appropriate information and advice based on user input.

[0520] The "means for returning generated information or advice to the user's terminal" refers to a means having a function for returning generated information or advice from the server to the user's terminal.

[0521] The "means for displaying the information or advice" refers to a means having a function for visually displaying the information or advice generated on the user's terminal.

[0522] The "emotion recognition means" is a means for detecting and analyzing emotional data such as the user's facial expressions and tone of voice.

[0523] "Emotion data" is data that indicates the current emotional state of the user, and is acquired by emotion recognition means.

[0524] "Means for adjusting the tone and content of information and advice based on emotional data" refers to means for changing the method of delivery and content of information and advice generated using emotional data to suit the emotional state of the user.

[0525] The "means for collecting data from a medical device in real time and transmitting the data to a server" refers to a means for transferring data acquired from a medical device to a server in real time.

[0526] "Means for generating appropriate usage instructions or warning messages when an abnormality is detected" refers to means for analyzing data from medical devices and generating appropriate response instructions or messages urging caution when an abnormality is detected.

[0527] The "means for the user to input a request for contact regarding home medical care" is an interface for the user to input a request into a terminal when the user desires to be contacted regarding home medical care.

[0528] The "means for sending a contact request to a server" is a means for transferring a contact request regarding home medical care input by a user to a server.

[0529] The "means for identifying appropriate medical staff" is a means for analyzing the contact request and user information and selecting appropriate medical staff.

[0530] "Means for creating a link or method for establishing contact" refers to a means for creating a link or method for establishing contact with medical staff.

[0531] The "means for displaying contact information on the user's terminal" refers to a means for visually displaying the generated contact information on the user's terminal.

[0532] As an embodiment of the present invention, the following system program, hardware, and software configuration will be described.

[0533] Hardware and Software Configuration

[0534] The system of the present invention includes smart glasses, a camera, a server, a terminal, a generative artificial intelligence model, and an emotion recognition engine. The smart glasses have a built-in camera that captures the user's facial expressions in real time. The server hosts the generative artificial intelligence model (generative AI model) and the emotion recognition engine, which analyzes the user's emotional state.

[0535] Program Processing Overview

[0536] 1. Enter your question:

[0537] The user inputs a health-related question through the smart glasses, such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[0538] 2. Collecting Emotional Data:

[0539] The smart glasses have a built-in camera that captures the user's facial expressions and body movements, and the emotion recognition engine analyzes the data. In this case, TensorFlow is used to build the emotion recognition model.

[0540] 3. Data transmission and analysis:

[0541] The question and emotion data are sent to a server, where a generative AI model (e.g., GPT-3) analyzes the question and emotion data and generates appropriate health information and advice.

[0542] 4. Coordination and return of information:

[0543] The content and tone of the generated information and advice are adjusted based on the emotional data and sent back to the user's terminal.

[0544] 5. Displaying information:

[0545] Tailored information and advice will be displayed on the user's device, such as "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and to create a relaxing routine," in a relaxing tone.

[0546] Software used

[0547] TensorFlow: Used to build an emotion recognition engine

[0548] GPT-3: Generative AI model

[0549] Specific examples

[0550] To use a specific example, the following situation can be considered.

[0551] A user uses smart glasses in a virtual store and stops by the health food section. He or she wonders, "Which health food is right for me?" The emotion recognition engine detects doubt from the user's facial expressions and gestures, and sends the data to the server. The generative AI model on the server analyzes the situation—"The user is interested, but has doubts about which health food is right for me"—and generates the following example prompt:

[0552] Example 1: "User is feeling curious. Recommend a health product that suits this emotion in the context of energy supplements."

[0553] Example 2: "User feels doubtful. Suggest a health product for daily vitamin intake to relieve this emotion in a virtual store."

[0554] In this way, personalized information tailored to the user's emotional state is provided, resulting in a more satisfying shopping experience.

[0555] summary

[0556] The overall process of this system recognizes the user's emotional state in real time and provides personalized information and advice accordingly. The hardware and software components used are clearly indicated to demonstrate the implementation of the invention, enabling it to provide appropriate and efficient assistance tailored to the user's needs.

[0557] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0558] Step 1:

[0559] A user puts on the smart glasses and inputs a health question.

[0560] The input is sent to the server via the smart glasses interface. This input data includes the question in text format. For example, "I haven't been able to sleep lately. Is there anything I can do about it?"

[0561] Step 2:

[0562] The emotion recognition means captures the user's facial expressions and body movements in real time from a camera built into the smart glasses.

[0563] Facial expression data and physical movement data captured by the camera are sent to an emotion recognition engine using TensorFlow, which analyzes the user's emotional state.

[0564] As an output, the analysis produces emotional data, e.g., stress, doubt, etc.

[0565] Step 3:

[0566] The terminal transmits question data and emotion data to the server.

[0567] The transmitted data includes the user's question text data and emotion data generated by the emotion recognition engine.

[0568] The server receives this data.

[0569] Step 4:

[0570] A generative AI model (GPT-3) on the server analyzes question data and emotion data.

[0571] Specifically, a prompt is generated based on the question and emotion data. For example, "User is feeling stressed and asked about sleep issues. Provide advice to help relax before sleep."

[0572] The generative AI model generates health information and advice based on this prompt, for example, "To ensure a good night's sleep, it's helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[0573] Step 5:

[0574] The content and tone of the information and advice generated are adjusted based on the emotional data.

[0575] Example of adjustment: For a user in a stressed state, sentences are generated in a relaxed tone.

[0576] Step 6:

[0577] The server sends tailored information and advice back to the device.

[0578] The output data includes the final information and advice that will be displayed to the user, for example, "To ensure a good night's sleep, it is helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[0579] Step 7:

[0580] The terminal displays the received information and advice to the user.

[0581] Tailored information and advice is visually presented on a display within the smart glasses.

[0582] This allows users to quickly access appropriate health information and advice based on their emotional state.

[0583] Through this process, users can receive personalized health information and advice tailored to their emotional state, resulting in a more satisfying experience.

[0584] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0585] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0586] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0587] [Second embodiment]

[0588] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0589] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0590] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0591] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0592] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0593] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0594] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0595] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0596] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0597] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0598] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0599] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0600] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system involves the user, terminal, and server working together to answer questions about the user's health, monitor medical equipment, and provide smooth support for home medical care.

[0601] Health consultation and information system

[0602] overview

[0603] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[0604] Specific examples

[0605] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[0606] Medical device monitoring system

[0607] overview

[0608] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[0609] Specific examples

[0610] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[0611] Home medical support system

[0612] overview

[0613] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff, and the server then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal.

[0614] Specific examples

[0615] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The terminal then displays this message to the user.

[0616] In this way, the present invention provides an efficient and effective medical support system that utilizes generative artificial intelligence models to solve various problems in remote medical care and home medical care. By having each module work in cooperation with each other, it is possible to improve user convenience and enhance the quality of medical care.

[0617] The processing flow will be explained below.

[0618] Health consultation and information system

[0619] Processing steps and specific operations

[0620] Step 1:

[0621] The user enters a health question into the terminal.

[0622] Specific behavior: The user types "I've been having trouble sleeping lately. Is there anything I can do?" into the device's input field.

[0623] Step 2:

[0624] The terminal sends the entered question to the server.

[0625] Specific operation: The device converts the entered question into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[0626] Step 3:

[0627] The server parses the received query.

[0628] Specific operation: The server inputs the question data into a generative artificial intelligence model, performs syntactic and semantic analysis, and clarifies the intent of the question.

[0629] Step 4:

[0630] The server generates appropriate health information or advice.

[0631] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[0632] Step 5:

[0633] The server generates a response and sends it back to the terminal.

[0634] Specific operation: The server returns the generated answer in JSON format to the terminal.

[0635] Step 6:

[0636] The terminal displays the received answer to the user.

[0637] Specific behavior: The device displays the answer it receives in the user interface, saying, "To create a relaxing routine, it's a good idea to create a good sleep environment."

[0638] Medical device monitoring system

[0639] Processing steps and specific operations

[0640] Step 1:

[0641] The medical device transmits data to the server in real time.

[0642] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[0643] Step 2:

[0644] The server parses the received data.

[0645] How it works: The server inputs data into a generative artificial intelligence model to detect abnormal values ​​and patterns.

[0646] Step 3:

[0647] The server evaluates the state based on the data.

[0648] Specific operation: The generative artificial intelligence model evaluates the patient's condition based on the analysis results and determines whether there are any abnormalities.

[0649] Step 4:

[0650] The server generates appropriate usage and warning messages.

[0651] Specific operation: If the server detects an abnormality, it generates a warning message such as, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[0652] Step 5:

[0653] The server sends the generated message to the terminal.

[0654] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[0655] Step 6:

[0656] The terminal displays the received message to the user.

[0657] Specific behavior: The device receives a message and displays it on the user interface, saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[0658] Home medical support system

[0659] Processing steps and specific operations

[0660] Step 1:

[0661] A user enters a home health care contact request.

[0662] Specific action: The user types into the device's input field, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[0663] Step 2:

[0664] The terminal sends a contact request to the server.

[0665] Specific operation: The terminal sends the entered contact request in JSON format to the server.

[0666] Step 3:

[0667] The server parses the contact request and the user information.

[0668] Specific operation: The server uses a generative artificial intelligence model to analyze the contact request and user information and evaluate the user's condition and symptoms.

[0669] Step 4:

[0670] The server identifies the appropriate medical staff.

[0671] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[0672] Step 5:

[0673] The server creates a link or method for establishing contact.

[0674] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[0675] Step 6:

[0676] The terminal displays the received contact information to the user.

[0677] Specific operation: The device displays the generated contact information in the user interface, displaying "Dr. Suzuki is available. Click here to contact him."

[0678] The above is a description of the specific processing steps and operations of each system.

[0679] Example 1

[0680] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0681] The current medical environment lacks a system that effectively supports telemedicine and home medical care. In particular, systems that can provide appropriate information and alerts in real time are needed for health consultations, medical device monitoring, and home medical care communications. This will improve convenience for patients and medical professionals and enable prompt and appropriate responses.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0683] In this invention, the server includes a means for analyzing a user's health-related questions and generating appropriate health information or advice, a means for analyzing data from medical devices and detecting abnormalities, and a means for analyzing contact requests and user information and identifying appropriate medical professionals, thereby enabling a quick response to the user's health consultation, real-time detection of abnormalities in medical devices, and quick establishment of contact with appropriate medical professionals.

[0684] "User" refers to an individual or patient who uses the system to receive health consultations, monitor medical devices, or contact home medical care.

[0685] A "question" is a question that a user inputs about a health-related question or symptom and requests an answer.

[0686] A "generative artificial intelligence model" is an artificial intelligence system that analyzes input text and generates appropriate information and advice.

[0687] A "server" is a computer system that hosts a generative artificial intelligence model, analyzes questions sent by users and medical device data, and generates and returns appropriate information.

[0688] A "terminal" is a hardware device used by a user, such as a smartphone, tablet, or PC.

[0689] "Health information or advice" refers to information or recommended actions provided by a generative artificial intelligence model in response to a health consultation.

[0690] "Medical devices" are devices used to monitor a patient's health, such as heart rate monitors and blood pressure monitors.

[0691] "Data" refers to the patient's vital signs and measurement results collected in real time by medical devices.

[0692] An "abnormality" is a phenomenon that is determined as a measurement value or state outside the normal range as a result of the generative artificial intelligence model analyzing data.

[0693] A "contact request" is a request by a user to contact a medical professional regarding home medical care.

[0694] "Medical professionals" are professionals such as doctors and nurses who provide medical services and advice to users.

[0695] A "link or method" is a communication method or URL generated to allow a user to contact the appropriate healthcare professional.

[0696] A "warning message" is the warning message provided by the generative artificial intelligence model when an abnormality in a medical device is detected.

[0697] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system operates in cooperation with users, terminals, and servers to answer users' health-related questions, monitor medical equipment, and provide smooth support for home medical care.

[0698] Hardware and software used

[0699] Hardware: User devices (smartphones, tablets, PCs), medical devices (heart rate monitors, blood pressure monitors, etc.)

[0700] Software: Generative AI model on the server (e.g., GPT-3, BERT, etc.), communication protocol (HTTP / S)

[0701] Health consultation and information system

[0702] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[0703] Specific examples

[0704] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[0705] Medical device monitoring system

[0706] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[0707] Specific examples

[0708] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[0709] Home medical support system

[0710] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff. The server then generates links and instructions for establishing contact with the medical staff and displays them on the user's terminal.

[0711] Specific examples

[0712] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Medical Professional A is available. Please click here to contact him." The terminal then displays this message to the user.

[0713] Prompt Sentence Examples

[0714] "Please tell me what to do about not being able to sleep."

[0715] What should I do if my heart rate is high?

[0716] "I'm feeling nauseous, which doctor should I consult?"

[0717] As a result, the present invention utilizes generative artificial intelligence models to efficiently solve various issues in remote medical care and home medical care, providing an effective medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[0718] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0719] Health consultation and information system

[0720] Processing Steps

[0721] Step 1:

[0722] The user uses the terminal to input health-related questions.

[0723] Input: The text question entered by the user into the terminal

[0724] Output: The device is ready to send the question data to the server.

[0725] Step 2:

[0726] The terminal transmits the entered question to the server.

[0727] Input: Question data from the user

[0728] Output: Query data arrives at the server

[0729] Step 3:

[0730] The server passes the received question to a generative AI model and analyzes the content.

[0731] Input: Query data arriving at the server

[0732] Output: The results of the generative AI model's analysis of the question

[0733] Step 4:

[0734] The server generates appropriate health information and advice based on the analysis results returned by the generative AI model.

[0735] Input: Question content analyzed by the generative AI model

[0736] Output: Health information or advice

[0737] Step 5:

[0738] The server sends the generated response to the terminal.

[0739] Input: Generated health information or advice

[0740] Output: The device receives the data

[0741] Step 6:

[0742] The terminal displays the received response to the user.

[0743] Input: Health information or advice received from the server

[0744] Output: A screen display where users can view their answers

[0745] Medical device monitoring system

[0746] Processing Steps

[0747] Step 1:

[0748] Medical devices collect data in real time.

[0749] Input: Patient's vital signs (e.g., heart rate)

[0750] Output: Collected data

[0751] Step 2:

[0752] The medical device sends the collected data to a server.

[0753] Input: Data from medical devices

[0754] Output: Data arrives at the server

[0755] Step 3:

[0756] The server analyzes the received data using the generated AI model and checks for any abnormalities.

[0757] Input: Data arriving at the server

[0758] Output: Analysis results of the generative AI model (normal or abnormal)

[0759] Step 4:

[0760] If the server detects an abnormality, it will generate appropriate usage or warning messages.

[0761] Input: Analysis results of the generative AI model (in case of anomalies)

[0762] Output: Warning message

[0763] Step 5:

[0764] The server sends the generated warning message to the terminal of the user or medical staff.

[0765] Input: warning message

[0766] Output: The device receives the data

[0767] Step 6:

[0768] The terminal displays the received warning message to the user.

[0769] Input: The warning message received from the server

[0770] Output: A screen display where the user can view the warning

[0771] Home medical support system

[0772] Processing Steps

[0773] Step 1:

[0774] The user inputs a contact request for home medical care from a terminal.

[0775] Input: Text information entered by the user into the device

[0776] Output: The device is ready to send contact request data to the server

[0777] Step 2:

[0778] The terminal sends a contact request to the server.

[0779] Input: Contact request data from the user

[0780] Output: Contact request data arrives at the server

[0781] Step 3:

[0782] The server uses a generative AI model to analyze the contact request and user information to identify appropriate medical staff.

[0783] Input: Contact request data and user information received by the server

[0784] Output: Identification of appropriate medical staff

[0785] Step 4:

[0786] The server creates links and methods for establishing contact with medical staff.

[0787] Input: Identification of appropriate medical staff

[0788] Output: Means of contact (link or method)

[0789] Step 5:

[0790] The server sends the generated contact information to the user's terminal.

[0791] Input: Contact Method

[0792] Output: The device receives the data

[0793] Step 6:

[0794] The terminal displays the received contact information to the user.

[0795] Input: Contact information received from the server

[0796] Output: A screen display where the user can view contact information

[0797] Through these steps, the system can quickly respond to user questions and contact requests and monitor medical devices in real time.

[0798] (Application example 1)

[0799] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] The present invention relates to a system for more efficient and rapid health management using smart devices. Its purpose is to provide a means for users to quickly and easily ask health-related questions and receive appropriate advice while on the go. It also aims to provide a comprehensive telemedicine system that also includes medical equipment monitoring and home medical care support.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0802] In this invention, the server includes means for receiving and analyzing voice input from the smart glasses, means for sending questions to a generative artificial intelligence model on the server and obtaining answers, and means for displaying the obtained answers on the display of the smart glasses, thereby enabling users to ask health-related questions in real time and receive appropriate advice even when they are out and about.

[0803] "User" means an individual or end user who uses the system.

[0804] "Health-related questions" refer to questions or concerns that a user may have about their own health condition or medical care.

[0805] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data from users and generates appropriate information and answers.

[0806] A "server" is a back-end system that hosts generative artificial intelligence models and processes data submitted by users.

[0807] "Terminal" refers to a device, such as smart glasses, a smartphone, a tablet, etc., that a user uses to send and receive input data.

[0808] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand the questions and data sent by the user and generate appropriate information.

[0809] "Smart glasses" are wearable devices that have a built-in display and can receive user input via voice or vision.

[0810] "Real-time" means that data is generated and analyzed almost simultaneously with user input.

[0811] "Medical device" refers to a hardware device used to monitor a user's physical condition, such as a heart rate monitor or blood pressure monitor.

[0812] A "display" is a display device for visually displaying generated information and answers.

[0813] A "link" is a means or method by which a user can easily contact designated medical personnel.

[0814] "Contact Request" means a notification or instruction for a User to seek home health care assistance.

[0815] The present invention provides a system that allows users to use smart glasses to ask health-related questions in real time and obtain appropriate health information and advice through a generative artificial intelligence model. Detailed embodiments for implementing this system are described below.

[0816] The system mainly consists of three main components: users, terminals, and servers.

[0817] 1. Users

[0818] As an end user of the system, a user wears smart glasses. They input health-related questions by voice, and the input is received by the microphone of the smart glasses. For example, they might ask, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?"

[0819] 2. Terminal (smart glasses)

[0820] The smart glasses have the following features:

[0821] Acquiring voice input: The smart glasses acquire the user's voice questions through a microphone.

[0822] Speech-to-text conversion: Convert the captured speech into text and send it to the server. This is done using a speech recognition library (for example, Python's speech_recognition library).

[0823] Communication with the server: The user's question is sent to the server and the answer is received from the server.

[0824] Display function: The answer received from the server is displayed on the smart glasses' display, and at the same time, feedback is given to the user via voice.

[0825] 3. Server

[0826] The server hosts the generative artificial intelligence model and has the following functions:

[0827] Question analysis: The text question sent by the user is analyzed and an appropriate answer is generated using a generative artificial intelligence model (e.g., GPT-3). The prompt sentence is the user's question, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?"

[0828] Answer generation: The generative artificial intelligence model generates appropriate health information and advice based on the analysis results and sends it back to the device.

[0829] Specific examples

[0830] In a specific usage scenario, a user wearing smart glasses asks a question while out and about: "If I consume caffeine before bed, I have trouble falling asleep. What should I do?" This question is picked up by the smart glasses' microphone and converted into text using a speech recognition library. The text question is sent to a server, where a generative artificial intelligence model generates an answer such as, "It's effective to avoid consuming caffeine before bed and to create a relaxing bedtime routine." The generated answer is sent back to the smart glasses and displayed on the user's display.

[0831] This allows users to solve health-related questions in real time and receive appropriate advice while on the go. Medical equipment monitoring and home medical care support are also analyzed on the server, and the necessary information is provided to users and medical staff.

[0832] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0833] Step 1:

[0834] The user puts on the smart glasses and speaks to ask a health-related question. The voice input is in the form of, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?" This voice data is picked up by the microphone device in the smart glasses.

[0835] Step 2:

[0836] The device (smart glasses) converts the acquired voice data into text data. This conversion is performed using a voice recognition library (for example, Python's speech_recognition library). The input is voice data, and the output is text data.

[0837] Step 3:

[0838] The terminal sends the user's question to the server in the form of text data. Here, the terminal sends the text data to the specified endpoint of the server using an HTTP POST request. The input is the text data, and the output is the request sent to the server.

[0839] Step 4:

[0840] The server passes the received text data to a generative artificial intelligence model and begins analyzing the question. This analysis is input to the model as a prompt sentence, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?" The input is text data (prompt sentence), and the output is answer data based on the analysis results.

[0841] Step 5:

[0842] A generative artificial intelligence model (e.g., GPT-3) analyzes the question and generates an appropriate answer. An example answer might be, "Headaches are a common side effect of medication, so we recommend consulting your doctor. Also, drink plenty of fluids and try to relax as much as possible." The input is the prompt, and the output is the generated answer data.

[0843] Step 6:

[0844] The server returns the answer data obtained from the generative AI model to the terminal. This process is performed as an HTTP response. The input is the generated answer data, and the output is the transmission of the response.

[0845] Step 7:

[0846] The terminal displays the received answer data on the display of the smart glasses. In addition, it also provides audio feedback to the user using a voice output library (e.g., Python's pyttsx3). The input is the answer data from the server, and the output is the display and audio output.

[0847] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0848] This invention relates to a telemedicine and home medical care support system that integrates a generative artificial intelligence model and an emotion engine. This system involves the user, terminal, and server working together to answer the user's health-related questions, monitor medical equipment, and provide seamless support for home medical care. This system also recognizes the user's emotional state and reflects this in its analysis and information provision, enabling more personalized medical support.

[0849] Health consultation and information system

[0850] overview

[0851] The user enters a health-related question into the device, and the question along with the user's emotional data detected by the emotion engine is sent to the server. A generative AI model on the server analyzes the question and emotional data and generates appropriate health information and advice. The generated information is sent back to the device and displayed to the user in a form adjusted based on the emotional data.

[0852] Specific examples

[0853] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects the user's level of stress from their facial expressions and tone of voice. The device then sends this question and emotional data to the server. The server uses a generative artificial intelligence model to analyze the question and emotional data and generate advice such as, "To create a good sleep environment, it is effective to limit your intake of caffeine and alcohol before bed and create a relaxing routine." The device then displays this advice to the user in a relaxed tone that takes into account the user's level of stress.

[0854] Medical device monitoring system

[0855] overview

[0856] Medical devices send data collected in real time to a server, where an emotion engine grasps the user's emotional state. A generative AI model on the server analyzes the data and emotion data, and generates appropriate usage instructions and warning messages if an abnormality is detected. This information is adjusted based on the emotion data and sent to and displayed on the user's or medical staff's device.

[0857] Specific examples

[0858] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. An emotion engine recognizes the user's current emotional state as stress. The server uses a generative artificial intelligence model to analyze the data and emotional information and generate a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is displayed to the user in a calm and gentle tone that takes into account the stressful state.

[0859] Home medical support system

[0860] overview

[0861] A user inputs a contact request for home medical care from a terminal, and the request and emotional data detected by the emotion engine are sent to the server. A generative artificial intelligence model on the server analyzes the contact request, emotional data, and user information to identify appropriate medical staff. It then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal in a format adjusted based on the emotional data.

[0862] Specific examples

[0863] The user types into the device, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to a doctor." The emotion engine detects anxiety from the user's facial expression. The device then sends this contact request and emotion data to the server. The server uses a generative artificial intelligence model to analyze the contact request and emotion information, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The device then displays this message to the user in a reassuring tone that takes into account the user's state of anxiety.

[0864] In this way, by integrating an emotion engine and a generative artificial intelligence model, this invention solves various issues in remote medical care and home medical care, and provides a personalized and efficient medical support system. By each module working together, it is possible to improve user convenience and enhance the quality of medical care.

[0865] The processing flow will be explained below.

[0866] Health consultation and information system

[0867] Processing steps and specific operations

[0868] Step 1:

[0869] The user enters a health question into the terminal.

[0870] Specific operation: The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?"

[0871] Step 2:

[0872] An emotion engine recognizes the user's emotional state.

[0873] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's stress level from their facial expressions and tone of voice.

[0874] Step 3:

[0875] The device sends the question and emotion data to the server.

[0876] Specific operation: The device converts the question and emotion data into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[0877] Step 4:

[0878] The server analyzes the question and sentiment data.

[0879] Specific operation: The server inputs the question and emotion data into the generative AI model, and performs syntactic and semantic analysis to clarify the intent and emotional state of the question.

[0880] Step 5:

[0881] The server generates appropriate health information or advice.

[0882] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[0883] Step 6:

[0884] The server generates a response and sends it back to the terminal.

[0885] Specific operation: The server returns the generated answer in JSON format to the terminal.

[0886] Step 7:

[0887] The terminal displays the received answer to the user.

[0888] What it does: The device adjusts the answer it receives based on the emotion data and displays it in the user interface, saying in a relaxed tone, "To ensure a good night's sleep, it's a good idea to create a relaxing routine."

[0889] Medical device monitoring system

[0890] Processing steps and specific operations

[0891] Step 1:

[0892] The medical device transmits data to the server in real time.

[0893] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[0894] Step 2:

[0895] An emotion engine recognizes the user's emotional state.

[0896] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of tension.

[0897] Step 3:

[0898] The server analyzes the received data and emotion data.

[0899] Specific operation: The server inputs data and emotional data into a generative artificial intelligence model to detect abnormal values, patterns, and emotional states.

[0900] Step 4:

[0901] The server evaluates the state based on the data.

[0902] Specific operation: Based on the analysis results, the generative artificial intelligence model evaluates the patient's condition and determines whether there are any abnormalities.

[0903] Step 5:

[0904] The server generates appropriate usage and warning messages.

[0905] Specific operation: If the server detects an abnormality, it generates a message saying, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[0906] Step 6:

[0907] The server sends the generated message to the terminal.

[0908] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[0909] Step 7:

[0910] The terminal displays the received message to the user.

[0911] Specific behavior: The device adjusts the warning message received based on the emotional data and displays it on the user interface, in a calm and gentle tone: "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[0912] Home medical support system

[0913] Processing steps and specific operations

[0914] Step 1:

[0915] A user enters a home health care contact request.

[0916] Specific operation: The user types into the terminal, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[0917] Step 2:

[0918] An emotion engine recognizes the user's emotional state.

[0919] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of anxiety.

[0920] Step 3:

[0921] The terminal transmits a contact request and emotion data to the server.

[0922] Specific operation: The device sends a contact request and emotion data in JSON format to the server.

[0923] Step 4:

[0924] The server analyzes the contact request and the emotion data.

[0925] Specific operation: The server inputs contact request and emotion data into the generative AI model, which evaluates the user's state and emotion.

[0926] Step 5:

[0927] The server identifies the appropriate medical staff.

[0928] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[0929] Step 6:

[0930] The server creates a link or method for establishing contact.

[0931] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[0932] Step 7:

[0933] The terminal displays the received contact information to the user.

[0934] What it does: The device adjusts the generated contact information based on the emotion data and displays it in the user interface, using a reassuring tone to say, "Dr. Suzuki is available. Click here to contact him."

[0935] The above is a description of the specific processing steps and operations of each system.

[0936] Example 2

[0937] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0938] In telemedicine and home medical care, there are issues with the efficient provision of information to users regarding their health, monitoring medical devices, and supporting home medical care. In particular, the lack of personalized information that takes into account the user's emotional state can lead to a decrease in user satisfaction.

[0939] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for sending a user's health-related question to a server hosting a generative artificial intelligence model; means for analyzing the question and emotion data on the server and generating appropriate health information or advice based on the question; means for adjusting the generated information or advice based on the emotion data and displaying it on the user's terminal; means for sending data from a medical device to the server; means for analyzing the data and emotion data and detecting abnormalities; means for generating appropriate usage instructions or warning messages adjusted based on the emotion data and sending and displaying the messages to the user's or medical staff's terminal; means for the user to send a home medical care contact request to the server; means for analyzing the contact request and user information using the generative artificial intelligence model and identifying appropriate medical staff; and means for generating a link or method for establishing contact with medical staff, adjusting the generated contact information based on the emotion data, and displaying it on the user's terminal. This enables personalized information provision that takes the user's emotional state into consideration.

[0940] "User" refers to an individual or organization that uses this system.

[0941] "Terminal" refers to a computing device used by a user, including a PC, smartphone, tablet, etc.

[0942] "Server" means the computer system hosting the generative artificial intelligence model and emotion engine, and is responsible for receiving, analyzing, and processing data sent by users.

[0943] A "generative artificial intelligence model" is a model that uses algorithms learned from large amounts of data to perform tasks such as natural language processing.

[0944] An "emotion engine" refers to a software or hardware system that analyzes a user's emotional state from facial expressions, tone of voice, etc., and acquires the data.

[0945] "Health information" refers to information related to the user's health condition, including preventative measures, improvement measures, and appropriate guidelines for action.

[0946] "Medical device" refers to a device for monitoring a user's health condition, including heart rate monitors, blood pressure monitors, and other vital signs monitoring devices.

[0947] "Emotion data" is data that indicates the emotional state of the user obtained by the emotion engine.

[0948] A "contact request" is a request that a user inputs into the system to the effect that they wish to be contacted by medical staff.

[0949] "Appropriate Medical Staff" refers to medical professionals selected to respond to a user's contact requests.

[0950] This invention relates to a telemedicine and home medical support system that integrates a generative artificial intelligence model and an emotion engine. This system works in conjunction with the user, terminal, and server elements to answer the user's health-related questions, monitor medical equipment, and provide smooth support for home medical care. This system also recognizes the user's emotional state and reflects it in the analysis and information provided, thereby achieving more personalized medical support.

[0951] Health consultation and information system

[0952] First, the user inputs a health-related question into the device's input screen. For example, "I haven't been able to sleep lately. Is there anything I can do about it?" At this stage, the device uses its built-in camera and microphone to detect the user's facial expressions and tone of voice and collect emotional data. The question and emotional data are then sent from the device to the server.

[0953] The server analyzes the received questions and emotional data using a generative artificial intelligence model (e.g., GPT-4) to generate appropriate health information and advice. For example, it might generate advice such as, "To ensure a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The generated information is returned to the device in a tone adjusted based on the emotional data, and the device displays it to the user.

[0954] Prompt Sentence Examples

[0955] The user types, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects stress and generates advice taking into account health information and stress levels, and displays it in a relaxed tone.

[0956] Medical device monitoring system

[0957] The medical device automatically collects real-time data such as heart rate and blood pressure. This data is sent from the medical device to a server, and the server uses an emotion engine to grasp the user's emotional state in real time. For example, if the user is in a state of tension, that information is also collected.

[0958] The server uses a generative artificial intelligence model to analyze the medical and emotional data to detect any abnormalities. If an abnormality is detected, the server generates a warning message such as, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is adjusted based on the emotional data and sent to the device in a calm and gentle tone, and displayed to the user.

[0959] Prompt Sentence Examples

[0960] A medical device collects heart rate data. An emotion engine detects the user's tension and generates and displays a calm message informing them of abnormal heart rates.

[0961] Home medical support system

[0962] The user inputs a request for contact regarding home medical care into the device. For example, "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." At this time, the device uses an emotion engine to detect the user's anxiety and collect emotional data.

[0963] The input contact request and emotional data are sent from the device to a server, which then uses a generative artificial intelligence model to analyze the contact request and emotional information and identify the appropriate medical staff. For example, a message such as "Dr. Suzuki is available. Please click here to contact him" is generated. This message is adjusted based on the emotional data and sent to the device and displayed to the user in a reassuring tone to ease anxiety.

[0964] Prompt Sentence Examples

[0965] A user types in "Nausea, thought to be a side effect of medication." Anxiety is detected. Contact instructions for a doctor are generated and displayed in a reassuring tone.

[0966] As described above, by integrating an emotion engine and a generative AI model, this invention can solve various issues in remote medical care and home medical care and provide a personalized and efficient medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[0967] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0968] Health consultation and information system

[0969] Step 1:

[0970] The user enters a question.

[0971] Specific operation: The user accesses the device's input screen and enters a health-related question such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[0972] Input: Question

[0973] Output: Question data saved on the device

[0974] Step 2:

[0975] The device collects emotional data.

[0976] Specific operation: Using the device's camera and microphone, the system detects the user's facial expressions and tone of voice, and obtains emotional data such as stress and anxiety.

[0977] Input: User's facial expressions and tone of voice

[0978] Output: Obtained emotion data

[0979] Step 3:

[0980] The question and emotion data are sent to the server.

[0981] Specific operation: The device sends the entered question and detected emotion data to the server via an HTTP request.

[0982] Input: Question data, emotion data

[0983] Output: Question data and emotion data stored on the server

[0984] Step 4:

[0985] The server analyzes the data.

[0986] Specific operation: The server inputs the received question content and emotional data into a generative AI model (e.g., GPT-4) to generate appropriate health information and advice.

[0987] Input: Question data, emotion data

[0988] Output: Generated health information and advice

[0989] Step 5:

[0990] The generated information is returned to the terminal.

[0991] Specific operation: The server returns the generated health information and advice to the device in a tone adjusted based on the emotional data.

[0992] Input: Generated health information, advice, and emotional data

[0993] Output: Tailored health information and advice sent.

[0994] Step 6:

[0995] The terminal displays the information to the user.

[0996] Specific behavior: The device displays the information it receives to the user in a relaxed tone.

[0997] Input: Tailored health information and advice

[0998] Output: What the user sees on the screen

[0999] Medical device monitoring system

[1000] Step 1:

[1001] Medical devices collect data.

[1002] What it does: Medical devices automatically collect information like heart rate and blood pressure in real time.

[1003] Input: User biometric data (heart rate, blood pressure, etc.)

[1004] Output: Collected biometric data

[1005] Step 2:

[1006] The medical device sends the data to the server.

[1007] Specific operation: Medical devices send collected data in real time to a server.

[1008] Input: Biometric data

[1009] Output: Biometric data stored on the server

[1010] Step 3:

[1011] The server captures the emotion data.

[1012] Specific operation: The server uses an emotion engine to analyze the received biometric data and the user's emotional state (e.g., tension) in real time.

[1013] Input: Biometric data, emotional state

[1014] Output: Emotion data

[1015] Step 4:

[1016] The server analyzes the data.

[1017] How it works: The server uses the generative AI model to analyze biometric and emotional data to detect any anomalies.

[1018] Input: Biometric data, emotional data

[1019] Output: Anomaly detection results

[1020] Step 5:

[1021] If an anomaly is detected, a message is generated.

[1022] Specific behavior: If the server detects an abnormality, it generates a warning message such as "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[1023] Input: Anomaly detection results, emotion data

[1024] Output: Warning message

[1025] Step 6:

[1026] Send the generated message.

[1027] Specific behavior: The server sends the generated message to the terminal in a calm and gentle tone.

[1028] Input: warning message

[1029] Output: Adjusted warning message sent

[1030] Step 7:

[1031] The terminal displays the message.

[1032] Specific behavior: Display the message received by the device to the user.

[1033] Input: Adjusted warning message

[1034] Output: The message the user sees on the screen

[1035] Home medical support system

[1036] Step 1:

[1037] A user enters a contact request.

[1038] Specific operation: The user inputs a contact request into the terminal, saying, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to speak to a doctor."

[1039] Input: Contact Request

[1040] Output: Contact request data stored on the device

[1041] Step 2:

[1042] The device collects emotional data.

[1043] Specific operation: The device uses an emotion engine to detect the user's anxiety and collect emotion data.

[1044] Input: User's facial expressions and tone of voice

[1045] Output: Collected emotion data

[1046] Step 3:

[1047] A contact request and emotion data are sent to the server.

[1048] Specific operation: The device sends a contact request and emotion data to the server.

[1049] Input: Contact request data, emotion data

[1050] Output: Contact request data and emotion data stored on the server

[1051] Step 4:

[1052] The server analyzes the data.

[1053] What it does: The server uses a generative AI model to analyze contact requests and emotional data to identify appropriate medical staff.

[1054] Input: Contact request data, emotion data

[1055] Output: Identified medical staff information

[1056] Step 5:

[1057] Generate contact methods.

[1058] Specific operation: The server generates a method of contacting the identified medical staff (e.g., a link where the medical staff is available).

[1059] Input: Medical staff information, emotion data

[1060] Output: Generated contact information

[1061] Step 6:

[1062] The generated information is transmitted.

[1063] What it does: The server sends the generated information to the device in a reassuring tone.

[1064] Input: Generated contact information

[1065] Output: Coordinated contact information sent

[1066] Step 7:

[1067] The terminal displays the information to the user.

[1068] Specific behavior: The device displays the information it receives to the user and provides links and methods for establishing contact with medical staff.

[1069] Input: Adjusted contact information

[1070] Output: What the user sees on the screen

[1071] (Application example 2)

[1072] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1073] In telemedicine and home medical care, there is a need to provide appropriate and personalized medical support that takes into account the user's emotional state. However, conventional systems can only generate uniform answers or warning messages in response to the user's health questions or data from medical devices, and lack feedback that reflects the user's emotional state. Therefore, a system is needed that can alleviate users' anxieties and doubts and provide more effective medical support.

[1074] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1075] In this invention, the server includes emotion recognition means for sensing and analyzing the user's emotional state, means for analyzing questions and data using a generative artificial intelligence model to generate appropriate information and advice, and means for adjusting the tone and content of the information and advice based on the emotion data, thereby enabling personalized medical information and advice according to the user's emotional state.

[1076] The "means for inputting questions about the user's health" is an interface that allows the user to input questions about their own health condition or symptoms into the terminal.

[1077] The "means for sending to a server hosting a generative artificial intelligence model" is a means having the function of transferring a question entered by a user to a server hosting a generative artificial intelligence model.

[1078] A "generative artificial intelligence model" is an artificial intelligence model that performs natural language processing and data analysis to generate appropriate information and advice based on user input.

[1079] The "means for returning generated information or advice to the user's terminal" refers to a means having a function for returning generated information or advice from the server to the user's terminal.

[1080] The "means for displaying the information or advice" refers to a means having a function for visually displaying the information or advice generated on the user's terminal.

[1081] The "emotion recognition means" is a means for detecting and analyzing emotional data such as the user's facial expressions and tone of voice.

[1082] "Emotion data" is data that indicates the current emotional state of the user, and is acquired by emotion recognition means.

[1083] "Means for adjusting the tone and content of information and advice based on emotional data" refers to means for changing the method of delivery and content of information and advice generated using emotional data to suit the emotional state of the user.

[1084] The "means for collecting data from a medical device in real time and transmitting the data to a server" refers to a means for transferring data acquired from a medical device to a server in real time.

[1085] "Means for generating appropriate usage instructions or warning messages when an abnormality is detected" refers to means for analyzing data from medical devices and generating appropriate response instructions or messages urging caution when an abnormality is detected.

[1086] The "means for the user to input a request for contact regarding home medical care" is an interface for the user to input a request into a terminal when the user desires to be contacted regarding home medical care.

[1087] The "means for sending a contact request to a server" is a means for transferring a contact request regarding home medical care input by a user to a server.

[1088] The "means for identifying appropriate medical staff" is a means for analyzing the contact request and user information and selecting appropriate medical staff.

[1089] "Means for creating a link or method for establishing contact" refers to a means for creating a link or method for establishing contact with medical staff.

[1090] The "means for displaying contact information on the user's terminal" refers to a means for visually displaying the generated contact information on the user's terminal.

[1091] As an embodiment of the present invention, the following system program, hardware, and software configuration will be described.

[1092] Hardware and Software Configuration

[1093] The system of the present invention includes smart glasses, a camera, a server, a terminal, a generative artificial intelligence model, and an emotion recognition engine. The smart glasses have a built-in camera that captures the user's facial expressions in real time. The server hosts the generative artificial intelligence model (generative AI model) and the emotion recognition engine, which analyzes the user's emotional state.

[1094] Program Processing Overview

[1095] 1. Enter your question:

[1096] The user inputs a health-related question through the smart glasses, such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[1097] 2. Collecting Emotional Data:

[1098] The smart glasses have a built-in camera that captures the user's facial expressions and body movements, and the emotion recognition engine analyzes the data. In this case, TensorFlow is used to build the emotion recognition model.

[1099] 3. Data transmission and analysis:

[1100] The question and emotion data are sent to a server, where a generative AI model (e.g., GPT-3) analyzes the question and emotion data and generates appropriate health information and advice.

[1101] 4. Coordination and return of information:

[1102] The content and tone of the generated information and advice are adjusted based on the emotional data and sent back to the user's terminal.

[1103] 5. Displaying information:

[1104] Tailored information and advice will be displayed on the user's device, such as "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and to create a relaxing routine," in a relaxing tone.

[1105] Software used

[1106] TensorFlow: Used to build an emotion recognition engine

[1107] GPT-3: Generative AI model

[1108] Specific examples

[1109] To use a specific example, the following situation can be considered.

[1110] A user uses smart glasses in a virtual store and stops by the health food section. He or she wonders, "Which health food is right for me?" The emotion recognition engine detects doubt from the user's facial expressions and gestures, and sends the data to the server. The generative AI model on the server analyzes the situation—"The user is interested, but has doubts about which health food is right for me"—and generates the following example prompt:

[1111] Example 1: "User is feeling curious. Recommend a health product that suits this emotion in the context of energy supplements."

[1112] Example 2: "User feels doubtful. Suggest a health product for daily vitamin intake to relieve this emotion in a virtual store."

[1113] In this way, personalized information tailored to the user's emotional state is provided, resulting in a more satisfying shopping experience.

[1114] summary

[1115] The overall process of this system recognizes the user's emotional state in real time and provides personalized information and advice accordingly. The hardware and software components used are clearly indicated to demonstrate the implementation of the invention, enabling it to provide appropriate and efficient assistance tailored to the user's needs.

[1116] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1117] Step 1:

[1118] A user puts on the smart glasses and inputs a health question.

[1119] The input is sent to the server via the smart glasses interface. This input data includes the question in text format. For example, "I haven't been able to sleep lately. Is there anything I can do about it?"

[1120] Step 2:

[1121] The emotion recognition means captures the user's facial expressions and body movements in real time from a camera built into the smart glasses.

[1122] Facial expression data and physical movement data captured by the camera are sent to an emotion recognition engine using TensorFlow, which analyzes the user's emotional state.

[1123] As an output, the analysis produces emotional data, e.g., stress, doubt, etc.

[1124] Step 3:

[1125] The terminal transmits question data and emotion data to the server.

[1126] The transmitted data includes the user's question text data and emotion data generated by the emotion recognition engine.

[1127] The server receives this data.

[1128] Step 4:

[1129] A generative AI model (GPT-3) on the server analyzes question data and emotion data.

[1130] Specifically, a prompt is generated based on the question and emotion data. For example, "User is feeling stressed and asked about sleep issues. Provide advice to help relax before sleep."

[1131] The generative AI model generates health information and advice based on this prompt, for example, "To ensure a good night's sleep, it's helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[1132] Step 5:

[1133] The content and tone of the information and advice generated are adjusted based on the emotional data.

[1134] Example of adjustment: For a user in a stressed state, sentences are generated in a relaxed tone.

[1135] Step 6:

[1136] The server sends tailored information and advice back to the device.

[1137] The output data includes the final information and advice that will be displayed to the user, for example, "To ensure a good night's sleep, it is helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[1138] Step 7:

[1139] The terminal displays the received information and advice to the user.

[1140] Tailored information and advice is visually presented on a display within the smart glasses.

[1141] This allows users to quickly access appropriate health information and advice based on their emotional state.

[1142] Through this process, users can receive personalized health information and advice tailored to their emotional state, resulting in a more satisfying experience.

[1143] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1144] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1145] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1146] [Third embodiment]

[1147] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1148] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1149] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1150] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1151] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1152] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1153] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1154] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1155] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1156] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1157] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1158] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1159] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system involves the user, terminal, and server working together to answer questions about the user's health, monitor medical equipment, and provide smooth support for home medical care.

[1160] Health consultation and information system

[1161] overview

[1162] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[1163] Specific examples

[1164] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[1165] Medical device monitoring system

[1166] overview

[1167] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[1168] Specific examples

[1169] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[1170] Home medical support system

[1171] overview

[1172] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff, and the server then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal.

[1173] Specific examples

[1174] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The terminal then displays this message to the user.

[1175] In this way, the present invention provides an efficient and effective medical support system that utilizes generative artificial intelligence models to solve various problems in remote medical care and home medical care. By having each module work in cooperation with each other, it is possible to improve user convenience and enhance the quality of medical care.

[1176] The processing flow will be explained below.

[1177] Health consultation and information system

[1178] Processing steps and specific operations

[1179] Step 1:

[1180] The user enters a health question into the terminal.

[1181] Specific behavior: The user types "I've been having trouble sleeping lately. Is there anything I can do?" into the device's input field.

[1182] Step 2:

[1183] The terminal sends the entered question to the server.

[1184] Specific operation: The device converts the entered question into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[1185] Step 3:

[1186] The server parses the received query.

[1187] Specific operation: The server inputs the question data into a generative artificial intelligence model, performs syntactic and semantic analysis, and clarifies the intent of the question.

[1188] Step 4:

[1189] The server generates appropriate health information or advice.

[1190] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[1191] Step 5:

[1192] The server generates a response and sends it back to the terminal.

[1193] Specific operation: The server returns the generated answer in JSON format to the terminal.

[1194] Step 6:

[1195] The terminal displays the received answer to the user.

[1196] Specific behavior: The device displays the answer it receives in the user interface, saying, "To create a relaxing routine, it's a good idea to create a good sleep environment."

[1197] Medical device monitoring system

[1198] Processing steps and specific operations

[1199] Step 1:

[1200] The medical device transmits data to the server in real time.

[1201] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[1202] Step 2:

[1203] The server parses the received data.

[1204] How it works: The server inputs data into a generative artificial intelligence model to detect abnormal values ​​and patterns.

[1205] Step 3:

[1206] The server evaluates the state based on the data.

[1207] Specific operation: The generative artificial intelligence model evaluates the patient's condition based on the analysis results and determines whether there are any abnormalities.

[1208] Step 4:

[1209] The server generates appropriate usage and warning messages.

[1210] Specific operation: If the server detects an abnormality, it generates a warning message such as, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[1211] Step 5:

[1212] The server sends the generated message to the terminal.

[1213] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[1214] Step 6:

[1215] The terminal displays the received message to the user.

[1216] Specific behavior: The device receives a message and displays it on the user interface, saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[1217] Home medical support system

[1218] Processing steps and specific operations

[1219] Step 1:

[1220] A user enters a home health care contact request.

[1221] Specific action: The user types into the device's input field, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[1222] Step 2:

[1223] The terminal sends a contact request to the server.

[1224] Specific operation: The terminal sends the entered contact request in JSON format to the server.

[1225] Step 3:

[1226] The server parses the contact request and the user information.

[1227] Specific operation: The server uses a generative artificial intelligence model to analyze the contact request and user information and evaluate the user's condition and symptoms.

[1228] Step 4:

[1229] The server identifies the appropriate medical staff.

[1230] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[1231] Step 5:

[1232] The server creates a link or method for establishing contact.

[1233] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[1234] Step 6:

[1235] The terminal displays the received contact information to the user.

[1236] Specific operation: The device displays the generated contact information in the user interface, displaying "Dr. Suzuki is available. Click here to contact him."

[1237] The above is a description of the specific processing steps and operations of each system.

[1238] Example 1

[1239] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1240] The current medical environment lacks a system that effectively supports telemedicine and home medical care. In particular, systems that can provide appropriate information and alerts in real time are needed for health consultations, medical device monitoring, and home medical care communications. This will improve convenience for patients and medical professionals and enable prompt and appropriate responses.

[1241] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1242] In this invention, the server includes a means for analyzing a user's health-related questions and generating appropriate health information or advice, a means for analyzing data from medical devices and detecting abnormalities, and a means for analyzing contact requests and user information and identifying appropriate medical professionals, thereby enabling a quick response to the user's health consultation, real-time detection of abnormalities in medical devices, and quick establishment of contact with appropriate medical professionals.

[1243] "User" refers to an individual or patient who uses the system to receive health consultations, monitor medical devices, or contact home medical care.

[1244] A "question" is a question that a user inputs about a health-related question or symptom and requests an answer.

[1245] A "generative artificial intelligence model" is an artificial intelligence system that analyzes input text and generates appropriate information and advice.

[1246] A "server" is a computer system that hosts a generative artificial intelligence model, analyzes questions sent by users and medical device data, and generates and returns appropriate information.

[1247] A "terminal" is a hardware device used by a user, such as a smartphone, tablet, or PC.

[1248] "Health information or advice" refers to information or recommended actions provided by a generative artificial intelligence model in response to a health consultation.

[1249] "Medical devices" are devices used to monitor a patient's health, such as heart rate monitors and blood pressure monitors.

[1250] "Data" refers to the patient's vital signs and measurement results collected in real time by medical devices.

[1251] An "abnormality" is a phenomenon that is determined as a measurement value or state outside the normal range as a result of the generative artificial intelligence model analyzing data.

[1252] A "contact request" is a request by a user to contact a medical professional regarding home medical care.

[1253] "Medical professionals" are professionals such as doctors and nurses who provide medical services and advice to users.

[1254] A "link or method" is a communication method or URL generated to allow a user to contact the appropriate healthcare professional.

[1255] A "warning message" is the warning message provided by the generative artificial intelligence model when an abnormality in a medical device is detected.

[1256] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system operates in cooperation with users, terminals, and servers to answer users' health-related questions, monitor medical equipment, and provide smooth support for home medical care.

[1257] Hardware and software used

[1258] Hardware: User devices (smartphones, tablets, PCs), medical devices (heart rate monitors, blood pressure monitors, etc.)

[1259] Software: Generative AI model on the server (e.g., GPT-3, BERT, etc.), communication protocol (HTTP / S)

[1260] Health consultation and information system

[1261] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[1262] Specific examples

[1263] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[1264] Medical device monitoring system

[1265] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[1266] Specific examples

[1267] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[1268] Home medical support system

[1269] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff. The server then generates links and instructions for establishing contact with the medical staff and displays them on the user's terminal.

[1270] Specific examples

[1271] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Medical Professional A is available. Please click here to contact him." The terminal then displays this message to the user.

[1272] Prompt Sentence Examples

[1273] "Please tell me what to do about not being able to sleep."

[1274] What should I do if my heart rate is high?

[1275] "I'm feeling nauseous, which doctor should I consult?"

[1276] As a result, the present invention utilizes generative artificial intelligence models to efficiently solve various issues in remote medical care and home medical care, providing an effective medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[1277] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1278] Health consultation and information system

[1279] Processing Steps

[1280] Step 1:

[1281] The user uses the terminal to input health-related questions.

[1282] Input: The text question entered by the user into the terminal

[1283] Output: The device is ready to send the question data to the server.

[1284] Step 2:

[1285] The terminal transmits the entered question to the server.

[1286] Input: Question data from the user

[1287] Output: Query data arrives at the server

[1288] Step 3:

[1289] The server passes the received question to a generative AI model and analyzes the content.

[1290] Input: Query data arriving at the server

[1291] Output: The results of the generative AI model's analysis of the question

[1292] Step 4:

[1293] The server generates appropriate health information and advice based on the analysis results returned by the generative AI model.

[1294] Input: Question content analyzed by the generative AI model

[1295] Output: Health information or advice

[1296] Step 5:

[1297] The server sends the generated response to the terminal.

[1298] Input: Generated health information or advice

[1299] Output: The device receives the data

[1300] Step 6:

[1301] The terminal displays the received response to the user.

[1302] Input: Health information or advice received from the server

[1303] Output: A screen display where users can view their answers

[1304] Medical device monitoring system

[1305] Processing Steps

[1306] Step 1:

[1307] Medical devices collect data in real time.

[1308] Input: Patient's vital signs (e.g., heart rate)

[1309] Output: Collected data

[1310] Step 2:

[1311] The medical device sends the collected data to a server.

[1312] Input: Data from medical devices

[1313] Output: Data arrives at the server

[1314] Step 3:

[1315] The server analyzes the received data using the generated AI model and checks for any abnormalities.

[1316] Input: Data arriving at the server

[1317] Output: Analysis results of the generative AI model (normal or abnormal)

[1318] Step 4:

[1319] If the server detects an abnormality, it will generate appropriate usage or warning messages.

[1320] Input: Analysis results of the generative AI model (in case of anomalies)

[1321] Output: Warning message

[1322] Step 5:

[1323] The server sends the generated warning message to the terminal of the user or medical staff.

[1324] Input: warning message

[1325] Output: The device receives the data

[1326] Step 6:

[1327] The terminal displays the received warning message to the user.

[1328] Input: The warning message received from the server

[1329] Output: A screen display where the user can view the warning

[1330] Home medical support system

[1331] Processing Steps

[1332] Step 1:

[1333] The user inputs a contact request for home medical care from a terminal.

[1334] Input: Text information entered by the user into the device

[1335] Output: The device is ready to send contact request data to the server

[1336] Step 2:

[1337] The terminal sends a contact request to the server.

[1338] Input: Contact request data from the user

[1339] Output: Contact request data arrives at the server

[1340] Step 3:

[1341] The server uses a generative AI model to analyze the contact request and user information to identify appropriate medical staff.

[1342] Input: Contact request data and user information received by the server

[1343] Output: Identification of appropriate medical staff

[1344] Step 4:

[1345] The server creates links and methods for establishing contact with medical staff.

[1346] Input: Identification of appropriate medical staff

[1347] Output: Means of contact (link or method)

[1348] Step 5:

[1349] The server sends the generated contact information to the user's terminal.

[1350] Input: Contact Method

[1351] Output: The device receives the data

[1352] Step 6:

[1353] The terminal displays the received contact information to the user.

[1354] Input: Contact information received from the server

[1355] Output: A screen display where the user can view contact information

[1356] Through these steps, the system can quickly respond to user questions and contact requests and monitor medical devices in real time.

[1357] (Application example 1)

[1358] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1359] The present invention relates to a system for more efficient and rapid health management using smart devices. Its purpose is to provide a means for users to quickly and easily ask health-related questions and receive appropriate advice while on the go. It also aims to provide a comprehensive telemedicine system that also includes medical equipment monitoring and home medical care support.

[1360] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1361] In this invention, the server includes means for receiving and analyzing voice input from the smart glasses, means for sending questions to a generative artificial intelligence model on the server and obtaining answers, and means for displaying the obtained answers on the display of the smart glasses, thereby enabling users to ask health-related questions in real time and receive appropriate advice even when they are out and about.

[1362] "User" means an individual or end user who uses the system.

[1363] "Health-related questions" refer to questions or concerns that a user may have about their own health condition or medical care.

[1364] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data from users and generates appropriate information and answers.

[1365] A "server" is a back-end system that hosts generative artificial intelligence models and processes data submitted by users.

[1366] "Terminal" refers to a device, such as smart glasses, a smartphone, a tablet, etc., that a user uses to send and receive input data.

[1367] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand the questions and data sent by the user and generate appropriate information.

[1368] "Smart glasses" are wearable devices that have a built-in display and can receive user input via voice or vision.

[1369] "Real-time" means that data is generated and analyzed almost simultaneously with user input.

[1370] "Medical device" refers to a hardware device used to monitor a user's physical condition, such as a heart rate monitor or blood pressure monitor.

[1371] A "display" is a display device for visually displaying generated information and answers.

[1372] A "link" is a means or method by which a user can easily contact designated medical personnel.

[1373] "Contact Request" means a notification or instruction for a User to seek home health care assistance.

[1374] The present invention provides a system that allows users to use smart glasses to ask health-related questions in real time and obtain appropriate health information and advice through a generative artificial intelligence model. Detailed embodiments for implementing this system are described below.

[1375] The system mainly consists of three main components: users, terminals, and servers.

[1376] 1. Users

[1377] As an end user of the system, a user wears smart glasses. They input health-related questions by voice, and the input is received by the microphone of the smart glasses. For example, they might ask, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?"

[1378] 2. Terminal (smart glasses)

[1379] The smart glasses have the following features:

[1380] Acquiring voice input: The smart glasses acquire the user's voice questions through a microphone.

[1381] Speech-to-text conversion: Convert the captured speech into text and send it to the server. This is done using a speech recognition library (for example, Python's speech_recognition library).

[1382] Communication with the server: The user's question is sent to the server and the answer is received from the server.

[1383] Display function: The answer received from the server is displayed on the smart glasses' display, and at the same time, feedback is given to the user via voice.

[1384] 3. Server

[1385] The server hosts the generative artificial intelligence model and has the following functions:

[1386] Question analysis: The text question sent by the user is analyzed and an appropriate answer is generated using a generative artificial intelligence model (e.g., GPT-3). The prompt sentence is the user's question, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?"

[1387] Answer generation: The generative artificial intelligence model generates appropriate health information and advice based on the analysis results and sends it back to the device.

[1388] Specific examples

[1389] In a specific usage scenario, a user wearing smart glasses asks a question while out and about: "If I consume caffeine before bed, I have trouble falling asleep. What should I do?" This question is picked up by the smart glasses' microphone and converted into text using a speech recognition library. The text question is sent to a server, where a generative artificial intelligence model generates an answer such as, "It's effective to avoid consuming caffeine before bed and to create a relaxing bedtime routine." The generated answer is sent back to the smart glasses and displayed on the user's display.

[1390] This allows users to solve health-related questions in real time and receive appropriate advice while on the go. Medical equipment monitoring and home medical care support are also analyzed on the server, and the necessary information is provided to users and medical staff.

[1391] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1392] Step 1:

[1393] The user puts on the smart glasses and speaks to ask a health-related question. The voice input is in the form of, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?" This voice data is picked up by the microphone device in the smart glasses.

[1394] Step 2:

[1395] The device (smart glasses) converts the acquired voice data into text data. This conversion is performed using a voice recognition library (for example, Python's speech_recognition library). The input is voice data, and the output is text data.

[1396] Step 3:

[1397] The terminal sends the user's question to the server in the form of text data. Here, the terminal sends the text data to the specified endpoint of the server using an HTTP POST request. The input is the text data, and the output is the request sent to the server.

[1398] Step 4:

[1399] The server passes the received text data to a generative artificial intelligence model and begins analyzing the question. This analysis is input to the model as a prompt sentence, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?" The input is text data (prompt sentence), and the output is answer data based on the analysis results.

[1400] Step 5:

[1401] A generative artificial intelligence model (e.g., GPT-3) analyzes the question and generates an appropriate answer. An example answer might be, "Headaches are a common side effect of medication, so we recommend consulting your doctor. Also, drink plenty of fluids and try to relax as much as possible." The input is the prompt, and the output is the generated answer data.

[1402] Step 6:

[1403] The server returns the answer data obtained from the generative AI model to the terminal. This process is performed as an HTTP response. The input is the generated answer data, and the output is the transmission of the response.

[1404] Step 7:

[1405] The terminal displays the received answer data on the display of the smart glasses. In addition, it also provides audio feedback to the user using a voice output library (e.g., Python's pyttsx3). The input is the answer data from the server, and the output is the display and audio output.

[1406] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1407] This invention relates to a telemedicine and home medical care support system that integrates a generative artificial intelligence model and an emotion engine. This system involves the user, terminal, and server working together to answer the user's health-related questions, monitor medical equipment, and provide seamless support for home medical care. This system also recognizes the user's emotional state and reflects this in its analysis and information provision, enabling more personalized medical support.

[1408] Health consultation and information system

[1409] overview

[1410] The user enters a health-related question into the device, and the question along with the user's emotional data detected by the emotion engine is sent to the server. A generative AI model on the server analyzes the question and emotional data and generates appropriate health information and advice. The generated information is sent back to the device and displayed to the user in a form adjusted based on the emotional data.

[1411] Specific examples

[1412] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects the user's level of stress from their facial expressions and tone of voice. The device then sends this question and emotional data to the server. The server uses a generative artificial intelligence model to analyze the question and emotional data and generate advice such as, "To create a good sleep environment, it is effective to limit your intake of caffeine and alcohol before bed and create a relaxing routine." The device then displays this advice to the user in a relaxed tone that takes into account the user's level of stress.

[1413] Medical device monitoring system

[1414] overview

[1415] Medical devices send data collected in real time to a server, where an emotion engine grasps the user's emotional state. A generative AI model on the server analyzes the data and emotion data, and generates appropriate usage instructions and warning messages if an abnormality is detected. This information is adjusted based on the emotion data and sent to and displayed on the user's or medical staff's device.

[1416] Specific examples

[1417] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. An emotion engine recognizes the user's current emotional state as stress. The server uses a generative artificial intelligence model to analyze the data and emotional information and generate a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is displayed to the user in a calm and gentle tone that takes into account the stressful state.

[1418] Home medical support system

[1419] overview

[1420] A user inputs a contact request for home medical care from a terminal, and the request and emotional data detected by the emotion engine are sent to the server. A generative artificial intelligence model on the server analyzes the contact request, emotional data, and user information to identify appropriate medical staff. It then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal in a format adjusted based on the emotional data.

[1421] Specific examples

[1422] The user types into the device, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to a doctor." The emotion engine detects anxiety from the user's facial expression. The device then sends this contact request and emotion data to the server. The server uses a generative artificial intelligence model to analyze the contact request and emotion information, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The device then displays this message to the user in a reassuring tone that takes into account the user's state of anxiety.

[1423] In this way, by integrating an emotion engine and a generative artificial intelligence model, this invention solves various issues in remote medical care and home medical care, and provides a personalized and efficient medical support system. By each module working together, it is possible to improve user convenience and enhance the quality of medical care.

[1424] The processing flow will be explained below.

[1425] Health consultation and information system

[1426] Processing steps and specific operations

[1427] Step 1:

[1428] The user enters a health question into the terminal.

[1429] Specific operation: The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?"

[1430] Step 2:

[1431] An emotion engine recognizes the user's emotional state.

[1432] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's stress level from their facial expressions and tone of voice.

[1433] Step 3:

[1434] The device sends the question and emotion data to the server.

[1435] Specific operation: The device converts the question and emotion data into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[1436] Step 4:

[1437] The server analyzes the question and sentiment data.

[1438] Specific operation: The server inputs the question and emotion data into the generative AI model, and performs syntactic and semantic analysis to clarify the intent and emotional state of the question.

[1439] Step 5:

[1440] The server generates appropriate health information or advice.

[1441] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[1442] Step 6:

[1443] The server generates a response and sends it back to the terminal.

[1444] Specific operation: The server returns the generated answer in JSON format to the terminal.

[1445] Step 7:

[1446] The terminal displays the received answer to the user.

[1447] What it does: The device adjusts the answer it receives based on the emotion data and displays it in the user interface, saying in a relaxed tone, "To ensure a good night's sleep, it's a good idea to create a relaxing routine."

[1448] Medical device monitoring system

[1449] Processing steps and specific operations

[1450] Step 1:

[1451] The medical device transmits data to the server in real time.

[1452] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[1453] Step 2:

[1454] An emotion engine recognizes the user's emotional state.

[1455] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of tension.

[1456] Step 3:

[1457] The server analyzes the received data and emotion data.

[1458] Specific operation: The server inputs data and emotional data into a generative artificial intelligence model to detect abnormal values, patterns, and emotional states.

[1459] Step 4:

[1460] The server evaluates the state based on the data.

[1461] Specific operation: Based on the analysis results, the generative artificial intelligence model evaluates the patient's condition and determines whether there are any abnormalities.

[1462] Step 5:

[1463] The server generates appropriate usage and warning messages.

[1464] Specific operation: If the server detects an abnormality, it generates a message saying, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[1465] Step 6:

[1466] The server sends the generated message to the terminal.

[1467] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[1468] Step 7:

[1469] The terminal displays the received message to the user.

[1470] Specific behavior: The device adjusts the warning message received based on the emotional data and displays it on the user interface, in a calm and gentle tone: "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[1471] Home medical support system

[1472] Processing steps and specific operations

[1473] Step 1:

[1474] A user enters a home health care contact request.

[1475] Specific operation: The user types into the terminal, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[1476] Step 2:

[1477] An emotion engine recognizes the user's emotional state.

[1478] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of anxiety.

[1479] Step 3:

[1480] The terminal transmits a contact request and emotion data to the server.

[1481] Specific operation: The device sends a contact request and emotion data in JSON format to the server.

[1482] Step 4:

[1483] The server analyzes the contact request and the emotion data.

[1484] Specific operation: The server inputs contact request and emotion data into the generative AI model, which evaluates the user's state and emotion.

[1485] Step 5:

[1486] The server identifies the appropriate medical staff.

[1487] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[1488] Step 6:

[1489] The server creates a link or method for establishing contact.

[1490] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[1491] Step 7:

[1492] The terminal displays the received contact information to the user.

[1493] What it does: The device adjusts the generated contact information based on the emotion data and displays it in the user interface, using a reassuring tone to say, "Dr. Suzuki is available. Click here to contact him."

[1494] The above is a description of the specific processing steps and operations of each system.

[1495] Example 2

[1496] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1497] In telemedicine and home medical care, there are issues with the efficient provision of information to users regarding their health, monitoring medical devices, and supporting home medical care. In particular, the lack of personalized information that takes into account the user's emotional state can lead to a decrease in user satisfaction.

[1498] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for sending a user's health-related question to a server hosting a generative artificial intelligence model; means for analyzing the question and emotion data on the server and generating appropriate health information or advice based on the question; means for adjusting the generated information or advice based on the emotion data and displaying it on the user's terminal; means for sending data from a medical device to the server; means for analyzing the data and emotion data and detecting abnormalities; means for generating appropriate usage instructions or warning messages adjusted based on the emotion data and sending and displaying the messages to the user's or medical staff's terminal; means for the user to send a home medical care contact request to the server; means for analyzing the contact request and user information using the generative artificial intelligence model and identifying appropriate medical staff; and means for generating a link or method for establishing contact with medical staff, adjusting the generated contact information based on the emotion data, and displaying it on the user's terminal. This enables personalized information provision that takes the user's emotional state into consideration.

[1499] "User" refers to an individual or organization that uses this system.

[1500] "Terminal" refers to a computing device used by a user, including a PC, smartphone, tablet, etc.

[1501] "Server" means the computer system hosting the generative artificial intelligence model and emotion engine, and is responsible for receiving, analyzing, and processing data sent by users.

[1502] A "generative artificial intelligence model" is a model that uses algorithms learned from large amounts of data to perform tasks such as natural language processing.

[1503] An "emotion engine" refers to a software or hardware system that analyzes a user's emotional state from facial expressions, tone of voice, etc., and acquires the data.

[1504] "Health information" refers to information related to the user's health condition, including preventative measures, improvement measures, and appropriate guidelines for action.

[1505] "Medical device" refers to a device for monitoring a user's health condition, including heart rate monitors, blood pressure monitors, and other vital signs monitoring devices.

[1506] "Emotion data" is data that indicates the emotional state of the user obtained by the emotion engine.

[1507] A "contact request" is a request that a user inputs into the system to the effect that they wish to be contacted by medical staff.

[1508] "Appropriate Medical Staff" refers to medical professionals selected to respond to a user's contact requests.

[1509] This invention relates to a telemedicine and home medical support system that integrates a generative artificial intelligence model and an emotion engine. This system works in conjunction with the user, terminal, and server elements to answer the user's health-related questions, monitor medical equipment, and provide smooth support for home medical care. This system also recognizes the user's emotional state and reflects it in the analysis and information provided, thereby achieving more personalized medical support.

[1510] Health consultation and information system

[1511] First, the user inputs a health-related question into the device's input screen. For example, "I haven't been able to sleep lately. Is there anything I can do about it?" At this stage, the device uses its built-in camera and microphone to detect the user's facial expressions and tone of voice and collect emotional data. The question and emotional data are then sent from the device to the server.

[1512] The server analyzes the received questions and emotional data using a generative artificial intelligence model (e.g., GPT-4) to generate appropriate health information and advice. For example, it might generate advice such as, "To ensure a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The generated information is returned to the device in a tone adjusted based on the emotional data, and the device displays it to the user.

[1513] Prompt Sentence Examples

[1514] The user types, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects stress and generates advice taking into account health information and stress levels, and displays it in a relaxed tone.

[1515] Medical device monitoring system

[1516] The medical device automatically collects real-time data such as heart rate and blood pressure. This data is sent from the medical device to a server, and the server uses an emotion engine to grasp the user's emotional state in real time. For example, if the user is in a state of tension, that information is also collected.

[1517] The server uses a generative artificial intelligence model to analyze the medical and emotional data to detect any abnormalities. If an abnormality is detected, the server generates a warning message such as, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is adjusted based on the emotional data and sent to the device in a calm and gentle tone, and displayed to the user.

[1518] Prompt Sentence Examples

[1519] A medical device collects heart rate data. An emotion engine detects the user's tension and generates and displays a calm message informing them of abnormal heart rates.

[1520] Home medical support system

[1521] The user inputs a request for contact regarding home medical care into the device. For example, "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." At this time, the device uses an emotion engine to detect the user's anxiety and collect emotional data.

[1522] The input contact request and emotional data are sent from the device to a server, which then uses a generative artificial intelligence model to analyze the contact request and emotional information and identify the appropriate medical staff. For example, a message such as "Dr. Suzuki is available. Please click here to contact him" is generated. This message is adjusted based on the emotional data and sent to the device and displayed to the user in a reassuring tone to ease anxiety.

[1523] Prompt Sentence Examples

[1524] A user types in "Nausea, thought to be a side effect of medication." Anxiety is detected. Contact instructions for a doctor are generated and displayed in a reassuring tone.

[1525] As described above, by integrating an emotion engine and a generative AI model, this invention can solve various issues in remote medical care and home medical care and provide a personalized and efficient medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[1526] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1527] Health consultation and information system

[1528] Step 1:

[1529] The user enters a question.

[1530] Specific operation: The user accesses the device's input screen and enters a health-related question such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[1531] Input: Question

[1532] Output: Question data saved on the device

[1533] Step 2:

[1534] The device collects emotional data.

[1535] Specific operation: Using the device's camera and microphone, the system detects the user's facial expressions and tone of voice, and obtains emotional data such as stress and anxiety.

[1536] Input: User's facial expressions and tone of voice

[1537] Output: Obtained emotion data

[1538] Step 3:

[1539] The question and emotion data are sent to the server.

[1540] Specific operation: The device sends the entered question and detected emotion data to the server via an HTTP request.

[1541] Input: Question data, emotion data

[1542] Output: Question data and emotion data stored on the server

[1543] Step 4:

[1544] The server analyzes the data.

[1545] Specific operation: The server inputs the received question content and emotional data into a generative AI model (e.g., GPT-4) to generate appropriate health information and advice.

[1546] Input: Question data, emotion data

[1547] Output: Generated health information and advice

[1548] Step 5:

[1549] The generated information is returned to the terminal.

[1550] Specific operation: The server returns the generated health information and advice to the device in a tone adjusted based on the emotional data.

[1551] Input: Generated health information, advice, and emotional data

[1552] Output: Tailored health information and advice sent.

[1553] Step 6:

[1554] The terminal displays the information to the user.

[1555] Specific behavior: The device displays the information it receives to the user in a relaxed tone.

[1556] Input: Tailored health information and advice

[1557] Output: What the user sees on the screen

[1558] Medical device monitoring system

[1559] Step 1:

[1560] Medical devices collect data.

[1561] What it does: Medical devices automatically collect information like heart rate and blood pressure in real time.

[1562] Input: User biometric data (heart rate, blood pressure, etc.)

[1563] Output: Collected biometric data

[1564] Step 2:

[1565] The medical device sends the data to the server.

[1566] Specific operation: Medical devices send collected data in real time to a server.

[1567] Input: Biometric data

[1568] Output: Biometric data stored on the server

[1569] Step 3:

[1570] The server captures the emotion data.

[1571] Specific operation: The server uses an emotion engine to analyze the received biometric data and the user's emotional state (e.g., tension) in real time.

[1572] Input: Biometric data, emotional state

[1573] Output: Emotion data

[1574] Step 4:

[1575] The server analyzes the data.

[1576] How it works: The server uses the generative AI model to analyze biometric and emotional data to detect any anomalies.

[1577] Input: Biometric data, emotional data

[1578] Output: Anomaly detection results

[1579] Step 5:

[1580] If an anomaly is detected, a message is generated.

[1581] Specific behavior: If the server detects an abnormality, it generates a warning message such as "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[1582] Input: Anomaly detection results, emotion data

[1583] Output: Warning message

[1584] Step 6:

[1585] Send the generated message.

[1586] Specific behavior: The server sends the generated message to the terminal in a calm and gentle tone.

[1587] Input: warning message

[1588] Output: Adjusted warning message sent

[1589] Step 7:

[1590] The terminal displays the message.

[1591] Specific behavior: Display the message received by the device to the user.

[1592] Input: Adjusted warning message

[1593] Output: The message the user sees on the screen

[1594] Home medical support system

[1595] Step 1:

[1596] A user enters a contact request.

[1597] Specific operation: The user inputs a contact request into the terminal, saying, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to speak to a doctor."

[1598] Input: Contact Request

[1599] Output: Contact request data stored on the device

[1600] Step 2:

[1601] The device collects emotional data.

[1602] Specific operation: The device uses an emotion engine to detect the user's anxiety and collect emotion data.

[1603] Input: User's facial expressions and tone of voice

[1604] Output: Collected emotion data

[1605] Step 3:

[1606] A contact request and emotion data are sent to the server.

[1607] Specific operation: The device sends a contact request and emotion data to the server.

[1608] Input: Contact request data, emotion data

[1609] Output: Contact request data and emotion data stored on the server

[1610] Step 4:

[1611] The server analyzes the data.

[1612] What it does: The server uses a generative AI model to analyze contact requests and emotional data to identify appropriate medical staff.

[1613] Input: Contact request data, emotion data

[1614] Output: Identified medical staff information

[1615] Step 5:

[1616] Generate contact methods.

[1617] Specific operation: The server generates a method of contacting the identified medical staff (e.g., a link where the medical staff is available).

[1618] Input: Medical staff information, emotion data

[1619] Output: Generated contact information

[1620] Step 6:

[1621] The generated information is transmitted.

[1622] What it does: The server sends the generated information to the device in a reassuring tone.

[1623] Input: Generated contact information

[1624] Output: Coordinated contact information sent

[1625] Step 7:

[1626] The terminal displays the information to the user.

[1627] Specific behavior: The device displays the information it receives to the user and provides links and methods for establishing contact with medical staff.

[1628] Input: Adjusted contact information

[1629] Output: What the user sees on the screen

[1630] (Application example 2)

[1631] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1632] In telemedicine and home medical care, there is a need to provide appropriate and personalized medical support that takes into account the user's emotional state. However, conventional systems can only generate uniform answers or warning messages in response to the user's health questions or data from medical devices, and lack feedback that reflects the user's emotional state. Therefore, a system is needed that can alleviate users' anxieties and doubts and provide more effective medical support.

[1633] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1634] In this invention, the server includes emotion recognition means for sensing and analyzing the user's emotional state, means for analyzing questions and data using a generative artificial intelligence model to generate appropriate information and advice, and means for adjusting the tone and content of the information and advice based on the emotion data, thereby enabling personalized medical information and advice according to the user's emotional state.

[1635] The "means for inputting questions about the user's health" is an interface that allows the user to input questions about their own health condition or symptoms into the terminal.

[1636] The "means for sending to a server hosting a generative artificial intelligence model" is a means having the function of transferring a question entered by a user to a server hosting a generative artificial intelligence model.

[1637] A "generative artificial intelligence model" is an artificial intelligence model that performs natural language processing and data analysis to generate appropriate information and advice based on user input.

[1638] The "means for returning generated information or advice to the user's terminal" refers to a means having a function for returning generated information or advice from the server to the user's terminal.

[1639] The "means for displaying the information or advice" refers to a means having a function for visually displaying the information or advice generated on the user's terminal.

[1640] The "emotion recognition means" is a means for detecting and analyzing emotional data such as the user's facial expressions and tone of voice.

[1641] "Emotion data" is data that indicates the current emotional state of the user, and is acquired by emotion recognition means.

[1642] "Means for adjusting the tone and content of information and advice based on emotional data" refers to means for changing the method of delivery and content of information and advice generated using emotional data to suit the emotional state of the user.

[1643] The "means for collecting data from a medical device in real time and transmitting the data to a server" refers to a means for transferring data acquired from a medical device to a server in real time.

[1644] "Means for generating appropriate usage instructions or warning messages when an abnormality is detected" refers to means for analyzing data from medical devices and generating appropriate response instructions or messages urging caution when an abnormality is detected.

[1645] The "means for the user to input a request for contact regarding home medical care" is an interface for the user to input a request into a terminal when the user desires to be contacted regarding home medical care.

[1646] The "means for sending a contact request to a server" is a means for transferring a contact request regarding home medical care input by a user to a server.

[1647] The "means for identifying appropriate medical staff" is a means for analyzing the contact request and user information and selecting appropriate medical staff.

[1648] "Means for creating a link or method for establishing contact" refers to a means for creating a link or method for establishing contact with medical staff.

[1649] The "means for displaying contact information on the user's terminal" refers to a means for visually displaying the generated contact information on the user's terminal.

[1650] As an embodiment of the present invention, the following system program, hardware, and software configuration will be described.

[1651] Hardware and Software Configuration

[1652] The system of the present invention includes smart glasses, a camera, a server, a terminal, a generative artificial intelligence model, and an emotion recognition engine. The smart glasses have a built-in camera that captures the user's facial expressions in real time. The server hosts the generative artificial intelligence model (generative AI model) and the emotion recognition engine, which analyzes the user's emotional state.

[1653] Program Processing Overview

[1654] 1. Enter your question:

[1655] The user inputs a health-related question through the smart glasses, such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[1656] 2. Collecting Emotional Data:

[1657] The smart glasses have a built-in camera that captures the user's facial expressions and body movements, and the emotion recognition engine analyzes the data. In this case, TensorFlow is used to build the emotion recognition model.

[1658] 3. Data transmission and analysis:

[1659] The question and emotion data are sent to a server, where a generative AI model (e.g., GPT-3) analyzes the question and emotion data and generates appropriate health information and advice.

[1660] 4. Coordination and return of information:

[1661] The content and tone of the generated information and advice are adjusted based on the emotional data and sent back to the user's terminal.

[1662] 5. Displaying information:

[1663] Tailored information and advice will be displayed on the user's device, such as "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and to create a relaxing routine," in a relaxing tone.

[1664] Software used

[1665] TensorFlow: Used to build an emotion recognition engine

[1666] GPT-3: Generative AI model

[1667] Specific examples

[1668] To use a specific example, the following situation can be considered.

[1669] A user uses smart glasses in a virtual store and stops by the health food section. He or she wonders, "Which health food is right for me?" The emotion recognition engine detects doubt from the user's facial expressions and gestures, and sends the data to the server. The generative AI model on the server analyzes the situation—"The user is interested, but has doubts about which health food is right for me"—and generates the following example prompt:

[1670] Example 1: "User is feeling curious. Recommend a health product that suits this emotion in the context of energy supplements."

[1671] Example 2: "User feels doubtful. Suggest a health product for daily vitamin intake to relieve this emotion in a virtual store."

[1672] In this way, personalized information tailored to the user's emotional state is provided, resulting in a more satisfying shopping experience.

[1673] summary

[1674] The overall process of this system recognizes the user's emotional state in real time and provides personalized information and advice accordingly. The hardware and software components used are clearly indicated to demonstrate the implementation of the invention, enabling it to provide appropriate and efficient assistance tailored to the user's needs.

[1675] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1676] Step 1:

[1677] A user puts on the smart glasses and inputs a health question.

[1678] The input is sent to the server via the smart glasses interface. This input data includes the question in text format. For example, "I haven't been able to sleep lately. Is there anything I can do about it?"

[1679] Step 2:

[1680] The emotion recognition means captures the user's facial expressions and body movements in real time from a camera built into the smart glasses.

[1681] Facial expression data and physical movement data captured by the camera are sent to an emotion recognition engine using TensorFlow, which analyzes the user's emotional state.

[1682] As an output, the analysis produces emotional data, e.g., stress, doubt, etc.

[1683] Step 3:

[1684] The terminal transmits question data and emotion data to the server.

[1685] The transmitted data includes the user's question text data and emotion data generated by the emotion recognition engine.

[1686] The server receives this data.

[1687] Step 4:

[1688] A generative AI model (GPT-3) on the server analyzes question data and emotion data.

[1689] Specifically, a prompt is generated based on the question and emotion data. For example, "User is feeling stressed and asked about sleep issues. Provide advice to help relax before sleep."

[1690] The generative AI model generates health information and advice based on this prompt, for example, "To ensure a good night's sleep, it's helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[1691] Step 5:

[1692] The content and tone of the information and advice generated are adjusted based on the emotional data.

[1693] Example of adjustment: For a user in a stressed state, sentences are generated in a relaxed tone.

[1694] Step 6:

[1695] The server sends tailored information and advice back to the device.

[1696] The output data includes the final information and advice that will be displayed to the user, for example, "To ensure a good night's sleep, it is helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[1697] Step 7:

[1698] The terminal displays the received information and advice to the user.

[1699] Tailored information and advice is visually presented on a display within the smart glasses.

[1700] This allows users to quickly access appropriate health information and advice based on their emotional state.

[1701] Through this process, users can receive personalized health information and advice tailored to their emotional state, resulting in a more satisfying experience.

[1702] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1703] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1704] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1705] [Fourth embodiment]

[1706] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1707] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1708] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1709] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1710] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1711] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1712] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1713] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1714] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1715] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1716] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1717] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1718] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1719] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system involves the user, terminal, and server working together to answer questions about the user's health, monitor medical equipment, and provide smooth support for home medical care.

[1720] Health consultation and information system

[1721] overview

[1722] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[1723] Specific examples

[1724] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[1725] Medical device monitoring system

[1726] overview

[1727] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[1728] Specific examples

[1729] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[1730] Home medical support system

[1731] overview

[1732] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff, and the server then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal.

[1733] Specific examples

[1734] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The terminal then displays this message to the user.

[1735] In this way, the present invention provides an efficient and effective medical support system that utilizes generative artificial intelligence models to solve various problems in remote medical care and home medical care. By having each module work in cooperation with each other, it is possible to improve user convenience and enhance the quality of medical care.

[1736] The processing flow will be explained below.

[1737] Health consultation and information system

[1738] Processing steps and specific operations

[1739] Step 1:

[1740] The user enters a health question into the terminal.

[1741] Specific behavior: The user types "I've been having trouble sleeping lately. Is there anything I can do?" into the device's input field.

[1742] Step 2:

[1743] The terminal sends the entered question to the server.

[1744] Specific operation: The device converts the entered question into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[1745] Step 3:

[1746] The server parses the received query.

[1747] Specific operation: The server inputs the question data into a generative artificial intelligence model, performs syntactic and semantic analysis, and clarifies the intent of the question.

[1748] Step 4:

[1749] The server generates appropriate health information or advice.

[1750] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[1751] Step 5:

[1752] The server generates a response and sends it back to the terminal.

[1753] Specific operation: The server returns the generated answer in JSON format to the terminal.

[1754] Step 6:

[1755] The terminal displays the received answer to the user.

[1756] Specific behavior: The device displays the answer it receives in the user interface, saying, "To create a relaxing routine, it's a good idea to create a good sleep environment."

[1757] Medical device monitoring system

[1758] Processing steps and specific operations

[1759] Step 1:

[1760] The medical device transmits data to the server in real time.

[1761] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[1762] Step 2:

[1763] The server parses the received data.

[1764] How it works: The server inputs data into a generative artificial intelligence model to detect abnormal values ​​and patterns.

[1765] Step 3:

[1766] The server evaluates the state based on the data.

[1767] Specific operation: The generative artificial intelligence model evaluates the patient's condition based on the analysis results and determines whether there are any abnormalities.

[1768] Step 4:

[1769] The server generates appropriate usage and warning messages.

[1770] Specific operation: If the server detects an abnormality, it generates a warning message such as, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[1771] Step 5:

[1772] The server sends the generated message to the terminal.

[1773] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[1774] Step 6:

[1775] The terminal displays the received message to the user.

[1776] Specific behavior: The device receives a message and displays it on the user interface, saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[1777] Home medical support system

[1778] Processing steps and specific operations

[1779] Step 1:

[1780] A user enters a home health care contact request.

[1781] Specific action: The user types into the device's input field, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[1782] Step 2:

[1783] The terminal sends a contact request to the server.

[1784] Specific operation: The terminal sends the entered contact request in JSON format to the server.

[1785] Step 3:

[1786] The server parses the contact request and the user information.

[1787] Specific operation: The server uses a generative artificial intelligence model to analyze the contact request and user information and evaluate the user's condition and symptoms.

[1788] Step 4:

[1789] The server identifies the appropriate medical staff.

[1790] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[1791] Step 5:

[1792] The server creates a link or method for establishing contact.

[1793] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[1794] Step 6:

[1795] The terminal displays the received contact information to the user.

[1796] Specific operation: The device displays the generated contact information in the user interface, displaying "Dr. Suzuki is available. Click here to contact him."

[1797] The above is a description of the specific processing steps and operations of each system.

[1798] Example 1

[1799] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1800] The current medical environment lacks a system that effectively supports telemedicine and home medical care. In particular, systems that can provide appropriate information and alerts in real time are needed for health consultations, medical device monitoring, and home medical care communications. This will improve convenience for patients and medical professionals and enable prompt and appropriate responses.

[1801] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1802] In this invention, the server includes a means for analyzing a user's health-related questions and generating appropriate health information or advice, a means for analyzing data from medical devices and detecting abnormalities, and a means for analyzing contact requests and user information and identifying appropriate medical professionals, thereby enabling a quick response to the user's health consultation, real-time detection of abnormalities in medical devices, and quick establishment of contact with appropriate medical professionals.

[1803] "User" refers to an individual or patient who uses the system to receive health consultations, monitor medical devices, or contact home medical care.

[1804] A "question" is a question that a user inputs about a health-related question or symptom and requests an answer.

[1805] A "generative artificial intelligence model" is an artificial intelligence system that analyzes input text and generates appropriate information and advice.

[1806] A "server" is a computer system that hosts a generative artificial intelligence model, analyzes questions sent by users and medical device data, and generates and returns appropriate information.

[1807] A "terminal" is a hardware device used by a user, such as a smartphone, tablet, or PC.

[1808] "Health information or advice" refers to information or recommended actions provided by a generative artificial intelligence model in response to a health consultation.

[1809] "Medical devices" are devices used to monitor a patient's health, such as heart rate monitors and blood pressure monitors.

[1810] "Data" refers to the patient's vital signs and measurement results collected in real time by medical devices.

[1811] An "abnormality" is a phenomenon that is determined as a measurement value or state outside the normal range as a result of the generative artificial intelligence model analyzing data.

[1812] A "contact request" is a request by a user to contact a medical professional regarding home medical care.

[1813] "Medical professionals" are professionals such as doctors and nurses who provide medical services and advice to users.

[1814] A "link or method" is a communication method or URL generated to allow a user to contact the appropriate healthcare professional.

[1815] A "warning message" is the warning message provided by the generative artificial intelligence model when an abnormality in a medical device is detected.

[1816] This invention relates to a telemedicine and home medical care support system that utilizes generative artificial intelligence models. This system operates in cooperation with users, terminals, and servers to answer users' health-related questions, monitor medical equipment, and provide smooth support for home medical care.

[1817] Hardware and software used

[1818] Hardware: User devices (smartphones, tablets, PCs), medical devices (heart rate monitors, blood pressure monitors, etc.)

[1819] Software: Generative AI model on the server (e.g., GPT-3, BERT, etc.), communication protocol (HTTP / S)

[1820] Health consultation and information system

[1821] When a user inputs a health-related question into a device and sends the question to a server, a generative AI model on the server analyzes the question and generates appropriate health information and advice. The generated information is then sent back to the device and displayed to the user.

[1822] Specific examples

[1823] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The device sends this question to the server. The server uses a generative artificial intelligence model to analyze the question and generate an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The device then displays this answer to the user.

[1824] Medical device monitoring system

[1825] Medical devices collect data in real time and send it to a server, where a generative artificial intelligence model analyzes the data. If an abnormality is detected, appropriate usage instructions or warning messages are generated and sent to the user or medical staff's device.

[1826] Specific examples

[1827] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. The server uses a generative artificial intelligence model to detect the abnormality and generates a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." The device then displays this message to the patient.

[1828] Home medical support system

[1829] When a user inputs a contact request for home medical care from a terminal and sends the request to the server, a generative artificial intelligence model on the server analyzes the contact request and user information to identify appropriate medical staff. The server then generates links and instructions for establishing contact with the medical staff and displays them on the user's terminal.

[1830] Specific examples

[1831] The user types and sends the following message into the terminal: "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." The server analyzes the user's symptoms, identifies the appropriate medical staff member, and generates a message saying, "Medical Professional A is available. Please click here to contact him." The terminal then displays this message to the user.

[1832] Prompt Sentence Examples

[1833] "Please tell me what to do about not being able to sleep."

[1834] What should I do if my heart rate is high?

[1835] "I'm feeling nauseous, which doctor should I consult?"

[1836] As a result, the present invention utilizes generative artificial intelligence models to efficiently solve various issues in remote medical care and home medical care, providing an effective medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[1837] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1838] Health consultation and information system

[1839] Processing Steps

[1840] Step 1:

[1841] The user uses the terminal to input health-related questions.

[1842] Input: The text question entered by the user into the terminal

[1843] Output: The device is ready to send the question data to the server.

[1844] Step 2:

[1845] The terminal transmits the entered question to the server.

[1846] Input: Question data from the user

[1847] Output: Query data arrives at the server

[1848] Step 3:

[1849] The server passes the received question to a generative AI model and analyzes the content.

[1850] Input: Query data arriving at the server

[1851] Output: The results of the generative AI model's analysis of the question

[1852] Step 4:

[1853] The server generates appropriate health information and advice based on the analysis results returned by the generative AI model.

[1854] Input: Question content analyzed by the generative AI model

[1855] Output: Health information or advice

[1856] Step 5:

[1857] The server sends the generated response to the terminal.

[1858] Input: Generated health information or advice

[1859] Output: The device receives the data

[1860] Step 6:

[1861] The terminal displays the received response to the user.

[1862] Input: Health information or advice received from the server

[1863] Output: A screen display where users can view their answers

[1864] Medical device monitoring system

[1865] Processing Steps

[1866] Step 1:

[1867] Medical devices collect data in real time.

[1868] Input: Patient's vital signs (e.g., heart rate)

[1869] Output: Collected data

[1870] Step 2:

[1871] The medical device sends the collected data to a server.

[1872] Input: Data from medical devices

[1873] Output: Data arrives at the server

[1874] Step 3:

[1875] The server analyzes the received data using the generated AI model and checks for any abnormalities.

[1876] Input: Data arriving at the server

[1877] Output: Analysis results of the generative AI model (normal or abnormal)

[1878] Step 4:

[1879] If the server detects an abnormality, it will generate appropriate usage or warning messages.

[1880] Input: Analysis results of the generative AI model (in case of anomalies)

[1881] Output: Warning message

[1882] Step 5:

[1883] The server sends the generated warning message to the terminal of the user or medical staff.

[1884] Input: warning message

[1885] Output: The device receives the data

[1886] Step 6:

[1887] The terminal displays the received warning message to the user.

[1888] Input: The warning message received from the server

[1889] Output: A screen display where the user can view the warning

[1890] Home medical support system

[1891] Processing Steps

[1892] Step 1:

[1893] The user inputs a contact request for home medical care from a terminal.

[1894] Input: Text information entered by the user into the device

[1895] Output: The device is ready to send contact request data to the server

[1896] Step 2:

[1897] The terminal sends a contact request to the server.

[1898] Input: Contact request data from the user

[1899] Output: Contact request data arrives at the server

[1900] Step 3:

[1901] The server uses a generative AI model to analyze the contact request and user information to identify appropriate medical staff.

[1902] Input: Contact request data and user information received by the server

[1903] Output: Identification of appropriate medical staff

[1904] Step 4:

[1905] The server creates links and methods for establishing contact with medical staff.

[1906] Input: Identification of appropriate medical staff

[1907] Output: Means of contact (link or method)

[1908] Step 5:

[1909] The server sends the generated contact information to the user's terminal.

[1910] Input: Contact Method

[1911] Output: The device receives the data

[1912] Step 6:

[1913] The terminal displays the received contact information to the user.

[1914] Input: Contact information received from the server

[1915] Output: A screen display where the user can view contact information

[1916] Through these steps, the system can quickly respond to user questions and contact requests and monitor medical devices in real time.

[1917] (Application example 1)

[1918] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1919] The present invention relates to a system for more efficient and rapid health management using smart devices. Its purpose is to provide a means for users to quickly and easily ask health-related questions and receive appropriate advice while on the go. It also aims to provide a comprehensive telemedicine system that also includes medical equipment monitoring and home medical care support.

[1920] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1921] In this invention, the server includes means for receiving and analyzing voice input from the smart glasses, means for sending questions to a generative artificial intelligence model on the server and obtaining answers, and means for displaying the obtained answers on the display of the smart glasses, thereby enabling users to ask health-related questions in real time and receive appropriate advice even when they are out and about.

[1922] "User" means an individual or end user who uses the system.

[1923] "Health-related questions" refer to questions or concerns that a user may have about their own health condition or medical care.

[1924] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes input data from users and generates appropriate information and answers.

[1925] A "server" is a back-end system that hosts generative artificial intelligence models and processes data submitted by users.

[1926] "Terminal" refers to a device, such as smart glasses, a smartphone, a tablet, etc., that a user uses to send and receive input data.

[1927] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand the questions and data sent by the user and generate appropriate information.

[1928] "Smart glasses" are wearable devices that have a built-in display and can receive user input via voice or vision.

[1929] "Real-time" means that data is generated and analyzed almost simultaneously with user input.

[1930] "Medical device" refers to a hardware device used to monitor a user's physical condition, such as a heart rate monitor or blood pressure monitor.

[1931] A "display" is a display device for visually displaying generated information and answers.

[1932] A "link" is a means or method by which a user can easily contact designated medical personnel.

[1933] "Contact Request" means a notification or instruction for a User to seek home health care assistance.

[1934] The present invention provides a system that allows users to use smart glasses to ask health-related questions in real time and obtain appropriate health information and advice through a generative artificial intelligence model. Detailed embodiments for implementing this system are described below.

[1935] The system mainly consists of three main components: users, terminals, and servers.

[1936] 1. Users

[1937] As an end user of the system, a user wears smart glasses. They input health-related questions by voice, and the input is received by the microphone of the smart glasses. For example, they might ask, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?"

[1938] 2. Terminal (smart glasses)

[1939] The smart glasses have the following features:

[1940] Acquiring voice input: The smart glasses acquire the user's voice questions through a microphone.

[1941] Speech-to-text conversion: Convert the captured speech into text and send it to the server. This is done using a speech recognition library (for example, Python's speech_recognition library).

[1942] Communication with the server: The user's question is sent to the server and the answer is received from the server.

[1943] Display function: The answer received from the server is displayed on the smart glasses' display, and at the same time, feedback is given to the user via voice.

[1944] 3. Server

[1945] The server hosts the generative artificial intelligence model and has the following functions:

[1946] Question analysis: The text question sent by the user is analyzed and an appropriate answer is generated using a generative artificial intelligence model (e.g., GPT-3). The prompt sentence is the user's question, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?"

[1947] Answer generation: The generative artificial intelligence model generates appropriate health information and advice based on the analysis results and sends it back to the device.

[1948] Specific examples

[1949] In a specific usage scenario, a user wearing smart glasses asks a question while out and about: "If I consume caffeine before bed, I have trouble falling asleep. What should I do?" This question is picked up by the smart glasses' microphone and converted into text using a speech recognition library. The text question is sent to a server, where a generative artificial intelligence model generates an answer such as, "It's effective to avoid consuming caffeine before bed and to create a relaxing bedtime routine." The generated answer is sent back to the smart glasses and displayed on the user's display.

[1950] This allows users to solve health-related questions in real time and receive appropriate advice while on the go. Medical equipment monitoring and home medical care support are also analyzed on the server, and the necessary information is provided to users and medical staff.

[1951] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1952] Step 1:

[1953] The user puts on the smart glasses and speaks to ask a health-related question. The voice input is in the form of, "I recently started taking a new medicine, but I'm having frequent headaches. What should I do?" This voice data is picked up by the microphone device in the smart glasses.

[1954] Step 2:

[1955] The device (smart glasses) converts the acquired voice data into text data. This conversion is performed using a voice recognition library (for example, Python's speech_recognition library). The input is voice data, and the output is text data.

[1956] Step 3:

[1957] The terminal sends the user's question to the server in the form of text data. Here, the terminal sends the text data to the specified endpoint of the server using an HTTP POST request. The input is the text data, and the output is the request sent to the server.

[1958] Step 4:

[1959] The server passes the received text data to a generative artificial intelligence model and begins analyzing the question. This analysis is input to the model as a prompt sentence, such as, "I recently started taking a new medication, but I'm having frequent headaches. What should I do?" The input is text data (prompt sentence), and the output is answer data based on the analysis results.

[1960] Step 5:

[1961] A generative artificial intelligence model (e.g., GPT-3) analyzes the question and generates an appropriate answer. An example answer might be, "Headaches are a common side effect of medication, so we recommend consulting your doctor. Also, drink plenty of fluids and try to relax as much as possible." The input is the prompt, and the output is the generated answer data.

[1962] Step 6:

[1963] The server returns the answer data obtained from the generative AI model to the terminal. This process is performed as an HTTP response. The input is the generated answer data, and the output is the transmission of the response.

[1964] Step 7:

[1965] The terminal displays the received answer data on the display of the smart glasses. In addition, it also provides audio feedback to the user using a voice output library (e.g., Python's pyttsx3). The input is the answer data from the server, and the output is the display and audio output.

[1966] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1967] This invention relates to a telemedicine and home medical care support system that integrates a generative artificial intelligence model and an emotion engine. This system involves the user, terminal, and server working together to answer the user's health-related questions, monitor medical equipment, and provide seamless support for home medical care. This system also recognizes the user's emotional state and reflects this in its analysis and information provision, enabling more personalized medical support.

[1968] Health consultation and information system

[1969] overview

[1970] The user enters a health-related question into the device, and the question along with the user's emotional data detected by the emotion engine is sent to the server. A generative AI model on the server analyzes the question and emotional data and generates appropriate health information and advice. The generated information is sent back to the device and displayed to the user in a form adjusted based on the emotional data.

[1971] Specific examples

[1972] The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects the user's level of stress from their facial expressions and tone of voice. The device then sends this question and emotional data to the server. The server uses a generative artificial intelligence model to analyze the question and emotional data and generate advice such as, "To create a good sleep environment, it is effective to limit your intake of caffeine and alcohol before bed and create a relaxing routine." The device then displays this advice to the user in a relaxed tone that takes into account the user's level of stress.

[1973] Medical device monitoring system

[1974] overview

[1975] Medical devices send data collected in real time to a server, where an emotion engine grasps the user's emotional state. A generative AI model on the server analyzes the data and emotion data, and generates appropriate usage instructions and warning messages if an abnormality is detected. This information is adjusted based on the emotion data and sent to and displayed on the user's or medical staff's device.

[1976] Specific examples

[1977] A patient's heart rate monitor detects an abnormal reading and sends the data to a server. An emotion engine recognizes the user's current emotional state as stress. The server uses a generative artificial intelligence model to analyze the data and emotional information and generate a message saying, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is displayed to the user in a calm and gentle tone that takes into account the stressful state.

[1978] Home medical support system

[1979] overview

[1980] A user inputs a contact request for home medical care from a terminal, and the request and emotional data detected by the emotion engine are sent to the server. A generative artificial intelligence model on the server analyzes the contact request, emotional data, and user information to identify appropriate medical staff. It then generates links and methods for establishing contact with the medical staff and displays them on the user's terminal in a format adjusted based on the emotional data.

[1981] Specific examples

[1982] The user types into the device, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to a doctor." The emotion engine detects anxiety from the user's facial expression. The device then sends this contact request and emotion data to the server. The server uses a generative artificial intelligence model to analyze the contact request and emotion information, identifies the appropriate medical staff member, and generates a message saying, "Dr. Suzuki is available. Please click here to contact him." The device then displays this message to the user in a reassuring tone that takes into account the user's state of anxiety.

[1983] In this way, by integrating an emotion engine and a generative artificial intelligence model, this invention solves various issues in remote medical care and home medical care, and provides a personalized and efficient medical support system. By each module working together, it is possible to improve user convenience and enhance the quality of medical care.

[1984] The processing flow will be explained below.

[1985] Health consultation and information system

[1986] Processing steps and specific operations

[1987] Step 1:

[1988] The user enters a health question into the terminal.

[1989] Specific operation: The user types into the device, "I haven't been able to sleep lately. Is there anything I can do?"

[1990] Step 2:

[1991] An emotion engine recognizes the user's emotional state.

[1992] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's stress level from their facial expressions and tone of voice.

[1993] Step 3:

[1994] The device sends the question and emotion data to the server.

[1995] Specific operation: The device converts the question and emotion data into an appropriate format (e.g., JSON) and sends it to the server via HTTPS.

[1996] Step 4:

[1997] The server analyzes the question and sentiment data.

[1998] Specific operation: The server inputs the question and emotion data into the generative AI model, and performs syntactic and semantic analysis to clarify the intent and emotional state of the question.

[1999] Step 5:

[2000] The server generates appropriate health information or advice.

[2001] Specific operation: Based on the analysis results, the generative artificial intelligence model generates an answer such as, "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine."

[2002] Step 6:

[2003] The server generates a response and sends it back to the terminal.

[2004] Specific operation: The server returns the generated answer in JSON format to the terminal.

[2005] Step 7:

[2006] The terminal displays the received answer to the user.

[2007] What it does: The device adjusts the answer it receives based on the emotion data and displays it in the user interface, saying in a relaxed tone, "To ensure a good night's sleep, it's a good idea to create a relaxing routine."

[2008] Medical device monitoring system

[2009] Processing steps and specific operations

[2010] Step 1:

[2011] The medical device transmits data to the server in real time.

[2012] Specific operation: Data collected by a medical device (e.g., heart rate monitor) is sent to the server in JSON format.

[2013] Step 2:

[2014] An emotion engine recognizes the user's emotional state.

[2015] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of tension.

[2016] Step 3:

[2017] The server analyzes the received data and emotion data.

[2018] Specific operation: The server inputs data and emotional data into a generative artificial intelligence model to detect abnormal values, patterns, and emotional states.

[2019] Step 4:

[2020] The server evaluates the state based on the data.

[2021] Specific operation: Based on the analysis results, the generative artificial intelligence model evaluates the patient's condition and determines whether there are any abnormalities.

[2022] Step 5:

[2023] The server generates appropriate usage and warning messages.

[2024] Specific operation: If the server detects an abnormality, it generates a message saying, "Your heart rate is abnormally high. Please check the settings of your medical device immediately and consult a doctor if necessary."

[2025] Step 6:

[2026] The server sends the generated message to the terminal.

[2027] Specific operation: The server sends the generated warning message in JSON format to the medical staff or patient's device.

[2028] Step 7:

[2029] The terminal displays the received message to the user.

[2030] Specific behavior: The device adjusts the warning message received based on the emotional data and displays it on the user interface, in a calm and gentle tone: "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[2031] Home medical support system

[2032] Processing steps and specific operations

[2033] Step 1:

[2034] A user enters a home health care contact request.

[2035] Specific operation: The user types into the terminal, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to talk to my doctor."

[2036] Step 2:

[2037] An emotion engine recognizes the user's emotional state.

[2038] Specific operation: Using the device's camera and microphone, the emotion engine detects the user's state of anxiety.

[2039] Step 3:

[2040] The terminal transmits a contact request and emotion data to the server.

[2041] Specific operation: The device sends a contact request and emotion data in JSON format to the server.

[2042] Step 4:

[2043] The server analyzes the contact request and the emotion data.

[2044] Specific operation: The server inputs contact request and emotion data into the generative AI model, which evaluates the user's state and emotion.

[2045] Step 5:

[2046] The server identifies the appropriate medical staff.

[2047] Specific operation: Based on the analysis results, the generative artificial intelligence model identifies appropriate medical staff (e.g., specialists, nurses, etc.).

[2048] Step 6:

[2049] The server creates a link or method for establishing contact.

[2050] Specific operation: The server generates links and methods for contacting medical staff (e.g., online medical consultation links).

[2051] Step 7:

[2052] The terminal displays the received contact information to the user.

[2053] What it does: The device adjusts the generated contact information based on the emotion data and displays it in the user interface, using a reassuring tone to say, "Dr. Suzuki is available. Click here to contact him."

[2054] The above is a description of the specific processing steps and operations of each system.

[2055] Example 2

[2056] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2057] In telemedicine and home medical care, there are issues with the efficient provision of information to users regarding their health, monitoring medical devices, and supporting home medical care. In particular, the lack of personalized information that takes into account the user's emotional state can lead to a decrease in user satisfaction.

[2058] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for sending a user's health-related question to a server hosting a generative artificial intelligence model; means for analyzing the question and emotion data on the server and generating appropriate health information or advice based on the question; means for adjusting the generated information or advice based on the emotion data and displaying it on the user's terminal; means for sending data from a medical device to the server; means for analyzing the data and emotion data and detecting abnormalities; means for generating appropriate usage instructions or warning messages adjusted based on the emotion data and sending and displaying the messages to the user's or medical staff's terminal; means for the user to send a home medical care contact request to the server; means for analyzing the contact request and user information using the generative artificial intelligence model and identifying appropriate medical staff; and means for generating a link or method for establishing contact with medical staff, adjusting the generated contact information based on the emotion data, and displaying it on the user's terminal. This enables personalized information provision that takes the user's emotional state into consideration.

[2059] "User" refers to an individual or organization that uses this system.

[2060] "Terminal" refers to a computing device used by a user, including a PC, smartphone, tablet, etc.

[2061] "Server" means the computer system hosting the generative artificial intelligence model and emotion engine, and is responsible for receiving, analyzing, and processing data sent by users.

[2062] A "generative artificial intelligence model" is a model that uses algorithms learned from large amounts of data to perform tasks such as natural language processing.

[2063] An "emotion engine" refers to a software or hardware system that analyzes a user's emotional state from facial expressions, tone of voice, etc., and acquires the data.

[2064] "Health information" refers to information related to the user's health condition, including preventative measures, improvement measures, and appropriate guidelines for action.

[2065] "Medical device" refers to a device for monitoring a user's health condition, including heart rate monitors, blood pressure monitors, and other vital signs monitoring devices.

[2066] "Emotion data" is data that indicates the emotional state of the user obtained by the emotion engine.

[2067] A "contact request" is a request that a user inputs into the system to the effect that they wish to be contacted by medical staff.

[2068] "Appropriate Medical Staff" refers to medical professionals selected to respond to a user's contact requests.

[2069] This invention relates to a telemedicine and home medical support system that integrates a generative artificial intelligence model and an emotion engine. This system works in conjunction with the user, terminal, and server elements to answer the user's health-related questions, monitor medical equipment, and provide smooth support for home medical care. This system also recognizes the user's emotional state and reflects it in the analysis and information provided, thereby achieving more personalized medical support.

[2070] Health consultation and information system

[2071] First, the user inputs a health-related question into the device's input screen. For example, "I haven't been able to sleep lately. Is there anything I can do about it?" At this stage, the device uses its built-in camera and microphone to detect the user's facial expressions and tone of voice and collect emotional data. The question and emotional data are then sent from the device to the server.

[2072] The server analyzes the received questions and emotional data using a generative artificial intelligence model (e.g., GPT-4) to generate appropriate health information and advice. For example, it might generate advice such as, "To ensure a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and create a relaxing routine." The generated information is returned to the device in a tone adjusted based on the emotional data, and the device displays it to the user.

[2073] Prompt Sentence Examples

[2074] The user types, "I haven't been able to sleep lately. Is there anything I can do?" The emotion engine detects stress and generates advice taking into account health information and stress levels, and displays it in a relaxed tone.

[2075] Medical device monitoring system

[2076] The medical device automatically collects real-time data such as heart rate and blood pressure. This data is sent from the medical device to a server, and the server uses an emotion engine to grasp the user's emotional state in real time. For example, if the user is in a state of tension, that information is also collected.

[2077] The server uses a generative artificial intelligence model to analyze the medical and emotional data to detect any abnormalities. If an abnormality is detected, the server generates a warning message such as, "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary." This message is adjusted based on the emotional data and sent to the device in a calm and gentle tone, and displayed to the user.

[2078] Prompt Sentence Examples

[2079] A medical device collects heart rate data. An emotion engine detects the user's tension and generates and displays a calm message informing them of abnormal heart rates.

[2080] Home medical support system

[2081] The user inputs a request for contact regarding home medical care into the device. For example, "I've recently been experiencing nausea, which I believe is a side effect of medication. I'd like to speak to a doctor." At this time, the device uses an emotion engine to detect the user's anxiety and collect emotional data.

[2082] The input contact request and emotional data are sent from the device to a server, which then uses a generative artificial intelligence model to analyze the contact request and emotional information and identify the appropriate medical staff. For example, a message such as "Dr. Suzuki is available. Please click here to contact him" is generated. This message is adjusted based on the emotional data and sent to the device and displayed to the user in a reassuring tone to ease anxiety.

[2083] Prompt Sentence Examples

[2084] A user types in "Nausea, thought to be a side effect of medication." Anxiety is detected. Contact instructions for a doctor are generated and displayed in a reassuring tone.

[2085] As described above, by integrating an emotion engine and a generative AI model, this invention can solve various issues in remote medical care and home medical care and provide a personalized and efficient medical support system. By having each module work in conjunction with each other, it is possible to improve user convenience and enhance the quality of medical care.

[2086] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2087] Health consultation and information system

[2088] Step 1:

[2089] The user enters a question.

[2090] Specific operation: The user accesses the device's input screen and enters a health-related question such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[2091] Input: Question

[2092] Output: Question data saved on the device

[2093] Step 2:

[2094] The device collects emotional data.

[2095] Specific operation: Using the device's camera and microphone, the system detects the user's facial expressions and tone of voice, and obtains emotional data such as stress and anxiety.

[2096] Input: User's facial expressions and tone of voice

[2097] Output: Obtained emotion data

[2098] Step 3:

[2099] The question and emotion data are sent to the server.

[2100] Specific operation: The device sends the entered question and detected emotion data to the server via an HTTP request.

[2101] Input: Question data, emotion data

[2102] Output: Question data and emotion data stored on the server

[2103] Step 4:

[2104] The server analyzes the data.

[2105] Specific operation: The server inputs the received question content and emotional data into a generative AI model (e.g., GPT-4) to generate appropriate health information and advice.

[2106] Input: Question data, emotion data

[2107] Output: Generated health information and advice

[2108] Step 5:

[2109] The generated information is returned to the terminal.

[2110] Specific operation: The server returns the generated health information and advice to the device in a tone adjusted based on the emotional data.

[2111] Input: Generated health information, advice, and emotional data

[2112] Output: Tailored health information and advice sent.

[2113] Step 6:

[2114] The terminal displays the information to the user.

[2115] Specific behavior: The device displays the information it receives to the user in a relaxed tone.

[2116] Input: Tailored health information and advice

[2117] Output: What the user sees on the screen

[2118] Medical device monitoring system

[2119] Step 1:

[2120] Medical devices collect data.

[2121] What it does: Medical devices automatically collect information like heart rate and blood pressure in real time.

[2122] Input: User biometric data (heart rate, blood pressure, etc.)

[2123] Output: Collected biometric data

[2124] Step 2:

[2125] The medical device sends the data to the server.

[2126] Specific operation: Medical devices send collected data in real time to a server.

[2127] Input: Biometric data

[2128] Output: Biometric data stored on the server

[2129] Step 3:

[2130] The server captures the emotion data.

[2131] Specific operation: The server uses an emotion engine to analyze the received biometric data and the user's emotional state (e.g., tension) in real time.

[2132] Input: Biometric data, emotional state

[2133] Output: Emotion data

[2134] Step 4:

[2135] The server analyzes the data.

[2136] How it works: The server uses the generative AI model to analyze biometric and emotional data to detect any anomalies.

[2137] Input: Biometric data, emotional data

[2138] Output: Anomaly detection results

[2139] Step 5:

[2140] If an anomaly is detected, a message is generated.

[2141] Specific behavior: If the server detects an abnormality, it generates a warning message such as "Your heart rate is abnormally high. Please check your medical device settings immediately and consult a doctor if necessary."

[2142] Input: Anomaly detection results, emotion data

[2143] Output: Warning message

[2144] Step 6:

[2145] Send the generated message.

[2146] Specific behavior: The server sends the generated message to the terminal in a calm and gentle tone.

[2147] Input: warning message

[2148] Output: Adjusted warning message sent

[2149] Step 7:

[2150] The terminal displays the message.

[2151] Specific behavior: Display the message received by the device to the user.

[2152] Input: Adjusted warning message

[2153] Output: The message the user sees on the screen

[2154] Home medical support system

[2155] Step 1:

[2156] A user enters a contact request.

[2157] Specific operation: The user inputs a contact request into the terminal, saying, "I've been feeling nauseous recently, which I think is a side effect of medication. I'd like to speak to a doctor."

[2158] Input: Contact Request

[2159] Output: Contact request data stored on the device

[2160] Step 2:

[2161] The device collects emotional data.

[2162] Specific operation: The device uses an emotion engine to detect the user's anxiety and collect emotion data.

[2163] Input: User's facial expressions and tone of voice

[2164] Output: Collected emotion data

[2165] Step 3:

[2166] A contact request and emotion data are sent to the server.

[2167] Specific operation: The device sends a contact request and emotion data to the server.

[2168] Input: Contact request data, emotion data

[2169] Output: Contact request data and emotion data stored on the server

[2170] Step 4:

[2171] The server analyzes the data.

[2172] What it does: The server uses a generative AI model to analyze contact requests and emotional data to identify appropriate medical staff.

[2173] Input: Contact request data, emotion data

[2174] Output: Identified medical staff information

[2175] Step 5:

[2176] Generate contact methods.

[2177] Specific operation: The server generates a method of contacting the identified medical staff (e.g., a link where the medical staff is available).

[2178] Input: Medical staff information, emotion data

[2179] Output: Generated contact information

[2180] Step 6:

[2181] The generated information is transmitted.

[2182] What it does: The server sends the generated information to the device in a reassuring tone.

[2183] Input: Generated contact information

[2184] Output: Coordinated contact information sent

[2185] Step 7:

[2186] The terminal displays the information to the user.

[2187] Specific behavior: The device displays the information it receives to the user and provides links and methods for establishing contact with medical staff.

[2188] Input: Adjusted contact information

[2189] Output: What the user sees on the screen

[2190] (Application example 2)

[2191] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2192] In telemedicine and home medical care, there is a need to provide appropriate and personalized medical support that takes into account the user's emotional state. However, conventional systems can only generate uniform answers or warning messages in response to the user's health questions or data from medical devices, and lack feedback that reflects the user's emotional state. Therefore, a system is needed that can alleviate users' anxieties and doubts and provide more effective medical support.

[2193] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2194] In this invention, the server includes emotion recognition means for sensing and analyzing the user's emotional state, means for analyzing questions and data using a generative artificial intelligence model to generate appropriate information and advice, and means for adjusting the tone and content of the information and advice based on the emotion data, thereby enabling personalized medical information and advice according to the user's emotional state.

[2195] The "means for inputting questions about the user's health" is an interface that allows the user to input questions about their own health condition or symptoms into the terminal.

[2196] The "means for sending to a server hosting a generative artificial intelligence model" is a means having the function of transferring a question entered by a user to a server hosting a generative artificial intelligence model.

[2197] A "generative artificial intelligence model" is an artificial intelligence model that performs natural language processing and data analysis to generate appropriate information and advice based on user input.

[2198] The "means for returning generated information or advice to the user's terminal" refers to a means having a function for returning generated information or advice from the server to the user's terminal.

[2199] The "means for displaying the information or advice" refers to a means having a function for visually displaying the information or advice generated on the user's terminal.

[2200] The "emotion recognition means" is a means for detecting and analyzing emotional data such as the user's facial expressions and tone of voice.

[2201] "Emotion data" is data that indicates the current emotional state of the user, and is acquired by emotion recognition means.

[2202] "Means for adjusting the tone and content of information and advice based on emotional data" refers to means for changing the method of delivery and content of information and advice generated using emotional data to suit the emotional state of the user.

[2203] The "means for collecting data from a medical device in real time and transmitting the data to a server" refers to a means for transferring data acquired from a medical device to a server in real time.

[2204] "Means for generating appropriate usage instructions or warning messages when an abnormality is detected" refers to means for analyzing data from medical devices and generating appropriate response instructions or messages urging caution when an abnormality is detected.

[2205] The "means for the user to input a request for contact regarding home medical care" is an interface for the user to input a request into a terminal when the user desires to be contacted regarding home medical care.

[2206] The "means for sending a contact request to a server" is a means for transferring a contact request regarding home medical care input by a user to a server.

[2207] The "means for identifying appropriate medical staff" is a means for analyzing the contact request and user information and selecting appropriate medical staff.

[2208] "Means for creating a link or method for establishing contact" refers to a means for creating a link or method for establishing contact with medical staff.

[2209] The "means for displaying contact information on the user's terminal" refers to a means for visually displaying the generated contact information on the user's terminal.

[2210] As an embodiment of the present invention, the following system program, hardware, and software configuration will be described.

[2211] Hardware and Software Configuration

[2212] The system of the present invention includes smart glasses, a camera, a server, a terminal, a generative artificial intelligence model, and an emotion recognition engine. The smart glasses have a built-in camera that captures the user's facial expressions in real time. The server hosts the generative artificial intelligence model (generative AI model) and the emotion recognition engine, which analyzes the user's emotional state.

[2213] Program Processing Overview

[2214] 1. Enter your question:

[2215] The user inputs a health-related question through the smart glasses, such as, "I've been having trouble sleeping lately. Is there anything I can do about it?"

[2216] 2. Collecting Emotional Data:

[2217] The smart glasses have a built-in camera that captures the user's facial expressions and body movements, and the emotion recognition engine analyzes the data. In this case, TensorFlow is used to build the emotion recognition model.

[2218] 3. Data transmission and analysis:

[2219] The question and emotion data are sent to a server, where a generative AI model (e.g., GPT-3) analyzes the question and emotion data and generates appropriate health information and advice.

[2220] 4. Coordination and return of information:

[2221] The content and tone of the generated information and advice are adjusted based on the emotional data and sent back to the user's terminal.

[2222] 5. Displaying information:

[2223] Tailored information and advice will be displayed on the user's device, such as "To create a good sleep environment, it is effective to limit caffeine and alcohol intake before bed and to create a relaxing routine," in a relaxing tone.

[2224] Software used

[2225] TensorFlow: Used to build an emotion recognition engine

[2226] GPT-3: Generative AI model

[2227] Specific examples

[2228] To use a specific example, the following situation can be considered.

[2229] A user uses smart glasses in a virtual store and stops by the health food section. He or she wonders, "Which health food is right for me?" The emotion recognition engine detects doubt from the user's facial expressions and gestures, and sends the data to the server. The generative AI model on the server analyzes the situation—"The user is interested, but has doubts about which health food is right for me"—and generates the following example prompt:

[2230] Example 1: "User is feeling curious. Recommend a health product that suits this emotion in the context of energy supplements."

[2231] Example 2: "User feels doubtful. Suggest a health product for daily vitamin intake to relieve this emotion in a virtual store."

[2232] In this way, personalized information tailored to the user's emotional state is provided, resulting in a more satisfying shopping experience.

[2233] summary

[2234] The overall process of this system recognizes the user's emotional state in real time and provides personalized information and advice accordingly. The hardware and software components used are clearly indicated to demonstrate the implementation of the invention, enabling it to provide appropriate and efficient assistance tailored to the user's needs.

[2235] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2236] Step 1:

[2237] A user puts on the smart glasses and inputs a health question.

[2238] The input is sent to the server via the smart glasses interface. This input data includes the question in text format. For example, "I haven't been able to sleep lately. Is there anything I can do about it?"

[2239] Step 2:

[2240] The emotion recognition means captures the user's facial expressions and body movements in real time from a camera built into the smart glasses.

[2241] Facial expression data and physical movement data captured by the camera are sent to an emotion recognition engine using TensorFlow, which analyzes the user's emotional state.

[2242] As an output, the analysis produces emotional data, e.g., stress, doubt, etc.

[2243] Step 3:

[2244] The terminal transmits question data and emotion data to the server.

[2245] The transmitted data includes the user's question text data and emotion data generated by the emotion recognition engine.

[2246] The server receives this data.

[2247] Step 4:

[2248] A generative AI model (GPT-3) on the server analyzes question data and emotion data.

[2249] Specifically, a prompt is generated based on the question and emotion data. For example, "User is feeling stressed and asked about sleep issues. Provide advice to help relax before sleep."

[2250] The generative AI model generates health information and advice based on this prompt, for example, "To ensure a good night's sleep, it's helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[2251] Step 5:

[2252] The content and tone of the information and advice generated are adjusted based on the emotional data.

[2253] Example of adjustment: For a user in a stressed state, sentences are generated in a relaxed tone.

[2254] Step 6:

[2255] The server sends tailored information and advice back to the device.

[2256] The output data includes the final information and advice that will be displayed to the user, for example, "To ensure a good night's sleep, it is helpful to limit caffeine and alcohol intake before bed and to develop a relaxing routine."

[2257] Step 7:

[2258] The terminal displays the received information and advice to the user.

[2259] Tailored information and advice is visually presented on a display within the smart glasses.

[2260] This allows users to quickly access appropriate health information and advice based on their emotional state.

[2261] Through this process, users can receive personalized health information and advice tailored to their emotional state, resulting in a more satisfying experience.

[2262] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2263] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2264] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2265] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2266] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2267] These emotion...

Claims

1. a means for inputting a user's health-related question; means for transmitting the question to a server hosting a generative artificial intelligence model; means for analyzing the question on the server and generating appropriate health information or advice based on the question; means for returning the generated information or advice to the user's terminal; means for displaying said information or advice; A system including:

2. means for collecting data from the medical device in real time and transmitting the data to a server; means for analyzing the data using a generative artificial intelligence model on the server to detect anomalies; means for generating appropriate usage instructions or warning messages when an anomaly is detected; means for transmitting and displaying the message on a terminal of a user or medical staff; The system of claim 1 , comprising:

3. a means for a user to input a home medical contact request; means for transmitting the contact request to a server; means for analyzing the contact request and user information using a generative artificial intelligence model on the server to identify appropriate medical staff; means for creating a link or method for establishing contact with said medical staff; means for displaying the generated contact information on the user's terminal; The system of claim 1 , comprising:

Images