system

Abstract

Provide a system. A method for a user to initiate a remote medical consultation; a means for analyzing a user's input using a generative AI model to generate a response; A means for displaying a response generated by the generative AI model on a user terminal; A means of acquiring and analyzing user health data in cooperation with a health management app; means for providing advice to the user based on the acquired health data; a means for forwarding the user's consultation to a medical professional and carrying out follow-up as necessary; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Due to the aging society and the spread of infectious diseases, the number of patients who have difficulty traveling is increasing, and the need for telemedicine is also rapidly increasing. Under these circumstances, patients who have difficulty traveling have difficulty receiving appropriate medical support, which tends to undermine equality in medical care. In addition, there is a lack of effective medical support for patients who are anxious about interacting directly with medical professionals or who have difficulty interpreting health management information. [Means for solving the problem]

[0005] The present invention aims to solve the above problem by providing a system that includes a means for a user to initiate a remote medical consultation, a means for analyzing the user's input using a generative AI model and generating a response, a means for displaying the response generated by the generative AI model on the user's terminal, a means for acquiring and analyzing the user's health data in cooperation with a health management app, a means for providing advice to the user based on the acquired health data, and a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary.

[0006] "User" refers to any individual or entity that uses the System.

[0007] "Telemedical consultation" refers to a medical consultation service provided via a network such as the Internet.

[0008] A "generative AI model" refers to an artificial intelligence model that analyzes input data from a user and generates an appropriate response.

[0009] "User terminal" refers to a device (e.g., smartphone, tablet, or PC) through which a user accesses and interacts with the system.

[0010] A "health management app" refers to application software for recording and managing a user's health status and activity data.

[0011] "Health Data" refers to information related to a user's health condition (e.g., heart rate, body temperature, blood pressure, amount of exercise).

[0012] "Medical professional" refers to a professional such as a doctor or nurse who has specialized medical knowledge and provides advice and treatment to users.

[0013] "Follow-up" refers to subsequent medical services to provide additional examination or advice after an initial medical consultation.

[0014] "Authentication Information" means information used to identify a user and authorize access to a system (e.g., username, password).

[0015] "Database" refers to a data storage system that organizes and stores data used by the system and makes it efficiently accessible. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[0038] Overall system configuration

[0039] 1. User Registration and Login

[0040] User: A user accesses the system through an app or web interface and enters the required registration information, including name, email address, and password.

[0041] Server: The server saves the entered user information in a database and notifies the user that registration is complete. After that, the user enters their authentication information on the login screen and is granted access to the system.

[0042] 2. Initiating a medical consultation

[0043] User: When the user presses the "Start medical consultation" button within the app, the server launches the generative AI model and displays the chatbot interface on the user's device.

[0044] Chatbot: The chatbot uses generative AI models to engage in natural conversations with users and identify their concerns and symptoms.

[0045] 3. Conversational exchanges

[0046] User: The user enters their symptoms and concerns into the chatbot. For example, they might say, "I've been coughing so much lately I can't sleep at night."

[0047] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[0048] 4. Health Management Data Linkage

[0049] User: Health data is collected when the user connects the health management app to the system on the settings screen.

[0050] Server: The server passes data obtained from the health management app to the generative AI model and analyzes the user's health condition.

[0051] Chatbot: Provides users with advice based on their health data, such as, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[0052] 5. Follow-up with a medical professional

[0053] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[0054] Server: Transfers the user's consultation to a medical professional and performs follow-up, if necessary.

[0055] Medical Expert: A medical expert provides additional diagnosis and advice to users via video or text chat.

[0056] Specific examples

[0057] 1. User Registration

[0058] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[0059] Server: Notify "Registration complete. Please log in."

[0060] 2. Medical consultation begins

[0061] User: Tap "Start Medical Consultation" on the app's home screen.

[0062] Server: The chatbot displays "Hello, how can we help you?"

[0063] 3. Conversational exchanges

[0064] User: Type "I've been coughing so much at night lately I can't sleep."

[0065] Chatbot: "That's terrible. How long have you had a cough?"

[0066] User: "About a week."

[0067] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[0068] 4. Health Management Data Linkage

[0069] User: Set up the connection to the health management app.

[0070] Server: Collects health management data and sends this information to the generative AI model.

[0071] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[0072] 5. Follow-up with a medical professional

[0073] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[0074] Server: Send follow-up requests to medical professionals and schedule video chats.

[0075] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[0076] The present invention enables even patients who have difficulty moving around to easily receive medical consultations from home, further supporting users' health management.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] User: Launches the application and clicks the "Sign Up" button.

[0080] On your device: Display a form for name, email address, password, etc.

[0081] User: Enter the required information and press the "Register" button.

[0082] Terminal: Sends the entered information to the server.

[0083] Step 2:

[0084] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[0085] Device: Display "Registration complete. Please log in."

[0086] User: Enter your email address and password on the login screen and click the "Login" button.

[0087] Device: Sends authentication information to the server.

[0088] Step 3:

[0089] Server: Retrieves user information from the database and verifies authentication information.

[0090] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[0091] On the device: Display a login success message and go to the home screen.

[0092] Step 4:

[0093] User: Press the "Start medical consultation" button on the home screen.

[0094] Device: Sends a request to the server to launch the generative AI model.

[0095] Server: Initializes the generative AI model and starts the chatbot session.

[0096] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[0097] Step 5:

[0098] User: Writes down symptoms and concerns in the chat interface and sends it.

[0099] Terminal: Sends the user's messages to the server.

[0100] Server: Passes messages to the generative AI model for analysis.

[0101] Chatbot: Generates appropriate response messages and returns the results to the server.

[0102] Server: Sends a response message to the terminal.

[0103] Terminal: Display the response message.

[0104] Step 6:

[0105] User: Enters additional information in response to the chatbot's questions and submits.

[0106] Terminal: Sends the user's messages to the server.

[0107] Server: Passes messages to the generative AI model.

[0108] Chatbot: Based on the analysis results, further questions and advice are generated and returned to the server.

[0109] Server: Sends a response message to the terminal.

[0110] Terminal: Display the response message.

[0111] Step 7:

[0112] User: Set up integration with the health management app on the app settings screen.

[0113] Device: Sends a request to connect with the health management app to the server.

[0114] Server: Accesses the health management app and retrieves the necessary data.

[0115] Server: Passes the acquired health data to the generative AI model.

[0116] Generative AI model: Analyzes data and assesses the user's health status.

[0117] Server: Sends the health assessment results to the device.

[0118] Device: Displays advice to the user based on health data.

[0119] Step 8:

[0120] Chatbot: A generative AI model analyzes the user's inquiry and determines whether it is urgent.

[0121] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[0122] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[0123] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[0124] Example 1

[0125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] In remote medical consultations, there is a need for a system that allows users to easily receive medical consultations from home, efficiently manage their health status, and quickly connect with medical professionals in emergencies. Conventional systems have complicated user registration and login procedures, and the quality of responses from generative AI models is insufficient. Furthermore, the collection and analysis of health information through integration with health management software was not performed effectively, resulting in a lack of accurate advice for users. Furthermore, there were issues with smooth follow-up with medical professionals in emergencies.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0128] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health information in cooperation with health management software; a means for providing the user with advice based on the acquired health information; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; a means for the user to enter authentication information such as their name, email address, and password to register and log in to the system; a means for transmitting the user's input symptoms and concerns to the generative AI model and generating an appropriate response; and a means for launching the generative AI model on the user's terminal and providing a chatbot interface that engages in natural conversation with the user. This allows users to conveniently receive medical consultations from home and receive high-quality responses through the generative AI model. Furthermore, through effective collection and analysis of health information in cooperation with health management software, the server can provide accurate advice to the user and quickly collaborate with medical professionals for follow-up in emergencies.

[0129] "User" refers to an individual who uses this system to conduct a remote medical consultation.

[0130] "Telemedical consultation" refers to medical consultation conducted remotely via the Internet.

[0131] A "generative AI model" refers to artificial intelligence that analyzes user input data and generates appropriate responses.

[0132] "User terminal" refers to the device (smartphone, PC, etc.) used by a user to access this system.

[0133] "Health management software" refers to applications and platforms for collecting, managing, and analyzing users' health data.

[0134] "Health information" refers to data related to the user's health condition (e.g., sleep time, amount of exercise, weight, etc.).

[0135] "Authentication Information" means the information, such as name, email address, and password, that a User uses to register and log in to the System.

[0136] "Chatbot interface" refers to the screen or application through which a user and a generative AI model interact.

[0137] "Medical Expert" refers to a doctor or other medical professional who provides follow-up care to the user based on the user's consultation.

[0138] "Follow-up" refers to providing additional diagnosis or advice to the user.

[0139] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[0140] User Registration and Login

[0141] Users access the system through an app or web interface and register by entering their name, email address, and password, which then sends the authentication information to the server.

[0142] The server stores the entered user authentication information in a database using a database management system such as MySQL (registered trademark). It also notifies the user when registration is complete.

[0143] The user logs in with the registered information and accesses the system. The server checks the user's authentication information against the database, and displays the home screen only if authentication is successful.

[0144] Initiating a medical consultation

[0145] When a user selects "Start Medical Consultation" on the home screen, the server launches a generative AI model (e.g., OpenAI's GPT-3).

[0146] The server initializes the generative AI model, starts the chatbot session, and displays the chatbot interface on the user's device, ready to begin a healthcare-related conversation.

[0147] Conversational exchange

[0148] Users can input their symptoms and concerns into the chatbot. For example, they can enter something like, "I've been coughing so much at night recently that I can't sleep."

[0149] The server sends this input to a generative AI model that generates a response, such as a question like, "How long has your cough lasted?"

[0150] The generative AI model digs deeper into the user's concerns through ongoing conversation and asks more detailed questions.

[0151] Health management data linkage

[0152] When a user links the system with health management software (for example, GOOGLE FI (registered trademark) or Apple Health) on the setting screen, the server obtains data from the health software.

[0153] The server sends the acquired health information to a generative AI model to analyze the user's health condition.

[0154] The chatbot will then provide appropriate advice to the user based on the analyzed data. For example, it could say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[0155] Follow-up with a medical professional

[0156] If the chatbot determines that the user's symptoms are urgent, it notifies the server of this information.

[0157] The server forwards the user's consultation to a medical professional for follow-up as needed.

[0158] Medical professionals can provide additional diagnosis and advice to users via video or text chat, allowing users to receive professional advice quickly.

[0159] Providing concrete examples

[0160] In a specific scenario in which a user is seeking medical advice, the following prompt sentences may be used:

[0161] Example prompt: "I've been coughing so much at night lately that I can't sleep. What should I do?"

[0162] This invention allows patients with mobility issues to easily receive medical consultations from home, receiving high-quality responses and accurate advice. It also enables prompt collaboration with medical professionals in emergencies, enabling appropriate follow-up. This system, utilizing generative AI models, further supports users' health management.

[0163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0164] Step 1: User Registration

[0165] Input: User's name, email address, and password

[0166] Specific operation:

[0167] Through the app or web interface, users enter their name, email address, and password and click the "Register" button.

[0168] The server stores the information entered by the user in a database using MySQL.

[0169] The server will notify you, "Registration complete. Please log in."

[0170] Output: Registration completion notification

[0171] Step 2: Login authentication

[0172] Input: User's email address, password

[0173] Specific operation:

[0174] The user enters the registered email address and password and clicks the "Login" button.

[0175] The server checks the user's credentials against a database.

[0176] If the server successfully authenticates the user, it displays the home screen, but if it fails, it notifies the user that "Authentication failed."

[0177] Output: Home screen display or authentication failure notification

[0178] Step 3: Initiating a medical consultation

[0179] Input: Click on the "Start medical consultation" button

[0180] Specific operation:

[0181] The user selects "Start medical consultation" on the home screen.

[0182] The server initializes a generative AI model (e.g., OpenAI's GPT-3).

[0183] The server sends the chatbot interface to the user terminal and starts the session.

[0184] Output: Chatbot interface displayed

[0185] Step 4: Accepting User Input

[0186] Input: User's symptoms and concerns

[0187] Specific operation:

[0188] The user enters their symptoms and concerns in the text box and clicks the send button.

[0189] Output: Text data of input symptoms and anxieties

[0190] Step 5: Generate a response using a generative AI model

[0191] Input: Text data of user symptoms and anxieties

[0192] Specific operation:

[0193] The server sends the user's input text to the generative AI model.

[0194] The generative AI model analyzes the input and generates an appropriate response.

[0195] The server sends the generated response to the user terminal.

[0196] Output: The generated response text

[0197] Step 6: View the response

[0198] Input: Generated response text

[0199] Specific operation:

[0200] The terminal displays the response received from the server on the chatbot interface.

[0201] Output: The response displayed on the chatbot interface

[0202] Step 7: Link with health management app

[0203] Input: Select health management app and click link button

[0204] Specific operation:

[0205] The user configures the settings screen to link with a health management app (for example, GOOGLE FIT (registered trademark) or Apple Health).

[0206] The server calls the API of the selected health management app and sets up the connection.

[0207] Output: Notification that the connection with the health management app has been completed

[0208] Step 8: Acquire and analyze health data

[0209] Input: Data obtained from health management app

[0210] Specific operation:

[0211] The server uses the health management app's API to obtain the user's health data.

[0212] The server sends the acquired data to the generative AI model.

[0213] The generative AI model analyzes health data and generates advice about the user's health status.

[0214] The server transmits the generated advice to the user terminal.

[0215] Output: Health advice

[0216] Step 9: Providing advice based on health data

[0217] Input: Generated advice

[0218] Specific operation:

[0219] The terminal displays the advice received from the server on the chatbot interface.

[0220] Output: Display of health advice

[0221] Step 10: Emergency response decision and notification

[0222] Input: User symptoms and health data

[0223] Specific operation:

[0224] The chatbot analyzes the user's symptoms and health data to determine whether emergency response is required.

[0225] If the chatbot determines that an emergency response is required, it notifies the server.

[0226] Output: Notification that urgent action is required

[0227] Step 11: Consult a medical professional

[0228] Input: Notification that emergency action is required

[0229] Specific operation:

[0230] The server notifies the medical professional of the user's consultation and the urgency of the consultation, and performs follow-up.

[0231] Medical professionals will provide additional diagnosis and advice to users via video or text chat.

[0232] Output: Notification that a follow-up with a medical professional will be performed

[0233] The system allows users to easily receive medical consultations from home, receive high-quality responses, and is excellent for quickly connecting with medical professionals in emergencies.

[0234] (Application example 1)

[0235] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] While remote medical consultation systems allow users to easily receive medical consultations from home, providing prompt and appropriate support during health consultations at physical stores has been difficult. For this reason, new methods are needed for store staff to provide customers with appropriate health advice in real time and efficiently recommend products.

[0237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0238] In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; and a means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses. This enables quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[0239] "User" refers to an individual who uses the system to receive remote medical consultations or in-store health consultations.

[0240] "Telemedical consultation" means a service that allows a user to consult with a medical professional online from the comfort of their own home or any other location.

[0241] A "generative AI model" refers to artificial intelligence technology that analyzes user input and generates appropriate responses and advice through natural conversation.

[0242] "User Terminal" means a device (e.g., smartphone, tablet, computer) through which a User accesses the System and conducts a remote medical consultation.

[0243] "Health management app" refers to an application that collects and manages a user's health data.

[0244] "Server" means the computer system that manages and operates the entire system and processes information from the generative AI model and database.

[0245] "Smart glasses" refer to wearable devices that provide information along the user's line of sight and assist staff in providing real-time health consultations in brick-and-mortar stores.

[0246] "Medical Professional" means a medical professional who is qualified to provide professional diagnosis and advice regarding the user's health inquiry.

[0247] The present invention provides a health consultation support system for use in a physical store, which allows users to receive health consultations in real time through store staff wearing smart glasses. Detailed embodiments of the present invention will be described below.

[0248] The server includes means for a user to initiate a remote medical consultation, means for analyzing the user's input using a generative AI model and generating a response, means for displaying the response generated by the generative AI model on the user's terminal, means for acquiring and analyzing the user's health data in cooperation with a health management app, means for providing advice to the user based on the acquired health data, means for transferring the user's consultation content to a medical professional and performing follow-up as necessary, and means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses.

[0249] 1. Overall system configuration

[0250] The system consists of the following main components:

[0251] User devices: smartphones, tablets, computers, etc.

[0252] Server: Cloud-based computing services.

[0253] Generative AI models: such as OpenAI's GPT-4 (registered trademark).

[0254] Smart glasses: Google® Glass®, etc.

[0255] Health management app: An application that collects and manages health data.

[0256] 2. System Operation

[0257] 2.1 User Registration and Login

[0258] The server provides a means for users to access the system through an app or web interface and enter the necessary registration information (name, email address, password, etc.) The server stores this information in a database and manages the authentication information.

[0259] 2.2 Starting a health consultation

[0260] When a user presses the "Start medical consultation" button, the server activates the generative AI model and displays a chatbot interface on the user's device. This chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation content and symptoms.

[0261] 2.3 Conversational exchanges

[0262] When a user inputs a question or symptom into the system, the generative AI model analyzes it and provides an appropriate answer, which is displayed on the user's device and the smart glasses' display.

[0263] 2.4 Health Management Data Linkage

[0264] When a user connects a health management app to the system, the server acquires this data and sends it to the generative AI model, which then analyzes the user's health condition and provides appropriate advice.

[0265] 2.5 Real-time health consultations at physical stores

[0266] When a store employee wearing smart glasses receives a health consultation from a customer, the server sends the information to the generative AI model, which generates a response. This response is displayed in real time on the employee's smart glasses, and the employee can then provide appropriate advice to the customer.

[0267] Specific examples

[0268] For example, if a customer asks how to choose vitamins, the following exchange might occur:

[0269] Staff: "What purpose are you looking for vitamins for?"

[0270] Customer: "I'm feeling tired and I'd like to have more energy."

[0271] Smart glasses display: "To increase your energy, we recommend supplements containing B and C vitamins. Iron may also be beneficial. Do you have any other questions? Learn more about these products."

[0272] Prompt Sentence Examples

[0273] "A client is asking about choosing vitamins. Can you recommend some vitamins or supplements to help them increase their energy?"

[0274] This will enable quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[0275] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0276] Step 1:

[0277] A user accesses the system through an app or web interface and registers. The user enters the required information, such as name, email address, and password, and sends it to the server. The server stores this information in a database and notifies the user that registration is complete.

[0278] Input: Name, Email Address, Password

[0279] Data processing: Saving user information to a database

[0280] Output: Registration completion notification

[0281] Step 2:

[0282] A user logs in to the system. The user enters the registered email address and password and sends the authentication information to the server. The server authenticates the user and allows the user to access the system.

[0283] Input: Email address, Password

[0284] Data Calculation: Authentication Check

[0285] Output: System access permissions

[0286] Step 3:

[0287] The user presses the "Start medical consultation" button. The server launches the generative AI model and displays the chatbot interface on the user's device. The user's device then begins a dialogue with the user.

[0288] Input: Medical consultation start request

[0289] Data processing: Displaying the chatbot interface

[0290] Output: Medical consultation initiated

[0291] Step 4:

[0292] The user inputs symptoms or questions into the chatbot. The user's device sends the input text to the generative AI model. The server receives it, analyzes it with the generative AI model, and generates an appropriate response.

[0293] Input: User question or symptom

[0294] Data Computation: Question Analysis and Answer Generation

[0295] Output: The generated response

[0296] Step 5:

[0297] The server sends the generated response to the user terminal for display, which displays it to the user for further interaction.

[0298] Input: The generated response

[0299] Data processing: Sending and displaying responses

[0300] Output: Display response to user

[0301] Step 6:

[0302] The user configures the health management app to work with the server, which then acquires the user's health data from the app and sends it to the generative AI model for analysis.

[0303] Input: Health management app data

[0304] Data Computing: Health Data Analysis

[0305] Output: Advice based on health status

[0306] Step 7:

[0307] A user consults a staff member wearing smart glasses at a physical store. The staff member's smart glasses send the user's consultation to the generative AI model, which analyzes it on the server. The server then sends the answer to the staff member's smart glasses and displays it.

[0308] Input: Customer Question

[0309] Data Computation: Question Analysis and Answer Generation

[0310] Output: Response display on smart glasses

[0311] Step 8:

[0312] If necessary, the server will follow up by transferring the user's consultation to a medical professional who will provide further diagnosis and advice via video or text chat.

[0313] Input: User's inquiry

[0314] Data processing: Transfer of consultation details

[0315] Output: Follow-up with medical professionals

[0316] The above processing steps make it possible to provide prompt and appropriate health consultations even in physical stores.

[0317] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0318] The present invention combines an emotion engine with a remote medical consultation system, which can recognize the user's emotional state and provide appropriate responses and medical support through a generative AI model. Specific embodiments of the present invention are described below.

[0319] Overall system configuration

[0320] 1. User Registration and Login

[0321] User: Accesses the system through an application or web interface and creates a new account by entering the required registration information (name, email address, password, etc.).

[0322] Server: Saves the entered user information in the database and notifies the user that registration is complete. Once the user enters their authentication information on the login screen, they are granted access to the system.

[0323] 2. Initiating a medical consultation

[0324] User: When the user presses the "Start medical consultation" button within the application, the server launches the generative AI model and displays the chatbot interface on the user's device.

[0325] Chatbot: The chatbot uses generative AI models to have natural conversations with users and confirm their concerns and symptoms.

[0326] 3. Conversational exchanges

[0327] User: Enters symptoms and concerns into the chatbot. For example, "I've been coughing so much lately I can't sleep at night."

[0328] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[0329] 4. Emotion Recognition by Emotion Engine

[0330] Server: Sends user input to the emotion engine and analyzes the emotional state.

[0331] Emotion engine: Recognizes user emotions (e.g., stress, anxiety, anger) and passes the results to a generative AI model.

[0332] Generative AI model: Tailors responses based on emotional state to provide appropriate support to users.

[0333] 5. Health Management Data Linkage

[0334] User: Health data is obtained by linking the health management app with the system from the settings screen.

[0335] Server: Sends data obtained from the health management app to the emotion engine and generative AI model to analyze the user's health condition.

[0336] Chatbot: Provides users with advice based on their health data and emotional state. For example, it might say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[0337] 6. Follow-up with a medical professional

[0338] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[0339] Server: Transfers the user's consultation to a medical professional, if necessary, and arranges for follow-up.

[0340] Medical Expert: Provides additional diagnosis and advice to users via video or text chat.

[0341] Specific examples

[0342] 1. User Registration

[0343] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[0344] Server: Notify "Registration complete. Please log in."

[0345] 2. Medical consultation begins

[0346] User: Tap "Start Medical Consultation" on the app's home screen.

[0347] Server: The chatbot displays "Hello, how can we help you?"

[0348] 3. Conversational exchanges

[0349] User: Type "I've been coughing so much at night lately I can't sleep."

[0350] Chatbot: "That's terrible. How long have you had a cough?"

[0351] User: "About a week."

[0352] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[0353] 4. Emotion Recognition by Emotion Engine

[0354] Server: Sends the user's input, "About a week" and "Are you experiencing any other symptoms such as fever or fatigue?" to the emotion engine.

[0355] Emotion engine: Recognizes user anxieties and passes that information to a generative AI model.

[0356] Generative AI model: Responds, "Okay, don't worry, this can get better with treatment."

[0357] 5. Health Management Data Linkage

[0358] User: Set up the connection to the health management app.

[0359] Server: Collects health management data and sends this information to the generative AI model and emotion engine.

[0360] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[0361] 6. Follow-up with a medical professional

[0362] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[0363] Server: Send follow-up requests to medical professionals and schedule video chats.

[0364] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[0365] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[0366] The processing flow will be explained below.

[0367] The present invention is a remote medical consultation system with an emotion engine built in. The specific processing flow will be explained below step by step.

[0368] Overall system configuration

[0369] Step 1:

[0370] User: Launches the application and clicks the "Sign Up" button.

[0371] On your device: Display a form for name, email address, password, etc.

[0372] User: Enter the required information and press the "Register" button.

[0373] Terminal: Sends the entered information to the server.

[0374] Step 2:

[0375] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[0376] On the device: Display "Registration successful. Please log in" to the user.

[0377] User: Enter your email address and password on the login screen and click the "Login" button.

[0378] Device: Sends authentication information to the server.

[0379] Step 3:

[0380] Server: Retrieves user information from the database and verifies authentication information.

[0381] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[0382] On the device: Display a login success message and go to the home screen.

[0383] Step 4:

[0384] User: Press the "Start medical consultation" button on the home screen.

[0385] Device: Sends a request to the server to launch the generative AI model.

[0386] Server: Initializes the generative AI model and starts the chatbot session.

[0387] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[0388] Step 5:

[0389] User: Writes down symptoms and concerns in the chat interface and sends it.

[0390] Terminal: Sends the user's messages to the server.

[0391] Server: Passes messages to the generative AI model for analysis.

[0392] Chatbot: Generates appropriate response messages and returns the results to the server.

[0393] Server: Sends a response message to the terminal.

[0394] Terminal: Display the response message.

[0395] Step 6:

[0396] User: Enters additional information in response to the chatbot's questions and submits.

[0397] Terminal: Sends the user's messages to the server.

[0398] Server: Passes messages to the generative AI model and emotion engine.

[0399] Emotion engine: Analyzes user input and recognizes emotional states (e.g., stress, anxiety, relief, etc.).

[0400] Generative AI model: Adjusts responses based on the analysis results of the emotion engine.

[0401] Chatbot: Generates appropriate responses corresponding to emotions and returns them to the server.

[0402] Server: Sends a response message to the terminal.

[0403] Terminal: Display a response message such as "Don't worry, this may improve with treatment."

[0404] Step 7:

[0405] User: Set up integration with the health management app on the app settings screen.

[0406] Device: Sends a request to connect with the health management app to the server.

[0407] Server: Accesses the health management app and retrieves the necessary data.

[0408] Server: Passes acquired health data to the generative AI model and emotion engine.

[0409] Generative AI model: Analyzes health data and assesses the user's health status.

[0410] Emotion Engine: Integrates health data with your current emotional state to provide comprehensive analysis.

[0411] Server: Sends health assessment results and advice to the device.

[0412] Device: Display advice such as, "It appears you've been getting less sleep recently. This may be contributing to your poor health."

[0413] Step 8:

[0414] Chatbot: The generative AI model determines whether the user's inquiry is urgent or not based on the content of the inquiry and the results of emotional analysis.

[0415] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[0416] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[0417] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[0418] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[0419] Example 2

[0420] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0421] In remote medical consultations, it can be difficult for users to accurately communicate their symptoms, and responses that ignore the user's emotional state can result in insufficient medical support.Furthermore, there is a lack of a system for appropriately utilizing users' health management data, making it difficult to provide appropriate advice and follow-up that is tailored to each individual user.

[0422] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0423] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; and a means for recognizing the user's emotional state and providing appropriate responses and medical support through the generative AI model. This enables accurate understanding of the user's symptoms and personalized medical support that takes their emotional state into account. Furthermore, by appropriately utilizing health management data, it is possible to provide advice tailored to each individual user and follow-up according to the level of urgency.

[0424] "Telemedical consultation" is a system that allows users to receive medical consultation through a communication network without physically visiting a medical institution.

[0425] A "generative AI model" is an algorithm or system that uses artificial intelligence to analyze and understand user input and generate an appropriate response.

[0426] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses to access the system.

[0427] A "health management app" is an application that collects and manages a user's health-related data (e.g., sleep data, exercise data, vital signs).

[0428] "Emotional state" refers to the user's emotional state (e.g., stress, anxiety, anger) as analyzed by the emotion engine.

[0429] The "emotion engine" is a component that analyzes emotions from user input and provides that information to the generative AI model.

[0430] "Medical professionals" are professionals with specialized medical knowledge, such as doctors, nurses, and pharmacists.

[0431] "Follow-up" refers to subsequent medical intervention to provide additional diagnosis or advice based on the user's consultation.

[0432] "Analysis" is the process of understanding input data and extracting or generating meaningful information based on it.

[0433] MODE FOR CARRYING OUT THE INVENTION

[0434] The present invention relates to a system for remote medical consultations. This system combines a generative AI model and an emotion engine to provide appropriate medical support based on the user's input and emotional state. Specific embodiments for implementing the present invention are described below.

[0435] System Configuration

[0436] This system mainly consists of the following components:

[0437] User devices (e.g. smartphones, tablets, PCs)

[0438] server

[0439] Generative AI Models

[0440] Emotion Engine

[0441] Health management app

[0442] Linking to medical professionals

[0443] The role of each component

[0444] 1. User Device

[0445] This is an electronic device that users use to conduct remote medical consultations. Through an application or web interface, users register and log in to an account and begin a medical consultation.

[0446] 2. Server

[0447] The server plays a central role in the entire system: it manages user authentication information, runs the generative AI model and emotion engine, sends responses to the user's device, and also coordinates data with the health management app and forwards follow-up information to medical professionals.

[0448] 3. Generative AI Models

[0449] It includes algorithms for analyzing user input and generating appropriate responses. The generative AI model generates natural conversations based on the user's symptoms and questions, providing accurate advice to the user.

[0450] 4. Emotion Engine

[0451] It analyzes user input and recognizes emotional states (e.g., stress, anxiety, anger). The emotion engine passes the results to a generative AI model, which then uses it to tailor responses.

[0452] 5. Health Management App

[0453] It is an application that collects and manages users' health data (e.g., sleep data, exercise data, vital signs). This data is sent to a server and analyzed by a generative AI model and emotion engine.

[0454] 6. Linking to medical professionals

[0455] If an emergency or specialized treatment is deemed necessary, the server will send a request to a medical professional who will provide the user with additional diagnosis and advice via video or text chat.

[0456] Example of operation

[0457] 1. User Registration and Login

[0458] A user launches the application, enters their name, email address, and password in the registration form, and clicks "Register."

[0459] The server saves the entered information in a database and notifies the user that "Registration is complete. Please log in."

[0460] 2. Initiating a medical consultation

[0461] The user taps "Start medical consultation" on the app's home screen.

[0462] The server launches an instance of the generative AI model and chatbot, which then displays "Hello, how can I help you?"

[0463] 3. Conversational exchanges

[0464] The user types, "Recently, I've been coughing so much at night that I can't sleep."

[0465] The chatbot responds, "That's terrible. How long have you had this cough?"

[0466] The user responds, "About a week."

[0467] The chatbot asks additional questions, such as, "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[0468] 4. Emotion Recognition by Emotion Engine

[0469] The server sends the user's input to the emotion engine.

[0470] The emotion engine recognizes the user's anxiety and passes that information to the generative AI model.

[0471] The generative AI model responds, "Okay, don't worry, this can get better with treatment."

[0472] 5. Health Management Data Linkage

[0473] The user configures the settings to link with the health management app.

[0474] A server collects health management data and sends this information to a generative AI model and emotion engine.

[0475] The chatbot advises, "It seems you've been sleeping less recently. This may be causing your health to deteriorate."

[0476] 6. Follow-up with a medical professional

[0477] The chatbot will notify the user, "If this condition persists, you may need to see a specialist."

[0478] The server sends follow-up requests to medical professionals and schedules video chats.

[0479] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[0480] Prompt Sentence Examples

[0481] "Provide advice to users based on their recent health data."

[0482] "Analyze the user's emotional state and generate a conversation to provide appropriate support."

[0483] "Please outline your follow-up procedures when urgent symptoms are reported."

[0484] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0485] A detailed explanation of the program's processing steps

[0486] Step 1: User registration and login

[0487] A user opens the application or web interface, enters their name, email address, and password in the new registration form, and clicks the "Register" button.

[0488] Input: Name, Email Address, Password

[0489] The server receives the entered information and stores it in a database.

[0490] Output: Notification of successful registration

[0491] The server notifies the user, "Registration complete. Please log in."

[0492] Step 2: Log in

[0493] The user enters their email address and password on the login screen and clicks the "Login" button.

[0494] Input: Email address, password

[0495] The server checks the entered credentials against its database.

[0496] Output: Notification of successful or failed login

[0497] If the authentication is successful, the server notifies the user that "Login was successful" and displays the dashboard screen.

[0498] Step 3: Initiating a medical consultation

[0499] The user taps "Start medical consultation" on the app's home screen.

[0500] Input: Request to start a medical consultation

[0501] The server launches an instance of the generated AI model and chatbot.

[0502] Output: Chatbot interface displayed

[0503] The server displays the chatbot interface on the user's device and asks the user, "Hello. What would you like to discuss with us?"

[0504] Step 4: Dialogue

[0505] The user inputs their symptoms into the chatbot (e.g., "I've been coughing so much at night lately that I can't sleep").

[0506] Input: User's symptoms

[0507] The chatbot uses a generative AI model to analyze user input.

[0508] Output: Generate an appropriate question (e.g., "How long has your cough lasted?")

[0509] The chatbot returns the generated question to the user.

[0510] Step 5: Emotion Recognition with the Emotion Engine

[0511] The server sends the user's input to the emotion engine.

[0512] Input: What the user types

[0513] The emotion engine analyzes the input and recognizes the user's emotional state (e.g., anxiety, stress, anger).

[0514] Output: Emotional state analysis results

[0515] The emotion engine passes the analysis results to the generative AI model.

[0516] The generative AI model adjusts its response based on the emotional state, saying, "Okay, don't worry, this can get better with treatment."

[0517] Step 6: Health management data integration

[0518] The user sets up linkage with the health management app on the settings screen.

[0519] Input: Health management app link information

[0520] The server acquires health data from the health management app.

[0521] Output: Health data collection

[0522] The server sends the collected health data to the generative AI model and emotion engine.

[0523] The generative AI model analyzes health data, and the chatbot advises, "It appears you've been sleeping less recently. This may be contributing to your poor health."

[0524] Step 7: Follow up with a medical professional

[0525] The chatbot analyzes the user's symptoms and input, and determines whether the condition is urgent or requires specialized treatment.

[0526] Input: User symptoms and input

[0527] The server sends a follow-up request to the medical professional.

[0528] Output: Send follow-up request

[0529] The server schedules the video chat and notifies the user.

[0530] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[0531] (Application example 2)

[0532] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0533] While remote medical consultation systems provide support for users regarding their health status, they face challenges in providing personalized advice that takes into account the user's emotional state and in providing insufficient security measures. Furthermore, because the user's emotional state can affect the overall response quality of the system, there is a need for an analysis of the user's emotional state and the generation of appropriate responses based on that state.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; hardware including an emotion engine that analyzes and recognizes the user's emotional state; and a means for generating a response to the user based on the recognized emotional state and providing security advice. This enables personalized medical support and security measures that take the user's emotional state into consideration.

[0535] A "means for a user to initiate a remote medical consultation" is an interface that allows a user to initiate a consultation with a medical professional using communication technology.

[0536] "Means of using a generative AI model to analyze user input and generate a response" refers to the process of using artificial intelligence to analyze information entered by a user and generate an appropriate response based on that information.

[0537] "Means for displaying the response generated by the generative AI model on the user's device" refers to a method for visualizing the response created by the generative AI model on the user's device.

[0538] The "means for acquiring and analyzing user health data in cooperation with a health management application" is a method for collecting and analyzing user health-related data in cooperation with a health management application.

[0539] The "means for providing advice to the user based on the acquired health data" is a process of analyzing the collected health data and providing appropriate advice to the user based on the data.

[0540] "Means for transferring the user's consultation to a medical professional as needed and for carrying out follow-up" refers to a method for transferring the user's consultation to a medical professional when the user's consultation requires specialized treatment and for carrying out continuous follow-up.

[0541] "Hardware including an emotion engine that analyzes and recognizes a user's emotional state" is a specific hardware device that has the function of analyzing and recognizing a user's emotion.

[0542] The "means for generating a response to a user based on a recognized emotional state and providing security advice" is a method for generating an appropriate response based on the user's emotions recognized by the emotion engine and further providing security advice to the user.

[0543] The present invention combines an emotion engine with a remote medical consultation system, recognizing the user's emotional state and providing appropriate responses and security advice through a generative AI model.

[0544] A system for implementing the present invention uses the following major hardware and software components:

[0545] Hardware

[0546] Smartphone: The device on which the user operates the application.

[0547] Server: A device that hosts data processing and generative AI models, emotion engines, and databases.

[0548] Emotion engine: Dedicated hardware for analyzing user input and recognizing emotional states.

[0549] software

[0550] Application interface: An app that allows users to initiate medical consultations and input emotional and health data.

[0551] Generative AI model: An algorithm that analyzes user input and generates an appropriate response.

[0552] REST API server: Software that handles communication between the client (smartphone app) and the server.

[0553] Health management app: Software that collects and provides user health data.

[0554] Database: A storage system for storing user information, health data, and emotional state data.

[0555] Processing steps

[0556] 1. User Registration

[0557] The user launches the smartphone app and enters the required registration information (name, email address, password). The application interface sends this information to the REST API server, which stores it in a database. Once registration is complete, the server returns a confirmation message.

[0558] 2. Log in

[0559] A user logs in from a smartphone app using their email address and password. The information is sent back to the REST API server, which verifies the authentication information in the database. If authentication is successful, the server issues a session token.

[0560] 3. Initiating a medical consultation

[0561] The user presses the "Start medical consultation" button within the application, and their smartphone sends this request to the server, which then activates the generative AI model and displays the chatbot interface on the user's device.

[0562] 4. Conversational exchanges

[0563] The chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation details and symptoms. For example, if a user inputs "I've been coughing so much at night recently that I can't sleep," the generative AI model will respond with "How long has this cough been going on for?"

[0564] 5. Emotion Recognition by Emotion Engine

[0565] The server sends the user's conversation content to the emotion engine, which recognizes the user's emotional state. The recognized emotional state is passed to the generative AI model, which adjusts the response. For example, if the user's anxiety is recognized, the generative AI model responds, "Don't worry, this can get better with treatment."

[0566] 6. Health Management Data Linkage

[0567] The user connects their health management app to the system, and health data (e.g., sleep time, steps taken, heart rate, etc.) is sent to the server. This data is analyzed using a generative AI model and an emotion engine. The chatbot then advises, "It seems you've been getting less sleep recently. This may be contributing to your poor health."

[0568] 7. Follow-up with a medical professional

[0569] If advanced medical support is deemed necessary, the generative AI model notifies the system and transfers the user's consultation to a medical professional, who can provide additional diagnosis and advice via video or text chat.

[0570] Examples of concrete examples and prompts

[0571] 1. Example:

[0572] User registration: A user registers in the app by entering "test_user", "user@example.com", and "password123".

[0573] Login: Log in with the same user information.

[0574] Security Session: Tap the "Start Security Session" button to start the session.

[0575] Emotional data transmission: Data expressing "anxiety" is transmitted via the app.

[0576] Security advice: "There has been an increase in the number of accesses to your device recently. Please set up two-step authentication."

[0577] 2. Example prompt:

[0578] User: I've been feeling a bit uneasy with my devices lately. Any advice?

[0579] Chatbot: That's unfortunate. We've noticed some suspicious activity in your recent logs. We recommend you enable two-factor authentication and change your password. If you'd like more information, please type "Tell me more."

[0580] This will specifically specify the mode for carrying out the present invention, and serve as a reference for others to accurately understand and practice the invention.

[0581] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0582] Step 1: User Registration

[0583] Input: A user uses a smartphone app to enter their name, email address, and password.

[0584] Processing: The smartphone app sends the entered information to the REST API server, which saves the information in a database and completes the user registration.

[0585] Output: The server sends a successful registration notification back to the app, and the app displays "Registration successful" to the user.

[0586] Step 2: Log in

[0587] Input: The user enters the email address and password they registered on the smartphone app.

[0588] Processing: The smartphone app sends the entered authentication information to the REST API server. The server checks the authentication information by referencing the database, and if authentication is successful, issues a session token.

[0589] Output: The server sends a successful authentication response and a session token back to the app, and the app notifies the user that the login was successful.

[0590] Step 3: Initiating a medical consultation

[0591] Input: The user presses the "Start medical consultation" button in the smartphone app.

[0592] Processing: The smartphone app sends a request to start a consultation to the REST API server. The server starts the generative AI model and prepares to display the chatbot interface on the user's device.

[0593] Output: The server displays the chatbot interface on the user device and asks the user, "Hello, how can I help you?"

[0594] Step 4: Dialogue

[0595] Input: The user inputs their symptoms and concerns into the chatbot (e.g., "I've been having a bad cough lately and can't sleep at night").

[0596] Processing: The chatbot uses a generative AI model to analyze the user's input data and generate an appropriate response (e.g., "How long have you had a cough?").

[0597] Output: The server sends the generated response to the user's device, where the chatbot displays it to the user.

[0598] Step 5: Emotion Recognition with the Emotion Engine

[0599] Input: Text data entered by the user (e.g., "I've been coughing so much at night lately I can't sleep").

[0600] Processing: The server sends the text data to the emotion engine, which analyzes it and recognizes the user's emotional state (e.g., anxiety) as a result of the analysis.

[0601] Output: The emotion engine passes the recognized emotional state to the generative AI model, which tailors the response based on the emotion. The tailored response is displayed on the user's device.

[0602] Step 6: Health management data integration

[0603] Input: Health data collected from health management apps (e.g., sleep duration, steps, heart rate, etc.).

[0604] Processing: The server connects with the health management app to acquire health data, which is then analyzed using a generative AI model and emotion engine to evaluate the user's health status.

[0605] Output: Health advice generated based on the analysis results (e.g., "It appears you've been sleeping less recently. This may be causing your health condition to worsen") is displayed on the user's device.

[0606] Step 7: Follow up with a medical professional

[0607] Input: User consultation details and health data for any issues deemed urgent or requiring specialized care.

[0608] Processing: Based on the judgment of the generated AI model, the server forwards the user's consultation to a medical professional, who provides further diagnosis and advice to the user via video chat or text chat.

[0609] Output: The user is notified that the follow-up appointment has been scheduled and completed. The user will then have a video or text conversation with a medical professional at the designated time.

[0610] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0611] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0612] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0613] [Second embodiment]

[0614] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0615] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0616] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0617] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0618] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0619] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0620] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0621] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0622] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0623] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0624] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0625] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0626] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[0627] Overall system configuration

[0628] 1. User Registration and Login

[0629] User: A user accesses the system through an app or web interface and enters the required registration information, including name, email address, and password.

[0630] Server: The server saves the entered user information in a database and notifies the user that registration is complete. After that, the user enters their authentication information on the login screen and is granted access to the system.

[0631] 2. Initiating a medical consultation

[0632] User: When the user presses the "Start medical consultation" button within the app, the server launches the generative AI model and displays the chatbot interface on the user's device.

[0633] Chatbot: The chatbot uses generative AI models to engage in natural conversations with users and identify their concerns and symptoms.

[0634] 3. Conversational exchanges

[0635] User: The user enters their symptoms and concerns into the chatbot. For example, they might say, "I've been coughing so much lately I can't sleep at night."

[0636] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[0637] 4. Health Management Data Linkage

[0638] User: Health data is collected when the user connects the health management app to the system on the settings screen.

[0639] Server: The server passes data obtained from the health management app to the generative AI model and analyzes the user's health condition.

[0640] Chatbot: Provides users with advice based on their health data, such as, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[0641] 5. Follow-up with a medical professional

[0642] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[0643] Server: Transfers the user's consultation to a medical professional and performs follow-up, if necessary.

[0644] Medical Expert: A medical expert provides additional diagnosis and advice to users via video or text chat.

[0645] Specific examples

[0646] 1. User Registration

[0647] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[0648] Server: Notify "Registration complete. Please log in."

[0649] 2. Medical consultation begins

[0650] User: Tap "Start Medical Consultation" on the app's home screen.

[0651] Server: The chatbot displays "Hello, how can we help you?"

[0652] 3. Conversational exchanges

[0653] User: Type "I've been coughing so much at night lately I can't sleep."

[0654] Chatbot: "That's terrible. How long have you had a cough?"

[0655] User: "About a week."

[0656] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[0657] 4. Health Management Data Linkage

[0658] User: Set up the connection to the health management app.

[0659] Server: Collects health management data and sends this information to the generative AI model.

[0660] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[0661] 5. Follow-up with a medical professional

[0662] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[0663] Server: Send follow-up requests to medical professionals and schedule video chats.

[0664] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[0665] The present invention enables even patients who have difficulty moving around to easily receive medical consultations from home, further supporting users' health management.

[0666] The processing flow will be explained below.

[0667] Step 1:

[0668] User: Launches the application and clicks the "Sign Up" button.

[0669] On your device: Display a form for name, email address, password, etc.

[0670] User: Enter the required information and press the "Register" button.

[0671] Terminal: Sends the entered information to the server.

[0672] Step 2:

[0673] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[0674] Device: Display "Registration complete. Please log in."

[0675] User: Enter your email address and password on the login screen and click the "Login" button.

[0676] Device: Sends authentication information to the server.

[0677] Step 3:

[0678] Server: Retrieves user information from the database and verifies authentication information.

[0679] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[0680] On the device: Display a login success message and go to the home screen.

[0681] Step 4:

[0682] User: Press the "Start medical consultation" button on the home screen.

[0683] Device: Sends a request to the server to launch the generative AI model.

[0684] Server: Initializes the generative AI model and starts the chatbot session.

[0685] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[0686] Step 5:

[0687] User: Writes down symptoms and concerns in the chat interface and sends it.

[0688] Terminal: Sends the user's messages to the server.

[0689] Server: Passes messages to the generative AI model for analysis.

[0690] Chatbot: Generates appropriate response messages and returns the results to the server.

[0691] Server: Sends a response message to the terminal.

[0692] Terminal: Display the response message.

[0693] Step 6:

[0694] User: Enters additional information in response to the chatbot's questions and submits.

[0695] Terminal: Sends the user's messages to the server.

[0696] Server: Passes messages to the generative AI model.

[0697] Chatbot: Based on the analysis results, further questions and advice are generated and returned to the server.

[0698] Server: Sends a response message to the terminal.

[0699] Terminal: Display the response message.

[0700] Step 7:

[0701] User: Set up integration with the health management app on the app settings screen.

[0702] Device: Sends a request to connect with the health management app to the server.

[0703] Server: Accesses the health management app and retrieves the necessary data.

[0704] Server: Passes the acquired health data to the generative AI model.

[0705] Generative AI model: Analyzes data and assesses the user's health status.

[0706] Server: Sends the health assessment results to the device.

[0707] Device: Displays advice to the user based on health data.

[0708] Step 8:

[0709] Chatbot: A generative AI model analyzes the user's inquiry and determines whether it is urgent.

[0710] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[0711] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[0712] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[0713] Example 1

[0714] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0715] In remote medical consultations, there is a need for a system that allows users to easily receive medical consultations from home, efficiently manage their health status, and quickly connect with medical professionals in emergencies. Conventional systems have complicated user registration and login procedures, and the quality of responses from generative AI models is insufficient. Furthermore, the collection and analysis of health information through integration with health management software was not performed effectively, resulting in a lack of accurate advice for users. Furthermore, there were issues with smooth follow-up with medical professionals in emergencies.

[0716] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0717] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health information in cooperation with health management software; a means for providing the user with advice based on the acquired health information; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; a means for the user to enter authentication information such as their name, email address, and password to register and log in to the system; a means for transmitting the user's input symptoms and concerns to the generative AI model and generating an appropriate response; and a means for launching the generative AI model on the user's terminal and providing a chatbot interface that engages in natural conversation with the user. This allows users to conveniently receive medical consultations from home and receive high-quality responses through the generative AI model. Furthermore, through effective collection and analysis of health information in cooperation with health management software, the server can provide accurate advice to the user and quickly collaborate with medical professionals for follow-up in emergencies.

[0718] "User" refers to an individual who uses this system to conduct a remote medical consultation.

[0719] "Telemedical consultation" refers to medical consultation conducted remotely via the Internet.

[0720] A "generative AI model" refers to artificial intelligence that analyzes user input data and generates appropriate responses.

[0721] "User terminal" refers to the device (smartphone, PC, etc.) used by a user to access this system.

[0722] "Health management software" refers to applications and platforms for collecting, managing, and analyzing users' health data.

[0723] "Health information" refers to data related to the user's health condition (e.g., sleep time, amount of exercise, weight, etc.).

[0724] "Authentication Information" means the information, such as name, email address, and password, that a User uses to register and log in to the System.

[0725] "Chatbot interface" refers to the screen or application through which a user and a generative AI model interact.

[0726] "Medical Expert" refers to a doctor or other medical professional who provides follow-up care to the user based on the user's consultation.

[0727] "Follow-up" refers to providing additional diagnosis or advice to the user.

[0728] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[0729] User Registration and Login

[0730] Users access the system through an app or web interface and register by entering their name, email address, and password, which then sends the authentication information to the server.

[0731] The server stores the entered user authentication information in a database using a database management system such as MySQL, and notifies the user when registration is complete.

[0732] The user logs in with the registered information and accesses the system. The server checks the user's authentication information against the database, and displays the home screen only if authentication is successful.

[0733] Initiating a medical consultation

[0734] When a user selects "Start medical consultation" on the home screen, the server launches a generative AI model (e.g., OpenAI's GPT-3).

[0735] The server initializes the generative AI model, starts the chatbot session, and displays the chatbot interface on the user's device, ready to begin a healthcare-related conversation.

[0736] Conversational exchange

[0737] Users can input their symptoms and concerns into the chatbot. For example, they can enter something like, "I've been coughing so much at night recently that I can't sleep."

[0738] The server sends this input to a generative AI model that generates a response, such as a question like, "How long has your cough lasted?"

[0739] The generative AI model digs deeper into the user's concerns through ongoing conversation and asks more detailed questions.

[0740] Health management data linkage

[0741] When a user connects the system to health management software (e.g., Google Fit or Apple Health) on the settings screen, the server obtains data from the health software.

[0742] The server sends the acquired health information to a generative AI model to analyze the user's health condition.

[0743] The chatbot will then provide appropriate advice to the user based on the analyzed data. For example, it could say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[0744] Follow-up with a medical professional

[0745] If the chatbot determines that the user's symptoms are urgent, it notifies the server of this information.

[0746] The server forwards the user's consultation to a medical professional for follow-up as needed.

[0747] Medical professionals can provide additional diagnosis and advice to users via video or text chat, allowing users to receive professional advice quickly.

[0748] Providing concrete examples

[0749] In a specific scenario in which a user is seeking medical advice, the following prompt sentences may be used:

[0750] Example prompt: "I've been coughing so much at night lately that I can't sleep. What should I do?"

[0751] This invention allows patients with mobility issues to easily receive medical consultations from home, receiving high-quality responses and accurate advice. It also enables prompt collaboration with medical professionals in emergencies, enabling appropriate follow-up. This system, utilizing generative AI models, further supports users' health management.

[0752] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0753] Step 1: User Registration

[0754] Input: User's name, email address, and password

[0755] Specific operation:

[0756] Through the app or web interface, users enter their name, email address, and password and click the "Register" button.

[0757] The server stores the information entered by the user in a database using MySQL.

[0758] The server will notify you, "Registration complete. Please log in."

[0759] Output: Registration completion notification

[0760] Step 2: Login authentication

[0761] Input: User's email address, password

[0762] Specific operation:

[0763] The user enters the registered email address and password and clicks the "Login" button.

[0764] The server checks the user's credentials against a database.

[0765] If the server successfully authenticates the user, it displays the home screen, but if it fails, it notifies the user that "Authentication failed."

[0766] Output: Home screen display or authentication failure notification

[0767] Step 3: Initiating a medical consultation

[0768] Input: Click on the "Start medical consultation" button

[0769] Specific operation:

[0770] The user selects "Start medical consultation" on the home screen.

[0771] The server initializes a generative AI model (e.g., OpenAI's GPT-3).

[0772] The server sends the chatbot interface to the user terminal and starts the session.

[0773] Output: Chatbot interface displayed

[0774] Step 4: Accepting User Input

[0775] Input: User's symptoms and concerns

[0776] Specific operation:

[0777] The user enters their symptoms and concerns in the text box and clicks the send button.

[0778] Output: Text data of input symptoms and anxieties

[0779] Step 5: Generate a response using a generative AI model

[0780] Input: Text data of user symptoms and anxieties

[0781] Specific operation:

[0782] The server sends the user's input text to the generative AI model.

[0783] The generative AI model analyzes the input and generates an appropriate response.

[0784] The server sends the generated response to the user terminal.

[0785] Output: The generated response text

[0786] Step 6: View the response

[0787] Input: Generated response text

[0788] Specific operation:

[0789] The terminal displays the response received from the server on the chatbot interface.

[0790] Output: The response displayed on the chatbot interface

[0791] Step 7: Link with health management app

[0792] Input: Select health management app and click link button

[0793] Specific operation:

[0794] The user configures the settings screen to link with a health management app (e.g., Google Fit or Apple Health).

[0795] The server calls the API of the selected health management app and sets up the connection.

[0796] Output: Notification that the connection with the health management app has been completed

[0797] Step 8: Acquire and analyze health data

[0798] Input: Data obtained from health management app

[0799] Specific operation:

[0800] The server uses the health management app's API to obtain the user's health data.

[0801] The server sends the acquired data to the generative AI model.

[0802] The generative AI model analyzes health data and generates advice about the user's health status.

[0803] The server transmits the generated advice to the user terminal.

[0804] Output: Health advice

[0805] Step 9: Providing advice based on health data

[0806] Input: Generated advice

[0807] Specific operation:

[0808] The terminal displays the advice received from the server on the chatbot interface.

[0809] Output: Display of health advice

[0810] Step 10: Emergency response decision and notification

[0811] Input: User symptoms and health data

[0812] Specific operation:

[0813] The chatbot analyzes the user's symptoms and health data to determine whether emergency response is required.

[0814] If the chatbot determines that an emergency response is required, it notifies the server.

[0815] Output: Notification that urgent action is required

[0816] Step 11: Consult a medical professional

[0817] Input: Notification that emergency action is required

[0818] Specific operation:

[0819] The server notifies the medical professional of the user's consultation and the urgency of the consultation, and performs follow-up.

[0820] Medical professionals will provide additional diagnosis and advice to users via video or text chat.

[0821] Output: Notification that a follow-up with a medical professional will be performed

[0822] The system allows users to easily receive medical consultations from home, receive high-quality responses, and is excellent for quickly connecting with medical professionals in emergencies.

[0823] (Application example 1)

[0824] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0825] While remote medical consultation systems allow users to easily receive medical consultations from home, providing prompt and appropriate support during health consultations at physical stores has been difficult. For this reason, new methods are needed for store staff to provide customers with appropriate health advice in real time and efficiently recommend products.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0827] In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; and a means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses. This enables quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[0828] "User" refers to an individual who uses the system to receive remote medical consultations or in-store health consultations.

[0829] "Telemedical consultation" means a service that allows a user to consult with a medical professional online from the comfort of their own home or any other location.

[0830] A "generative AI model" refers to artificial intelligence technology that analyzes user input and generates appropriate responses and advice through natural conversation.

[0831] "User Terminal" means a device (e.g., smartphone, tablet, computer) through which a User accesses the System and conducts a remote medical consultation.

[0832] "Health management app" refers to an application that collects and manages a user's health data.

[0833] "Server" means the computer system that manages and operates the entire system and processes information from the generative AI model and database.

[0834] "Smart glasses" refer to wearable devices that provide information along the user's line of sight and assist staff in providing real-time health consultations in brick-and-mortar stores.

[0835] "Medical Professional" means a medical professional who is qualified to provide professional diagnosis and advice regarding the user's health inquiry.

[0836] The present invention provides a health consultation support system for use in a physical store, which allows users to receive health consultations in real time through store staff wearing smart glasses. Detailed embodiments of the present invention will be described below.

[0837] The server includes means for a user to initiate a remote medical consultation, means for analyzing the user's input using a generative AI model and generating a response, means for displaying the response generated by the generative AI model on the user's terminal, means for acquiring and analyzing the user's health data in cooperation with a health management app, means for providing advice to the user based on the acquired health data, means for transferring the user's consultation content to a medical professional and performing follow-up as necessary, and means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses.

[0838] 1. Overall system configuration

[0839] The system consists of the following main components:

[0840] User devices: smartphones, tablets, computers, etc.

[0841] Server: Cloud-based computing services.

[0842] Generative AI models: such as OpenAI's GPT-4.

[0843] Smart glasses: Google Glass, etc.

[0844] Health management app: An application that collects and manages health data.

[0845] 2. System Operation

[0846] 2.1 User Registration and Login

[0847] The server provides a means for users to access the system through an app or web interface and enter the necessary registration information (name, email address, password, etc.) The server stores this information in a database and manages the authentication information.

[0848] 2.2 Starting a health consultation

[0849] When a user presses the "Start medical consultation" button, the server activates the generative AI model and displays a chatbot interface on the user's device. This chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation content and symptoms.

[0850] 2.3 Conversational exchanges

[0851] When a user inputs a question or symptom into the system, the generative AI model analyzes it and provides an appropriate answer, which is displayed on the user's device and the smart glasses' display.

[0852] 2.4 Health Management Data Linkage

[0853] When a user connects a health management app to the system, the server acquires this data and sends it to the generative AI model, which then analyzes the user's health condition and provides appropriate advice.

[0854] 2.5 Real-time health consultations at physical stores

[0855] When a store employee wearing smart glasses receives a health consultation from a customer, the server sends the information to the generative AI model, which generates a response. This response is displayed in real time on the employee's smart glasses, and the employee can then provide appropriate advice to the customer.

[0856] Specific examples

[0857] For example, if a customer asks how to choose vitamins, the following exchange might occur:

[0858] Staff: "What purpose are you looking for vitamins for?"

[0859] Customer: "I'm feeling tired and I'd like to have more energy."

[0860] Smart glasses display: "To increase your energy, we recommend supplements containing B and C vitamins. Iron may also be beneficial. Do you have any other questions? Learn more about these products."

[0861] Prompt Sentence Examples

[0862] "A client is asking about choosing vitamins. Can you recommend some vitamins or supplements to help them increase their energy?"

[0863] This will enable quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[0864] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0865] Step 1:

[0866] A user accesses the system through an app or web interface and registers. The user enters the required information, such as name, email address, and password, and sends it to the server. The server stores this information in a database and notifies the user that registration is complete.

[0867] Input: Name, Email Address, Password

[0868] Data processing: Saving user information to a database

[0869] Output: Registration completion notification

[0870] Step 2:

[0871] A user logs in to the system. The user enters the registered email address and password and sends the authentication information to the server. The server authenticates the user and allows the user to access the system.

[0872] Input: Email address, Password

[0873] Data Calculation: Authentication Check

[0874] Output: System access permissions

[0875] Step 3:

[0876] The user presses the "Start medical consultation" button. The server launches the generative AI model and displays the chatbot interface on the user's device. The user's device then begins a dialogue with the user.

[0877] Input: Medical consultation start request

[0878] Data processing: Displaying the chatbot interface

[0879] Output: Medical consultation initiated

[0880] Step 4:

[0881] The user inputs symptoms or questions into the chatbot. The user's device sends the input text to the generative AI model. The server receives it, analyzes it with the generative AI model, and generates an appropriate response.

[0882] Input: User question or symptom

[0883] Data Computation: Question Analysis and Answer Generation

[0884] Output: The generated response

[0885] Step 5:

[0886] The server sends the generated response to the user terminal for display, which displays it to the user for further interaction.

[0887] Input: The generated response

[0888] Data processing: Sending and displaying responses

[0889] Output: Display response to user

[0890] Step 6:

[0891] The user configures the health management app to work with the server, which then acquires the user's health data from the app and sends it to the generative AI model for analysis.

[0892] Input: Health management app data

[0893] Data Computing: Health Data Analysis

[0894] Output: Advice based on health status

[0895] Step 7:

[0896] A user consults a staff member wearing smart glasses at a physical store. The staff member's smart glasses send the user's consultation to the generative AI model, which analyzes it on the server. The server then sends the answer to the staff member's smart glasses and displays it.

[0897] Input: Customer Question

[0898] Data Computation: Question Analysis and Answer Generation

[0899] Output: Response display on smart glasses

[0900] Step 8:

[0901] If necessary, the server will follow up by transferring the user's consultation to a medical professional who will provide further diagnosis and advice via video or text chat.

[0902] Input: User's inquiry

[0903] Data processing: Transfer of consultation details

[0904] Output: Follow-up with medical professionals

[0905] The above processing steps make it possible to provide prompt and appropriate health consultations even in physical stores.

[0906] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0907] The present invention combines an emotion engine with a remote medical consultation system, which can recognize the user's emotional state and provide appropriate responses and medical support through a generative AI model. Specific embodiments of the present invention are described below.

[0908] Overall system configuration

[0909] 1. User Registration and Login

[0910] User: Accesses the system through an application or web interface and creates a new account by entering the required registration information (name, email address, password, etc.).

[0911] Server: Saves the entered user information in the database and notifies the user that registration is complete. Once the user enters their authentication information on the login screen, they are granted access to the system.

[0912] 2. Initiating a medical consultation

[0913] User: When the user presses the "Start medical consultation" button within the application, the server launches the generative AI model and displays the chatbot interface on the user's device.

[0914] Chatbot: The chatbot uses generative AI models to have natural conversations with users and confirm their concerns and symptoms.

[0915] 3. Conversational exchanges

[0916] User: Enters symptoms and concerns into the chatbot. For example, "I've been coughing so much lately I can't sleep at night."

[0917] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[0918] 4. Emotion Recognition by Emotion Engine

[0919] Server: Sends user input to the emotion engine and analyzes the emotional state.

[0920] Emotion engine: Recognizes user emotions (e.g., stress, anxiety, anger) and passes the results to a generative AI model.

[0921] Generative AI model: Tailors responses based on emotional state to provide appropriate support to users.

[0922] 5. Health Management Data Linkage

[0923] User: Health data is obtained by linking the health management app with the system from the settings screen.

[0924] Server: Sends data obtained from the health management app to the emotion engine and generative AI model to analyze the user's health condition.

[0925] Chatbot: Provides users with advice based on their health data and emotional state. For example, it might say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[0926] 6. Follow-up with a medical professional

[0927] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[0928] Server: Transfers the user's consultation to a medical professional, if necessary, and arranges for follow-up.

[0929] Medical Expert: Provides additional diagnosis and advice to users via video or text chat.

[0930] Specific examples

[0931] 1. User Registration

[0932] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[0933] Server: Notify "Registration complete. Please log in."

[0934] 2. Medical consultation begins

[0935] User: Tap "Start Medical Consultation" on the app's home screen.

[0936] Server: The chatbot displays "Hello, how can we help you?"

[0937] 3. Conversational exchanges

[0938] User: Type "I've been coughing so much at night lately I can't sleep."

[0939] Chatbot: "That's terrible. How long have you had a cough?"

[0940] User: "About a week."

[0941] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[0942] 4. Emotion Recognition by Emotion Engine

[0943] Server: Sends the user's input, "About a week" and "Are you experiencing any other symptoms such as fever or fatigue?" to the emotion engine.

[0944] Emotion engine: Recognizes user anxieties and passes that information to a generative AI model.

[0945] Generative AI model: Responds, "Okay, don't worry, this can get better with treatment."

[0946] 5. Health Management Data Linkage

[0947] User: Set up the connection to the health management app.

[0948] Server: Collects health management data and sends this information to the generative AI model and emotion engine.

[0949] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[0950] 6. Follow-up with a medical professional

[0951] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[0952] Server: Send follow-up requests to medical professionals and schedule video chats.

[0953] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[0954] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[0955] The processing flow will be explained below.

[0956] The present invention is a remote medical consultation system with an emotion engine built in. The specific processing flow will be explained below step by step.

[0957] Overall system configuration

[0958] Step 1:

[0959] User: Launches the application and clicks the "Sign Up" button.

[0960] On your device: Display a form for name, email address, password, etc.

[0961] User: Enter the required information and press the "Register" button.

[0962] Terminal: Sends the entered information to the server.

[0963] Step 2:

[0964] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[0965] On the device: Display "Registration successful. Please log in" to the user.

[0966] User: Enter your email address and password on the login screen and click the "Login" button.

[0967] Device: Sends authentication information to the server.

[0968] Step 3:

[0969] Server: Retrieves user information from the database and verifies authentication information.

[0970] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[0971] On the device: Display a login success message and go to the home screen.

[0972] Step 4:

[0973] User: Press the "Start medical consultation" button on the home screen.

[0974] Device: Sends a request to the server to launch the generative AI model.

[0975] Server: Initializes the generative AI model and starts the chatbot session.

[0976] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[0977] Step 5:

[0978] User: Writes down symptoms and concerns in the chat interface and sends it.

[0979] Terminal: Sends the user's messages to the server.

[0980] Server: Passes messages to the generative AI model for analysis.

[0981] Chatbot: Generates appropriate response messages and returns the results to the server.

[0982] Server: Sends a response message to the terminal.

[0983] Terminal: Display the response message.

[0984] Step 6:

[0985] User: Enters additional information in response to the chatbot's questions and submits.

[0986] Terminal: Sends the user's messages to the server.

[0987] Server: Passes messages to the generative AI model and emotion engine.

[0988] Emotion engine: Analyzes user input and recognizes emotional states (e.g., stress, anxiety, relief, etc.).

[0989] Generative AI model: Adjusts responses based on the analysis results of the emotion engine.

[0990] Chatbot: Generates appropriate responses corresponding to emotions and returns them to the server.

[0991] Server: Sends a response message to the terminal.

[0992] Terminal: Display a response message such as "Don't worry, this may improve with treatment."

[0993] Step 7:

[0994] User: Set up integration with the health management app on the app settings screen.

[0995] Device: Sends a request to connect with the health management app to the server.

[0996] Server: Accesses the health management app and retrieves the necessary data.

[0997] Server: Passes acquired health data to the generative AI model and emotion engine.

[0998] Generative AI model: Analyzes health data and assesses the user's health status.

[0999] Emotion Engine: Integrates health data with your current emotional state to provide comprehensive analysis.

[1000] Server: Sends health assessment results and advice to the device.

[1001] Device: Display advice such as, "It appears you've been getting less sleep recently. This may be contributing to your poor health."

[1002] Step 8:

[1003] Chatbot: The generative AI model determines whether the user's inquiry is urgent or not based on the content of the inquiry and the results of emotional analysis.

[1004] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[1005] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[1006] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[1007] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[1008] Example 2

[1009] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1010] In remote medical consultations, it can be difficult for users to accurately communicate their symptoms, and responses that ignore the user's emotional state can result in insufficient medical support.Furthermore, there is a lack of a system for appropriately utilizing users' health management data, making it difficult to provide appropriate advice and follow-up that is tailored to each individual user.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1012] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; and a means for recognizing the user's emotional state and providing appropriate responses and medical support through the generative AI model. This enables accurate understanding of the user's symptoms and personalized medical support that takes their emotional state into account. Furthermore, by appropriately utilizing health management data, it is possible to provide advice tailored to each individual user and follow-up according to the level of urgency.

[1013] "Telemedical consultation" is a system that allows users to receive medical consultation through a communication network without physically visiting a medical institution.

[1014] A "generative AI model" is an algorithm or system that uses artificial intelligence to analyze and understand user input and generate an appropriate response.

[1015] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses to access the system.

[1016] A "health management app" is an application that collects and manages a user's health-related data (e.g., sleep data, exercise data, vital signs).

[1017] "Emotional state" refers to the user's emotional state (e.g., stress, anxiety, anger) as analyzed by the emotion engine.

[1018] The "emotion engine" is a component that analyzes emotions from user input and provides that information to the generative AI model.

[1019] "Medical professionals" are professionals with specialized medical knowledge, such as doctors, nurses, and pharmacists.

[1020] "Follow-up" refers to subsequent medical intervention to provide additional diagnosis or advice based on the user's consultation.

[1021] "Analysis" is the process of understanding input data and extracting or generating meaningful information based on it.

[1022] MODE FOR CARRYING OUT THE INVENTION

[1023] The present invention relates to a system for remote medical consultations. This system combines a generative AI model and an emotion engine to provide appropriate medical support based on the user's input and emotional state. Specific embodiments for implementing the present invention are described below.

[1024] System Configuration

[1025] This system mainly consists of the following components:

[1026] User devices (e.g. smartphones, tablets, PCs)

[1027] server

[1028] Generative AI Models

[1029] Emotion Engine

[1030] Health management app

[1031] Linking to medical professionals

[1032] The role of each component

[1033] 1. User Device

[1034] This is an electronic device that users use to conduct remote medical consultations. Through an application or web interface, users register and log in to an account and begin a medical consultation.

[1035] 2. Server

[1036] The server plays a central role in the entire system: it manages user authentication information, runs the generative AI model and emotion engine, sends responses to the user's device, and also coordinates data with the health management app and forwards follow-up information to medical professionals.

[1037] 3. Generative AI Models

[1038] It includes algorithms for analyzing user input and generating appropriate responses. The generative AI model generates natural conversations based on the user's symptoms and questions, providing accurate advice to the user.

[1039] 4. Emotion Engine

[1040] It analyzes user input and recognizes emotional states (e.g., stress, anxiety, anger). The emotion engine passes the results to a generative AI model, which then uses it to tailor responses.

[1041] 5. Health Management App

[1042] It is an application that collects and manages users' health data (e.g., sleep data, exercise data, vital signs). This data is sent to a server and analyzed by a generative AI model and emotion engine.

[1043] 6. Linking to medical professionals

[1044] If an emergency or specialized treatment is deemed necessary, the server will send a request to a medical professional who will provide the user with additional diagnosis and advice via video or text chat.

[1045] Example of operation

[1046] 1. User Registration and Login

[1047] A user launches the application, enters their name, email address, and password in the registration form, and clicks "Register."

[1048] The server saves the entered information in a database and notifies the user that "Registration is complete. Please log in."

[1049] 2. Initiating a medical consultation

[1050] The user taps "Start medical consultation" on the app's home screen.

[1051] The server launches an instance of the generative AI model and chatbot, which then displays "Hello, how can I help you?"

[1052] 3. Conversational exchanges

[1053] The user types, "Recently, I've been coughing so much at night that I can't sleep."

[1054] The chatbot responds, "That's terrible. How long have you had this cough?"

[1055] The user responds, "About a week."

[1056] The chatbot asks additional questions, such as, "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[1057] 4. Emotion Recognition by Emotion Engine

[1058] The server sends the user's input to the emotion engine.

[1059] The emotion engine recognizes the user's anxiety and passes that information to the generative AI model.

[1060] The generative AI model responds, "Okay, don't worry, this can get better with treatment."

[1061] 5. Health Management Data Linkage

[1062] The user configures the settings to link with the health management app.

[1063] A server collects health management data and sends this information to a generative AI model and emotion engine.

[1064] The chatbot advises, "It seems you've been sleeping less recently. This may be causing your health to deteriorate."

[1065] 6. Follow-up with a medical professional

[1066] The chatbot will notify the user, "If this condition persists, you may need to see a specialist."

[1067] The server sends follow-up requests to medical professionals and schedules video chats.

[1068] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[1069] Prompt Sentence Examples

[1070] "Provide advice to users based on their recent health data."

[1071] "Analyze the user's emotional state and generate a conversation to provide appropriate support."

[1072] "Please outline your follow-up procedures when urgent symptoms are reported."

[1073] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1074] A detailed explanation of the program's processing steps

[1075] Step 1: User registration and login

[1076] A user opens the application or web interface, enters their name, email address, and password in the new registration form, and clicks the "Register" button.

[1077] Input: Name, Email Address, Password

[1078] The server receives the entered information and stores it in a database.

[1079] Output: Notification of successful registration

[1080] The server notifies the user, "Registration complete. Please log in."

[1081] Step 2: Log in

[1082] The user enters their email address and password on the login screen and clicks the "Login" button.

[1083] Input: Email address, password

[1084] The server checks the entered credentials against its database.

[1085] Output: Notification of successful or failed login

[1086] If the authentication is successful, the server notifies the user that "Login was successful" and displays the dashboard screen.

[1087] Step 3: Initiating a medical consultation

[1088] The user taps "Start medical consultation" on the app's home screen.

[1089] Input: Request to start a medical consultation

[1090] The server launches an instance of the generated AI model and chatbot.

[1091] Output: Chatbot interface displayed

[1092] The server displays the chatbot interface on the user's device and asks the user, "Hello. What would you like to discuss with us?"

[1093] Step 4: Dialogue

[1094] The user inputs their symptoms into the chatbot (e.g., "I've been coughing so much at night lately that I can't sleep").

[1095] Input: User's symptoms

[1096] The chatbot uses a generative AI model to analyze user input.

[1097] Output: Generate an appropriate question (e.g., "How long has your cough lasted?")

[1098] The chatbot returns the generated question to the user.

[1099] Step 5: Emotion Recognition with the Emotion Engine

[1100] The server sends the user's input to the emotion engine.

[1101] Input: What the user types

[1102] The emotion engine analyzes the input and recognizes the user's emotional state (e.g., anxiety, stress, anger).

[1103] Output: Emotional state analysis results

[1104] The emotion engine passes the analysis results to the generative AI model.

[1105] The generative AI model adjusts its response based on the emotional state, saying, "Okay, don't worry, this can get better with treatment."

[1106] Step 6: Health management data integration

[1107] The user sets up linkage with the health management app on the settings screen.

[1108] Input: Health management app link information

[1109] The server acquires health data from the health management app.

[1110] Output: Health data collection

[1111] The server sends the collected health data to the generative AI model and emotion engine.

[1112] The generative AI model analyzes health data, and the chatbot advises, "It appears you've been sleeping less recently. This may be contributing to your poor health."

[1113] Step 7: Follow up with a medical professional

[1114] The chatbot analyzes the user's symptoms and input, and determines whether the condition is urgent or requires specialized treatment.

[1115] Input: User symptoms and input

[1116] The server sends a follow-up request to the medical professional.

[1117] Output: Send follow-up request

[1118] The server schedules the video chat and notifies the user.

[1119] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[1120] (Application example 2)

[1121] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1122] While remote medical consultation systems provide support for users regarding their health status, they face challenges in providing personalized advice that takes into account the user's emotional state and in providing insufficient security measures. Furthermore, because the user's emotional state can affect the overall response quality of the system, there is a need for an analysis of the user's emotional state and the generation of appropriate responses based on that state.

[1123] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; hardware including an emotion engine that analyzes and recognizes the user's emotional state; and a means for generating a response to the user based on the recognized emotional state and providing security advice. This enables personalized medical support and security measures that take the user's emotional state into consideration.

[1124] A "means for a user to initiate a remote medical consultation" is an interface that allows a user to initiate a consultation with a medical professional using communication technology.

[1125] "Means of using a generative AI model to analyze user input and generate a response" refers to the process of using artificial intelligence to analyze information entered by a user and generate an appropriate response based on that information.

[1126] "Means for displaying the response generated by the generative AI model on the user's device" refers to a method for visualizing the response created by the generative AI model on the user's device.

[1127] The "means for acquiring and analyzing user health data in cooperation with a health management application" is a method for collecting and analyzing user health-related data in cooperation with a health management application.

[1128] The "means for providing advice to the user based on the acquired health data" is a process of analyzing the collected health data and providing appropriate advice to the user based on the data.

[1129] "Means for transferring the user's consultation to a medical professional as needed and for carrying out follow-up" refers to a method for transferring the user's consultation to a medical professional when the user's consultation requires specialized treatment and for carrying out continuous follow-up.

[1130] "Hardware including an emotion engine that analyzes and recognizes a user's emotional state" is a specific hardware device that has the function of analyzing and recognizing a user's emotion.

[1131] The "means for generating a response to a user based on a recognized emotional state and providing security advice" is a method for generating an appropriate response based on the user's emotions recognized by the emotion engine and further providing security advice to the user.

[1132] The present invention combines an emotion engine with a remote medical consultation system, recognizing the user's emotional state and providing appropriate responses and security advice through a generative AI model.

[1133] A system for implementing the present invention uses the following major hardware and software components:

[1134] Hardware

[1135] Smartphone: The device on which the user operates the application.

[1136] Server: A device that hosts data processing and generative AI models, emotion engines, and databases.

[1137] Emotion engine: Dedicated hardware for analyzing user input and recognizing emotional states.

[1138] software

[1139] Application interface: An app that allows users to initiate medical consultations and input emotional and health data.

[1140] Generative AI model: An algorithm that analyzes user input and generates an appropriate response.

[1141] REST API server: Software that handles communication between the client (smartphone app) and the server.

[1142] Health management app: Software that collects and provides user health data.

[1143] Database: A storage system for storing user information, health data, and emotional state data.

[1144] Processing steps

[1145] 1. User Registration

[1146] The user launches the smartphone app and enters the required registration information (name, email address, password). The application interface sends this information to the REST API server, which stores it in a database. Once registration is complete, the server returns a confirmation message.

[1147] 2. Log in

[1148] A user logs in from a smartphone app using their email address and password. The information is sent back to the REST API server, which verifies the authentication information in the database. If authentication is successful, the server issues a session token.

[1149] 3. Initiating a medical consultation

[1150] The user presses the "Start medical consultation" button within the application, and their smartphone sends this request to the server, which then activates the generative AI model and displays the chatbot interface on the user's device.

[1151] 4. Conversational exchanges

[1152] The chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation details and symptoms. For example, if a user inputs "I've been coughing so much at night recently that I can't sleep," the generative AI model will respond with "How long has this cough been going on for?"

[1153] 5. Emotion Recognition by Emotion Engine

[1154] The server sends the user's conversation content to the emotion engine, which recognizes the user's emotional state. The recognized emotional state is passed to the generative AI model, which adjusts the response. For example, if the user's anxiety is recognized, the generative AI model responds, "Don't worry, this can get better with treatment."

[1155] 6. Health Management Data Linkage

[1156] The user connects their health management app to the system, and health data (e.g., sleep time, steps taken, heart rate, etc.) is sent to the server. This data is analyzed using a generative AI model and an emotion engine. The chatbot then advises, "It seems you've been getting less sleep recently. This may be contributing to your poor health."

[1157] 7. Follow-up with a medical professional

[1158] If advanced medical support is deemed necessary, the generative AI model notifies the system and transfers the user's consultation to a medical professional, who can provide additional diagnosis and advice via video or text chat.

[1159] Examples of concrete examples and prompts

[1160] 1. Example:

[1161] User registration: A user registers in the app by entering "test_user", "user@example.com", and "password123".

[1162] Login: Log in with the same user information.

[1163] Security Session: Tap the "Start Security Session" button to start the session.

[1164] Emotional data transmission: Data expressing "anxiety" is transmitted via the app.

[1165] Security advice: "There has been an increase in the number of accesses to your device recently. Please set up two-step authentication."

[1166] 2. Example prompt:

[1167] User: I've been feeling a bit uneasy with my devices lately. Any advice?

[1168] Chatbot: That's unfortunate. We've noticed some suspicious activity in your recent logs. We recommend you enable two-factor authentication and change your password. If you'd like more information, please type "Tell me more."

[1169] This will specifically specify the mode for carrying out the present invention, and serve as a reference for others to accurately understand and practice the invention.

[1170] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1171] Step 1: User Registration

[1172] Input: A user uses a smartphone app to enter their name, email address, and password.

[1173] Processing: The smartphone app sends the entered information to the REST API server, which saves the information in a database and completes the user registration.

[1174] Output: The server sends a successful registration notification back to the app, and the app displays "Registration successful" to the user.

[1175] Step 2: Log in

[1176] Input: The user enters the email address and password they registered on the smartphone app.

[1177] Processing: The smartphone app sends the entered authentication information to the REST API server. The server checks the authentication information by referencing the database, and if authentication is successful, issues a session token.

[1178] Output: The server sends a successful authentication response and a session token back to the app, and the app notifies the user that the login was successful.

[1179] Step 3: Initiating a medical consultation

[1180] Input: The user presses the "Start medical consultation" button in the smartphone app.

[1181] Processing: The smartphone app sends a request to start a consultation to the REST API server. The server starts the generative AI model and prepares to display the chatbot interface on the user's device.

[1182] Output: The server displays the chatbot interface on the user device and asks the user, "Hello, how can I help you?"

[1183] Step 4: Dialogue

[1184] Input: The user inputs their symptoms and concerns into the chatbot (e.g., "I've been having a bad cough lately and can't sleep at night").

[1185] Processing: The chatbot uses a generative AI model to analyze the user's input data and generate an appropriate response (e.g., "How long have you had a cough?").

[1186] Output: The server sends the generated response to the user's device, where the chatbot displays it to the user.

[1187] Step 5: Emotion Recognition with the Emotion Engine

[1188] Input: Text data entered by the user (e.g., "I've been coughing so much at night lately I can't sleep").

[1189] Processing: The server sends the text data to the emotion engine, which analyzes it and recognizes the user's emotional state (e.g., anxiety) as a result of the analysis.

[1190] Output: The emotion engine passes the recognized emotional state to the generative AI model, which tailors the response based on the emotion. The tailored response is displayed on the user's device.

[1191] Step 6: Health management data integration

[1192] Input: Health data collected from health management apps (e.g., sleep duration, steps, heart rate, etc.).

[1193] Processing: The server connects with the health management app to acquire health data, which is then analyzed using a generative AI model and emotion engine to evaluate the user's health status.

[1194] Output: Health advice generated based on the analysis results (e.g., "It appears you've been sleeping less recently. This may be causing your health condition to worsen") is displayed on the user's device.

[1195] Step 7: Follow up with a medical professional

[1196] Input: User consultation details and health data for any issues deemed urgent or requiring specialized care.

[1197] Processing: Based on the judgment of the generated AI model, the server forwards the user's consultation to a medical professional, who provides further diagnosis and advice to the user via video chat or text chat.

[1198] Output: The user is notified that the follow-up appointment has been scheduled and completed. The user will then have a video or text conversation with a medical professional at the designated time.

[1199] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1200] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1201] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1202] [Third embodiment]

[1203] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1204] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1205] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1206] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1207] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1209] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1210] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1211] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1212] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1213] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1214] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1215] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[1216] Overall system configuration

[1217] 1. User Registration and Login

[1218] User: A user accesses the system through an app or web interface and enters the required registration information, including name, email address, and password.

[1219] Server: The server saves the entered user information in a database and notifies the user that registration is complete. After that, the user enters their authentication information on the login screen and is granted access to the system.

[1220] 2. Initiating a medical consultation

[1221] User: When the user presses the "Start medical consultation" button within the app, the server launches the generative AI model and displays the chatbot interface on the user's device.

[1222] Chatbot: The chatbot uses generative AI models to engage in natural conversations with users and identify their concerns and symptoms.

[1223] 3. Conversational exchanges

[1224] User: The user enters their symptoms and concerns into the chatbot. For example, they might say, "I've been coughing so much lately I can't sleep at night."

[1225] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[1226] 4. Health Management Data Linkage

[1227] User: Health data is collected when the user connects the health management app to the system on the settings screen.

[1228] Server: The server passes data obtained from the health management app to the generative AI model and analyzes the user's health condition.

[1229] Chatbot: Provides users with advice based on their health data, such as, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[1230] 5. Follow-up with a medical professional

[1231] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[1232] Server: Transfers the user's consultation to a medical professional and performs follow-up, if necessary.

[1233] Medical Expert: A medical expert provides additional diagnosis and advice to users via video or text chat.

[1234] Specific examples

[1235] 1. User Registration

[1236] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[1237] Server: Notify "Registration complete. Please log in."

[1238] 2. Medical consultation begins

[1239] User: Tap "Start Medical Consultation" on the app's home screen.

[1240] Server: The chatbot displays "Hello, how can we help you?"

[1241] 3. Conversational exchanges

[1242] User: Type "I've been coughing so much at night lately I can't sleep."

[1243] Chatbot: "That's terrible. How long have you had a cough?"

[1244] User: "About a week."

[1245] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[1246] 4. Health Management Data Linkage

[1247] User: Set up the connection to the health management app.

[1248] Server: Collects health management data and sends this information to the generative AI model.

[1249] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[1250] 5. Follow-up with a medical professional

[1251] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[1252] Server: Send follow-up requests to medical professionals and schedule video chats.

[1253] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[1254] The present invention enables even patients who have difficulty moving around to easily receive medical consultations from home, further supporting users' health management.

[1255] The processing flow will be explained below.

[1256] Step 1:

[1257] User: Launches the application and clicks the "Sign Up" button.

[1258] On your device: Display a form for name, email address, password, etc.

[1259] User: Enter the required information and press the "Register" button.

[1260] Terminal: Sends the entered information to the server.

[1261] Step 2:

[1262] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[1263] Device: Display "Registration complete. Please log in."

[1264] User: Enter your email address and password on the login screen and click the "Login" button.

[1265] Device: Sends authentication information to the server.

[1266] Step 3:

[1267] Server: Retrieves user information from the database and verifies authentication information.

[1268] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[1269] On the device: Display a login success message and go to the home screen.

[1270] Step 4:

[1271] User: Press the "Start medical consultation" button on the home screen.

[1272] Device: Sends a request to the server to launch the generative AI model.

[1273] Server: Initializes the generative AI model and starts the chatbot session.

[1274] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[1275] Step 5:

[1276] User: Writes down symptoms and concerns in the chat interface and sends it.

[1277] Terminal: Sends the user's messages to the server.

[1278] Server: Passes messages to the generative AI model for analysis.

[1279] Chatbot: Generates appropriate response messages and returns the results to the server.

[1280] Server: Sends a response message to the terminal.

[1281] Terminal: Display the response message.

[1282] Step 6:

[1283] User: Enters additional information in response to the chatbot's questions and submits.

[1284] Terminal: Sends the user's messages to the server.

[1285] Server: Passes messages to the generative AI model.

[1286] Chatbot: Based on the analysis results, further questions and advice are generated and returned to the server.

[1287] Server: Sends a response message to the terminal.

[1288] Terminal: Display the response message.

[1289] Step 7:

[1290] User: Set up integration with the health management app on the app settings screen.

[1291] Device: Sends a request to connect with the health management app to the server.

[1292] Server: Accesses the health management app and retrieves the necessary data.

[1293] Server: Passes the acquired health data to the generative AI model.

[1294] Generative AI model: Analyzes data and assesses the user's health status.

[1295] Server: Sends the health assessment results to the device.

[1296] Device: Displays advice to the user based on health data.

[1297] Step 8:

[1298] Chatbot: A generative AI model analyzes the user's inquiry and determines whether it is urgent.

[1299] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[1300] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[1301] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[1302] Example 1

[1303] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1304] In remote medical consultations, there is a need for a system that allows users to easily receive medical consultations from home, efficiently manage their health status, and quickly connect with medical professionals in emergencies. Conventional systems have complicated user registration and login procedures, and the quality of responses from generative AI models is insufficient. Furthermore, the collection and analysis of health information through integration with health management software was not performed effectively, resulting in a lack of accurate advice for users. Furthermore, there were issues with smooth follow-up with medical professionals in emergencies.

[1305] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1306] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health information in cooperation with health management software; a means for providing the user with advice based on the acquired health information; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; a means for the user to enter authentication information such as their name, email address, and password to register and log in to the system; a means for transmitting the user's input symptoms and concerns to the generative AI model and generating an appropriate response; and a means for launching the generative AI model on the user's terminal and providing a chatbot interface that engages in natural conversation with the user. This allows users to conveniently receive medical consultations from home and receive high-quality responses through the generative AI model. Furthermore, through effective collection and analysis of health information in cooperation with health management software, the server can provide accurate advice to the user and quickly collaborate with medical professionals for follow-up in emergencies.

[1307] "User" refers to an individual who uses this system to conduct a remote medical consultation.

[1308] "Telemedical consultation" refers to medical consultation conducted remotely via the Internet.

[1309] A "generative AI model" refers to artificial intelligence that analyzes user input data and generates appropriate responses.

[1310] "User terminal" refers to the device (smartphone, PC, etc.) used by a user to access this system.

[1311] "Health management software" refers to applications and platforms for collecting, managing, and analyzing users' health data.

[1312] "Health information" refers to data related to the user's health condition (e.g., sleep time, amount of exercise, weight, etc.).

[1313] "Authentication Information" means the information, such as name, email address, and password, that a User uses to register and log in to the System.

[1314] "Chatbot interface" refers to the screen or application through which a user and a generative AI model interact.

[1315] "Medical Expert" refers to a doctor or other medical professional who provides follow-up care to the user based on the user's consultation.

[1316] "Follow-up" refers to providing additional diagnosis or advice to the user.

[1317] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[1318] User Registration and Login

[1319] Users access the system through an app or web interface and register by entering their name, email address, and password, which then sends the authentication information to the server.

[1320] The server stores the entered user authentication information in a database using a database management system such as MySQL, and notifies the user when registration is complete.

[1321] The user logs in with the registered information and accesses the system. The server checks the user's authentication information against the database, and displays the home screen only if authentication is successful.

[1322] Initiating a medical consultation

[1323] When a user selects "Start medical consultation" on the home screen, the server launches a generative AI model (e.g., OpenAI's GPT-3).

[1324] The server initializes the generative AI model, starts the chatbot session, and displays the chatbot interface on the user's device, ready to begin a healthcare-related conversation.

[1325] Conversational exchange

[1326] Users can input their symptoms and concerns into the chatbot. For example, they can enter something like, "I've been coughing so much at night recently that I can't sleep."

[1327] The server sends this input to a generative AI model that generates a response, such as a question like, "How long has your cough lasted?"

[1328] The generative AI model digs deeper into the user's concerns through ongoing conversation and asks more detailed questions.

[1329] Health management data linkage

[1330] When a user connects the system to health management software (e.g., Google Fit or Apple Health) on the settings screen, the server obtains data from the health software.

[1331] The server sends the acquired health information to a generative AI model to analyze the user's health condition.

[1332] The chatbot will then provide appropriate advice to the user based on the analyzed data. For example, it could say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[1333] Follow-up with a medical professional

[1334] If the chatbot determines that the user's symptoms are urgent, it notifies the server of this information.

[1335] The server forwards the user's consultation to a medical professional for follow-up as needed.

[1336] Medical professionals can provide additional diagnosis and advice to users via video or text chat, allowing users to receive professional advice quickly.

[1337] Providing concrete examples

[1338] In a specific scenario in which a user is seeking medical advice, the following prompt sentences may be used:

[1339] Example prompt: "I've been coughing so much at night lately that I can't sleep. What should I do?"

[1340] This invention allows patients with mobility issues to easily receive medical consultations from home, receiving high-quality responses and accurate advice. It also enables prompt collaboration with medical professionals in emergencies, enabling appropriate follow-up. This system, utilizing generative AI models, further supports users' health management.

[1341] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1342] Step 1: User Registration

[1343] Input: User's name, email address, and password

[1344] Specific operation:

[1345] Through the app or web interface, users enter their name, email address, and password and click the "Register" button.

[1346] The server stores the information entered by the user in a database using MySQL.

[1347] The server will notify you, "Registration complete. Please log in."

[1348] Output: Registration completion notification

[1349] Step 2: Login authentication

[1350] Input: User's email address, password

[1351] Specific operation:

[1352] The user enters the registered email address and password and clicks the "Login" button.

[1353] The server checks the user's credentials against a database.

[1354] If the server successfully authenticates the user, it displays the home screen, but if it fails, it notifies the user that "Authentication failed."

[1355] Output: Home screen display or authentication failure notification

[1356] Step 3: Initiating a medical consultation

[1357] Input: Click on the "Start medical consultation" button

[1358] Specific operation:

[1359] The user selects "Start medical consultation" on the home screen.

[1360] The server initializes a generative AI model (e.g., OpenAI's GPT-3).

[1361] The server sends the chatbot interface to the user terminal and starts the session.

[1362] Output: Chatbot interface displayed

[1363] Step 4: Accepting User Input

[1364] Input: User's symptoms and concerns

[1365] Specific operation:

[1366] The user enters their symptoms and concerns in the text box and clicks the send button.

[1367] Output: Text data of input symptoms and anxieties

[1368] Step 5: Generate a response using a generative AI model

[1369] Input: Text data of user symptoms and anxieties

[1370] Specific operation:

[1371] The server sends the user's input text to the generative AI model.

[1372] The generative AI model analyzes the input and generates an appropriate response.

[1373] The server sends the generated response to the user terminal.

[1374] Output: The generated response text

[1375] Step 6: View the response

[1376] Input: Generated response text

[1377] Specific operation:

[1378] The terminal displays the response received from the server on the chatbot interface.

[1379] Output: The response displayed on the chatbot interface

[1380] Step 7: Link with health management app

[1381] Input: Select health management app and click link button

[1382] Specific operation:

[1383] The user configures the settings screen to link with a health management app (e.g., Google Fit or Apple Health).

[1384] The server calls the API of the selected health management app and sets up the connection.

[1385] Output: Notification that the connection with the health management app has been completed

[1386] Step 8: Acquire and analyze health data

[1387] Input: Data obtained from health management app

[1388] Specific operation:

[1389] The server uses the health management app's API to obtain the user's health data.

[1390] The server sends the acquired data to the generative AI model.

[1391] The generative AI model analyzes health data and generates advice about the user's health status.

[1392] The server transmits the generated advice to the user terminal.

[1393] Output: Health advice

[1394] Step 9: Providing advice based on health data

[1395] Input: Generated advice

[1396] Specific operation:

[1397] The terminal displays the advice received from the server on the chatbot interface.

[1398] Output: Display of health advice

[1399] Step 10: Emergency response decision and notification

[1400] Input: User symptoms and health data

[1401] Specific operation:

[1402] The chatbot analyzes the user's symptoms and health data to determine whether emergency response is required.

[1403] If the chatbot determines that an emergency response is required, it notifies the server.

[1404] Output: Notification that urgent action is required

[1405] Step 11: Consult a medical professional

[1406] Input: Notification that emergency action is required

[1407] Specific operation:

[1408] The server notifies the medical professional of the user's consultation and the urgency of the consultation, and performs follow-up.

[1409] Medical professionals will provide additional diagnosis and advice to users via video or text chat.

[1410] Output: Notification that a follow-up with a medical professional will be performed

[1411] The system allows users to easily receive medical consultations from home, receive high-quality responses, and is excellent for quickly connecting with medical professionals in emergencies.

[1412] (Application example 1)

[1413] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] While remote medical consultation systems allow users to easily receive medical consultations from home, providing prompt and appropriate support during health consultations at physical stores has been difficult. For this reason, new methods are needed for store staff to provide customers with appropriate health advice in real time and efficiently recommend products.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1416] In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; and a means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses. This enables quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[1417] "User" refers to an individual who uses the system to receive remote medical consultations or in-store health consultations.

[1418] "Telemedical consultation" means a service that allows a user to consult with a medical professional online from the comfort of their own home or any other location.

[1419] A "generative AI model" refers to artificial intelligence technology that analyzes user input and generates appropriate responses and advice through natural conversation.

[1420] "User Terminal" means a device (e.g., smartphone, tablet, computer) through which a User accesses the System and conducts a remote medical consultation.

[1421] "Health management app" refers to an application that collects and manages a user's health data.

[1422] "Server" means the computer system that manages and operates the entire system and processes information from the generative AI model and database.

[1423] "Smart glasses" refer to wearable devices that provide information along the user's line of sight and assist staff in providing real-time health consultations in brick-and-mortar stores.

[1424] "Medical Professional" means a medical professional who is qualified to provide professional diagnosis and advice regarding the user's health inquiry.

[1425] The present invention provides a health consultation support system for use in a physical store, which allows users to receive health consultations in real time through store staff wearing smart glasses. Detailed embodiments of the present invention will be described below.

[1426] The server includes means for a user to initiate a remote medical consultation, means for analyzing the user's input using a generative AI model and generating a response, means for displaying the response generated by the generative AI model on the user's terminal, means for acquiring and analyzing the user's health data in cooperation with a health management app, means for providing advice to the user based on the acquired health data, means for transferring the user's consultation content to a medical professional and performing follow-up as necessary, and means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses.

[1427] 1. Overall system configuration

[1428] The system consists of the following main components:

[1429] User devices: smartphones, tablets, computers, etc.

[1430] Server: Cloud-based computing services.

[1431] Generative AI models: such as OpenAI's GPT-4.

[1432] Smart glasses: Google Glass, etc.

[1433] Health management app: An application that collects and manages health data.

[1434] 2. System Operation

[1435] 2.1 User Registration and Login

[1436] The server provides a means for users to access the system through an app or web interface and enter the necessary registration information (name, email address, password, etc.) The server stores this information in a database and manages the authentication information.

[1437] 2.2 Starting a health consultation

[1438] When a user presses the "Start medical consultation" button, the server activates the generative AI model and displays a chatbot interface on the user's device. This chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation content and symptoms.

[1439] 2.3 Conversational exchanges

[1440] When a user inputs a question or symptom into the system, the generative AI model analyzes it and provides an appropriate answer, which is displayed on the user's device and the smart glasses' display.

[1441] 2.4 Health Management Data Linkage

[1442] When a user connects a health management app to the system, the server acquires this data and sends it to the generative AI model, which then analyzes the user's health condition and provides appropriate advice.

[1443] 2.5 Real-time health consultations at physical stores

[1444] When a store employee wearing smart glasses receives a health consultation from a customer, the server sends the information to the generative AI model, which generates a response. This response is displayed in real time on the employee's smart glasses, and the employee can then provide appropriate advice to the customer.

[1445] Specific examples

[1446] For example, if a customer asks how to choose vitamins, the following exchange might occur:

[1447] Staff: "What purpose are you looking for vitamins for?"

[1448] Customer: "I'm feeling tired and I'd like to have more energy."

[1449] Smart glasses display: "To increase your energy, we recommend supplements containing B and C vitamins. Iron may also be beneficial. Do you have any other questions? Learn more about these products."

[1450] Prompt Sentence Examples

[1451] "A client is asking about choosing vitamins. Can you recommend some vitamins or supplements to help them increase their energy?"

[1452] This will enable quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[1453] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1454] Step 1:

[1455] A user accesses the system through an app or web interface and registers. The user enters the required information, such as name, email address, and password, and sends it to the server. The server stores this information in a database and notifies the user that registration is complete.

[1456] Input: Name, Email Address, Password

[1457] Data processing: Saving user information to a database

[1458] Output: Registration completion notification

[1459] Step 2:

[1460] A user logs in to the system. The user enters the registered email address and password and sends the authentication information to the server. The server authenticates the user and allows the user to access the system.

[1461] Input: Email address, Password

[1462] Data Calculation: Authentication Check

[1463] Output: System access permissions

[1464] Step 3:

[1465] The user presses the "Start medical consultation" button. The server launches the generative AI model and displays the chatbot interface on the user's device. The user's device then begins a dialogue with the user.

[1466] Input: Medical consultation start request

[1467] Data processing: Displaying the chatbot interface

[1468] Output: Medical consultation initiated

[1469] Step 4:

[1470] The user inputs symptoms or questions into the chatbot. The user's device sends the input text to the generative AI model. The server receives it, analyzes it with the generative AI model, and generates an appropriate response.

[1471] Input: User question or symptom

[1472] Data Computation: Question Analysis and Answer Generation

[1473] Output: The generated response

[1474] Step 5:

[1475] The server sends the generated response to the user terminal for display, which displays it to the user for further interaction.

[1476] Input: The generated response

[1477] Data processing: Sending and displaying responses

[1478] Output: Display response to user

[1479] Step 6:

[1480] The user configures the health management app to work with the server, which then acquires the user's health data from the app and sends it to the generative AI model for analysis.

[1481] Input: Health management app data

[1482] Data Computing: Health Data Analysis

[1483] Output: Advice based on health status

[1484] Step 7:

[1485] A user consults a staff member wearing smart glasses at a physical store. The staff member's smart glasses send the user's consultation to the generative AI model, which analyzes it on the server. The server then sends the answer to the staff member's smart glasses and displays it.

[1486] Input: Customer Question

[1487] Data Computation: Question Analysis and Answer Generation

[1488] Output: Response display on smart glasses

[1489] Step 8:

[1490] If necessary, the server will follow up by transferring the user's consultation to a medical professional who will provide further diagnosis and advice via video or text chat.

[1491] Input: User's inquiry

[1492] Data processing: Transfer of consultation details

[1493] Output: Follow-up with medical professionals

[1494] The above processing steps make it possible to provide prompt and appropriate health consultations even in physical stores.

[1495] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1496] The present invention combines an emotion engine with a remote medical consultation system, which can recognize the user's emotional state and provide appropriate responses and medical support through a generative AI model. Specific embodiments of the present invention are described below.

[1497] Overall system configuration

[1498] 1. User Registration and Login

[1499] User: Accesses the system through an application or web interface and creates a new account by entering the required registration information (name, email address, password, etc.).

[1500] Server: Saves the entered user information in the database and notifies the user that registration is complete. Once the user enters their authentication information on the login screen, they are granted access to the system.

[1501] 2. Initiating a medical consultation

[1502] User: When the user presses the "Start medical consultation" button within the application, the server launches the generative AI model and displays the chatbot interface on the user's device.

[1503] Chatbot: The chatbot uses generative AI models to have natural conversations with users and confirm their concerns and symptoms.

[1504] 3. Conversational exchanges

[1505] User: Enters symptoms and concerns into the chatbot. For example, "I've been coughing so much lately I can't sleep at night."

[1506] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[1507] 4. Emotion Recognition by Emotion Engine

[1508] Server: Sends user input to the emotion engine and analyzes the emotional state.

[1509] Emotion engine: Recognizes user emotions (e.g., stress, anxiety, anger) and passes the results to a generative AI model.

[1510] Generative AI model: Tailors responses based on emotional state to provide appropriate support to users.

[1511] 5. Health Management Data Linkage

[1512] User: Health data is obtained by linking the health management app with the system from the settings screen.

[1513] Server: Sends data obtained from the health management app to the emotion engine and generative AI model to analyze the user's health condition.

[1514] Chatbot: Provides users with advice based on their health data and emotional state. For example, it might say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[1515] 6. Follow-up with a medical professional

[1516] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[1517] Server: Transfers the user's consultation to a medical professional, if necessary, and arranges for follow-up.

[1518] Medical Expert: Provides additional diagnosis and advice to users via video or text chat.

[1519] Specific examples

[1520] 1. User Registration

[1521] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[1522] Server: Notify "Registration complete. Please log in."

[1523] 2. Medical consultation begins

[1524] User: Tap "Start Medical Consultation" on the app's home screen.

[1525] Server: The chatbot displays "Hello, how can we help you?"

[1526] 3. Conversational exchanges

[1527] User: Type "I've been coughing so much at night lately I can't sleep."

[1528] Chatbot: "That's terrible. How long have you had a cough?"

[1529] User: "About a week."

[1530] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[1531] 4. Emotion Recognition by Emotion Engine

[1532] Server: Sends the user's input, "About a week" and "Are you experiencing any other symptoms such as fever or fatigue?" to the emotion engine.

[1533] Emotion engine: Recognizes user anxieties and passes that information to a generative AI model.

[1534] Generative AI model: Responds, "Okay, don't worry, this can get better with treatment."

[1535] 5. Health Management Data Linkage

[1536] User: Set up the connection to the health management app.

[1537] Server: Collects health management data and sends this information to the generative AI model and emotion engine.

[1538] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[1539] 6. Follow-up with a medical professional

[1540] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[1541] Server: Send follow-up requests to medical professionals and schedule video chats.

[1542] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[1543] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[1544] The processing flow will be explained below.

[1545] The present invention is a remote medical consultation system with an emotion engine built in. The specific processing flow will be explained below step by step.

[1546] Overall system configuration

[1547] Step 1:

[1548] User: Launches the application and clicks the "Sign Up" button.

[1549] On your device: Display a form for name, email address, password, etc.

[1550] User: Enter the required information and press the "Register" button.

[1551] Terminal: Sends the entered information to the server.

[1552] Step 2:

[1553] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[1554] On the device: Display "Registration successful. Please log in" to the user.

[1555] User: Enter your email address and password on the login screen and click the "Login" button.

[1556] Device: Sends authentication information to the server.

[1557] Step 3:

[1558] Server: Retrieves user information from the database and verifies authentication information.

[1559] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[1560] On the device: Display a login success message and go to the home screen.

[1561] Step 4:

[1562] User: Press the "Start medical consultation" button on the home screen.

[1563] Device: Sends a request to the server to launch the generative AI model.

[1564] Server: Initializes the generative AI model and starts the chatbot session.

[1565] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[1566] Step 5:

[1567] User: Writes down symptoms and concerns in the chat interface and sends it.

[1568] Terminal: Sends the user's messages to the server.

[1569] Server: Passes messages to the generative AI model for analysis.

[1570] Chatbot: Generates appropriate response messages and returns the results to the server.

[1571] Server: Sends a response message to the terminal.

[1572] Terminal: Display the response message.

[1573] Step 6:

[1574] User: Enters additional information in response to the chatbot's questions and submits.

[1575] Terminal: Sends the user's messages to the server.

[1576] Server: Passes messages to the generative AI model and emotion engine.

[1577] Emotion engine: Analyzes user input and recognizes emotional states (e.g., stress, anxiety, relief, etc.).

[1578] Generative AI model: Adjusts responses based on the analysis results of the emotion engine.

[1579] Chatbot: Generates appropriate responses corresponding to emotions and returns them to the server.

[1580] Server: Sends a response message to the terminal.

[1581] Terminal: Display a response message such as "Don't worry, this may improve with treatment."

[1582] Step 7:

[1583] User: Set up integration with the health management app on the app settings screen.

[1584] Device: Sends a request to connect with the health management app to the server.

[1585] Server: Accesses the health management app and retrieves the necessary data.

[1586] Server: Passes acquired health data to the generative AI model and emotion engine.

[1587] Generative AI model: Analyzes health data and assesses the user's health status.

[1588] Emotion Engine: Integrates health data with your current emotional state to provide comprehensive analysis.

[1589] Server: Sends health assessment results and advice to the device.

[1590] Device: Display advice such as, "It appears you've been getting less sleep recently. This may be contributing to your poor health."

[1591] Step 8:

[1592] Chatbot: The generative AI model determines whether the user's inquiry is urgent or not based on the content of the inquiry and the results of emotional analysis.

[1593] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[1594] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[1595] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[1596] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[1597] Example 2

[1598] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1599] In remote medical consultations, it can be difficult for users to accurately communicate their symptoms, and responses that ignore the user's emotional state can result in insufficient medical support.Furthermore, there is a lack of a system for appropriately utilizing users' health management data, making it difficult to provide appropriate advice and follow-up that is tailored to each individual user.

[1600] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1601] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; and a means for recognizing the user's emotional state and providing appropriate responses and medical support through the generative AI model. This enables accurate understanding of the user's symptoms and personalized medical support that takes their emotional state into account. Furthermore, by appropriately utilizing health management data, it is possible to provide advice tailored to each individual user and follow-up according to the level of urgency.

[1602] "Telemedical consultation" is a system that allows users to receive medical consultation through a communication network without physically visiting a medical institution.

[1603] A "generative AI model" is an algorithm or system that uses artificial intelligence to analyze and understand user input and generate an appropriate response.

[1604] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses to access the system.

[1605] A "health management app" is an application that collects and manages a user's health-related data (e.g., sleep data, exercise data, vital signs).

[1606] "Emotional state" refers to the user's emotional state (e.g., stress, anxiety, anger) as analyzed by the emotion engine.

[1607] The "emotion engine" is a component that analyzes emotions from user input and provides that information to the generative AI model.

[1608] "Medical professionals" are professionals with specialized medical knowledge, such as doctors, nurses, and pharmacists.

[1609] "Follow-up" refers to subsequent medical intervention to provide additional diagnosis or advice based on the user's consultation.

[1610] "Analysis" is the process of understanding input data and extracting or generating meaningful information based on it.

[1611] MODE FOR CARRYING OUT THE INVENTION

[1612] The present invention relates to a system for remote medical consultations. This system combines a generative AI model and an emotion engine to provide appropriate medical support based on the user's input and emotional state. Specific embodiments for implementing the present invention are described below.

[1613] System Configuration

[1614] This system mainly consists of the following components:

[1615] User devices (e.g. smartphones, tablets, PCs)

[1616] server

[1617] Generative AI Models

[1618] Emotion Engine

[1619] Health management app

[1620] Linking to medical professionals

[1621] The role of each component

[1622] 1. User Device

[1623] This is an electronic device that users use to conduct remote medical consultations. Through an application or web interface, users register and log in to an account and begin a medical consultation.

[1624] 2. Server

[1625] The server plays a central role in the entire system: it manages user authentication information, runs the generative AI model and emotion engine, sends responses to the user's device, and also coordinates data with the health management app and forwards follow-up information to medical professionals.

[1626] 3. Generative AI Models

[1627] It includes algorithms for analyzing user input and generating appropriate responses. The generative AI model generates natural conversations based on the user's symptoms and questions, providing accurate advice to the user.

[1628] 4. Emotion Engine

[1629] It analyzes user input and recognizes emotional states (e.g., stress, anxiety, anger). The emotion engine passes the results to a generative AI model, which then uses it to tailor responses.

[1630] 5. Health Management App

[1631] It is an application that collects and manages users' health data (e.g., sleep data, exercise data, vital signs). This data is sent to a server and analyzed by a generative AI model and emotion engine.

[1632] 6. Linking to medical professionals

[1633] If an emergency or specialized treatment is deemed necessary, the server will send a request to a medical professional who will provide the user with additional diagnosis and advice via video or text chat.

[1634] Example of operation

[1635] 1. User Registration and Login

[1636] A user launches the application, enters their name, email address, and password in the registration form, and clicks "Register."

[1637] The server saves the entered information in a database and notifies the user that "Registration is complete. Please log in."

[1638] 2. Initiating a medical consultation

[1639] The user taps "Start medical consultation" on the app's home screen.

[1640] The server launches an instance of the generative AI model and chatbot, which then displays "Hello, how can I help you?"

[1641] 3. Conversational exchanges

[1642] The user types, "Recently, I've been coughing so much at night that I can't sleep."

[1643] The chatbot responds, "That's terrible. How long have you had this cough?"

[1644] The user responds, "About a week."

[1645] The chatbot asks additional questions, such as, "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[1646] 4. Emotion Recognition by Emotion Engine

[1647] The server sends the user's input to the emotion engine.

[1648] The emotion engine recognizes the user's anxiety and passes that information to the generative AI model.

[1649] The generative AI model responds, "Okay, don't worry, this can get better with treatment."

[1650] 5. Health Management Data Linkage

[1651] The user configures the settings to link with the health management app.

[1652] A server collects health management data and sends this information to a generative AI model and emotion engine.

[1653] The chatbot advises, "It seems you've been sleeping less recently. This may be causing your health to deteriorate."

[1654] 6. Follow-up with a medical professional

[1655] The chatbot will notify the user, "If this condition persists, you may need to see a specialist."

[1656] The server sends follow-up requests to medical professionals and schedules video chats.

[1657] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[1658] Prompt Sentence Examples

[1659] "Provide advice to users based on their recent health data."

[1660] "Analyze the user's emotional state and generate a conversation to provide appropriate support."

[1661] "Please outline your follow-up procedures when urgent symptoms are reported."

[1662] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1663] A detailed explanation of the program's processing steps

[1664] Step 1: User registration and login

[1665] A user opens the application or web interface, enters their name, email address, and password in the new registration form, and clicks the "Register" button.

[1666] Input: Name, Email Address, Password

[1667] The server receives the entered information and stores it in a database.

[1668] Output: Notification of successful registration

[1669] The server notifies the user, "Registration complete. Please log in."

[1670] Step 2: Log in

[1671] The user enters their email address and password on the login screen and clicks the "Login" button.

[1672] Input: Email address, password

[1673] The server checks the entered credentials against its database.

[1674] Output: Notification of successful or failed login

[1675] If the authentication is successful, the server notifies the user that "Login was successful" and displays the dashboard screen.

[1676] Step 3: Initiating a medical consultation

[1677] The user taps "Start medical consultation" on the app's home screen.

[1678] Input: Request to start a medical consultation

[1679] The server launches an instance of the generated AI model and chatbot.

[1680] Output: Chatbot interface displayed

[1681] The server displays the chatbot interface on the user's device and asks the user, "Hello. What would you like to discuss with us?"

[1682] Step 4: Dialogue

[1683] The user inputs their symptoms into the chatbot (e.g., "I've been coughing so much at night lately that I can't sleep").

[1684] Input: User's symptoms

[1685] The chatbot uses a generative AI model to analyze user input.

[1686] Output: Generate an appropriate question (e.g., "How long has your cough lasted?")

[1687] The chatbot returns the generated question to the user.

[1688] Step 5: Emotion Recognition with the Emotion Engine

[1689] The server sends the user's input to the emotion engine.

[1690] Input: What the user types

[1691] The emotion engine analyzes the input and recognizes the user's emotional state (e.g., anxiety, stress, anger).

[1692] Output: Emotional state analysis results

[1693] The emotion engine passes the analysis results to the generative AI model.

[1694] The generative AI model adjusts its response based on the emotional state, saying, "Okay, don't worry, this can get better with treatment."

[1695] Step 6: Health management data integration

[1696] The user sets up linkage with the health management app on the settings screen.

[1697] Input: Health management app link information

[1698] The server acquires health data from the health management app.

[1699] Output: Health data collection

[1700] The server sends the collected health data to the generative AI model and emotion engine.

[1701] The generative AI model analyzes health data, and the chatbot advises, "It appears you've been sleeping less recently. This may be contributing to your poor health."

[1702] Step 7: Follow up with a medical professional

[1703] The chatbot analyzes the user's symptoms and input, and determines whether the condition is urgent or requires specialized treatment.

[1704] Input: User symptoms and input

[1705] The server sends a follow-up request to the medical professional.

[1706] Output: Send follow-up request

[1707] The server schedules the video chat and notifies the user.

[1708] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[1709] (Application example 2)

[1710] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1711] While remote medical consultation systems provide support for users regarding their health status, they face challenges in providing personalized advice that takes into account the user's emotional state and in providing insufficient security measures. Furthermore, because the user's emotional state can affect the overall response quality of the system, there is a need for an analysis of the user's emotional state and the generation of appropriate responses based on that state.

[1712] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; hardware including an emotion engine that analyzes and recognizes the user's emotional state; and a means for generating a response to the user based on the recognized emotional state and providing security advice. This enables personalized medical support and security measures that take the user's emotional state into consideration.

[1713] A "means for a user to initiate a remote medical consultation" is an interface that allows a user to initiate a consultation with a medical professional using communication technology.

[1714] "Means of using a generative AI model to analyze user input and generate a response" refers to the process of using artificial intelligence to analyze information entered by a user and generate an appropriate response based on that information.

[1715] "Means for displaying the response generated by the generative AI model on the user's device" refers to a method for visualizing the response created by the generative AI model on the user's device.

[1716] The "means for acquiring and analyzing user health data in cooperation with a health management application" is a method for collecting and analyzing user health-related data in cooperation with a health management application.

[1717] The "means for providing advice to the user based on the acquired health data" is a process of analyzing the collected health data and providing appropriate advice to the user based on the data.

[1718] "Means for transferring the user's consultation to a medical professional as needed and for carrying out follow-up" refers to a method for transferring the user's consultation to a medical professional when the user's consultation requires specialized treatment and for carrying out continuous follow-up.

[1719] "Hardware including an emotion engine that analyzes and recognizes a user's emotional state" is a specific hardware device that has the function of analyzing and recognizing a user's emotion.

[1720] The "means for generating a response to a user based on a recognized emotional state and providing security advice" is a method for generating an appropriate response based on the user's emotions recognized by the emotion engine and further providing security advice to the user.

[1721] The present invention combines an emotion engine with a remote medical consultation system, recognizing the user's emotional state and providing appropriate responses and security advice through a generative AI model.

[1722] A system for implementing the present invention uses the following major hardware and software components:

[1723] Hardware

[1724] Smartphone: The device on which the user operates the application.

[1725] Server: A device that hosts data processing and generative AI models, emotion engines, and databases.

[1726] Emotion engine: Dedicated hardware for analyzing user input and recognizing emotional states.

[1727] software

[1728] Application interface: An app that allows users to initiate medical consultations and input emotional and health data.

[1729] Generative AI model: An algorithm that analyzes user input and generates an appropriate response.

[1730] REST API server: Software that handles communication between the client (smartphone app) and the server.

[1731] Health management app: Software that collects and provides user health data.

[1732] Database: A storage system for storing user information, health data, and emotional state data.

[1733] Processing steps

[1734] 1. User Registration

[1735] The user launches the smartphone app and enters the required registration information (name, email address, password). The application interface sends this information to the REST API server, which stores it in a database. Once registration is complete, the server returns a confirmation message.

[1736] 2. Log in

[1737] A user logs in from a smartphone app using their email address and password. The information is sent back to the REST API server, which verifies the authentication information in the database. If authentication is successful, the server issues a session token.

[1738] 3. Initiating a medical consultation

[1739] The user presses the "Start medical consultation" button within the application, and their smartphone sends this request to the server, which then activates the generative AI model and displays the chatbot interface on the user's device.

[1740] 4. Conversational exchanges

[1741] The chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation details and symptoms. For example, if a user inputs "I've been coughing so much at night recently that I can't sleep," the generative AI model will respond with "How long has this cough been going on for?"

[1742] 5. Emotion Recognition by Emotion Engine

[1743] The server sends the user's conversation content to the emotion engine, which recognizes the user's emotional state. The recognized emotional state is passed to the generative AI model, which adjusts the response. For example, if the user's anxiety is recognized, the generative AI model responds, "Don't worry, this can get better with treatment."

[1744] 6. Health Management Data Linkage

[1745] The user connects their health management app to the system, and health data (e.g., sleep time, steps taken, heart rate, etc.) is sent to the server. This data is analyzed using a generative AI model and an emotion engine. The chatbot then advises, "It seems you've been getting less sleep recently. This may be contributing to your poor health."

[1746] 7. Follow-up with a medical professional

[1747] If advanced medical support is deemed necessary, the generative AI model notifies the system and transfers the user's consultation to a medical professional, who can provide additional diagnosis and advice via video or text chat.

[1748] Examples of concrete examples and prompts

[1749] 1. Example:

[1750] User registration: A user registers in the app by entering "test_user", "user@example.com", and "password123".

[1751] Login: Log in with the same user information.

[1752] Security Session: Tap the "Start Security Session" button to start the session.

[1753] Emotional data transmission: Data expressing "anxiety" is transmitted via the app.

[1754] Security advice: "There has been an increase in the number of accesses to your device recently. Please set up two-step authentication."

[1755] 2. Example prompt:

[1756] User: I've been feeling a bit uneasy with my devices lately. Any advice?

[1757] Chatbot: That's unfortunate. We've noticed some suspicious activity in your recent logs. We recommend you enable two-factor authentication and change your password. If you'd like more information, please type "Tell me more."

[1758] This will specifically specify the mode for carrying out the present invention, and serve as a reference for others to accurately understand and practice the invention.

[1759] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1760] Step 1: User Registration

[1761] Input: A user uses a smartphone app to enter their name, email address, and password.

[1762] Processing: The smartphone app sends the entered information to the REST API server, which saves the information in a database and completes the user registration.

[1763] Output: The server sends a successful registration notification back to the app, and the app displays "Registration successful" to the user.

[1764] Step 2: Log in

[1765] Input: The user enters the email address and password they registered on the smartphone app.

[1766] Processing: The smartphone app sends the entered authentication information to the REST API server. The server checks the authentication information by referencing the database, and if authentication is successful, issues a session token.

[1767] Output: The server sends a successful authentication response and a session token back to the app, and the app notifies the user that the login was successful.

[1768] Step 3: Initiating a medical consultation

[1769] Input: The user presses the "Start medical consultation" button in the smartphone app.

[1770] Processing: The smartphone app sends a request to start a consultation to the REST API server. The server starts the generative AI model and prepares to display the chatbot interface on the user's device.

[1771] Output: The server displays the chatbot interface on the user device and asks the user, "Hello, how can I help you?"

[1772] Step 4: Dialogue

[1773] Input: The user inputs their symptoms and concerns into the chatbot (e.g., "I've been having a bad cough lately and can't sleep at night").

[1774] Processing: The chatbot uses a generative AI model to analyze the user's input data and generate an appropriate response (e.g., "How long have you had a cough?").

[1775] Output: The server sends the generated response to the user's device, where the chatbot displays it to the user.

[1776] Step 5: Emotion Recognition with the Emotion Engine

[1777] Input: Text data entered by the user (e.g., "I've been coughing so much at night lately I can't sleep").

[1778] Processing: The server sends the text data to the emotion engine, which analyzes it and recognizes the user's emotional state (e.g., anxiety) as a result of the analysis.

[1779] Output: The emotion engine passes the recognized emotional state to the generative AI model, which tailors the response based on the emotion. The tailored response is displayed on the user's device.

[1780] Step 6: Health management data integration

[1781] Input: Health data collected from health management apps (e.g., sleep duration, steps, heart rate, etc.).

[1782] Processing: The server connects with the health management app to acquire health data, which is then analyzed using a generative AI model and emotion engine to evaluate the user's health status.

[1783] Output: Health advice generated based on the analysis results (e.g., "It appears you've been sleeping less recently. This may be causing your health condition to worsen") is displayed on the user's device.

[1784] Step 7: Follow up with a medical professional

[1785] Input: User consultation details and health data for any issues deemed urgent or requiring specialized care.

[1786] Processing: Based on the judgment of the generated AI model, the server forwards the user's consultation to a medical professional, who provides further diagnosis and advice to the user via video chat or text chat.

[1787] Output: The user is notified that the follow-up appointment has been scheduled and completed. The user will then have a video or text conversation with a medical professional at the designated time.

[1788] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1789] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1790] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1791] [Fourth embodiment]

[1792] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1793] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1794] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1795] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1796] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1797] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1798] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1799] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1800] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1801] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1802] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1803] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1804] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1805] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[1806] Overall system configuration

[1807] 1. User Registration and Login

[1808] User: A user accesses the system through an app or web interface and enters the required registration information, including name, email address, and password.

[1809] Server: The server saves the entered user information in a database and notifies the user that registration is complete. After that, the user enters their authentication information on the login screen and is granted access to the system.

[1810] 2. Initiating a medical consultation

[1811] User: When the user presses the "Start medical consultation" button within the app, the server launches the generative AI model and displays the chatbot interface on the user's device.

[1812] Chatbot: The chatbot uses generative AI models to engage in natural conversations with users and identify their concerns and symptoms.

[1813] 3. Conversational exchanges

[1814] User: The user enters their symptoms and concerns into the chatbot. For example, they might say, "I've been coughing so much lately I can't sleep at night."

[1815] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[1816] 4. Health Management Data Linkage

[1817] User: Health data is collected when the user connects the health management app to the system on the settings screen.

[1818] Server: The server passes data obtained from the health management app to the generative AI model and analyzes the user's health condition.

[1819] Chatbot: Provides users with advice based on their health data, such as, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[1820] 5. Follow-up with a medical professional

[1821] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[1822] Server: Transfers the user's consultation to a medical professional and performs follow-up, if necessary.

[1823] Medical Expert: A medical expert provides additional diagnosis and advice to users via video or text chat.

[1824] Specific examples

[1825] 1. User Registration

[1826] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[1827] Server: Notify "Registration complete. Please log in."

[1828] 2. Medical consultation begins

[1829] User: Tap "Start Medical Consultation" on the app's home screen.

[1830] Server: The chatbot displays "Hello, how can we help you?"

[1831] 3. Conversational exchanges

[1832] User: Type "I've been coughing so much at night lately I can't sleep."

[1833] Chatbot: "That's terrible. How long have you had a cough?"

[1834] User: "About a week."

[1835] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[1836] 4. Health Management Data Linkage

[1837] User: Set up the connection to the health management app.

[1838] Server: Collects health management data and sends this information to the generative AI model.

[1839] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[1840] 5. Follow-up with a medical professional

[1841] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[1842] Server: Send follow-up requests to medical professionals and schedule video chats.

[1843] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[1844] The present invention enables even patients who have difficulty moving around to easily receive medical consultations from home, further supporting users' health management.

[1845] The processing flow will be explained below.

[1846] Step 1:

[1847] User: Launches the application and clicks the "Sign Up" button.

[1848] On your device: Display a form for name, email address, password, etc.

[1849] User: Enter the required information and press the "Register" button.

[1850] Terminal: Sends the entered information to the server.

[1851] Step 2:

[1852] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[1853] Device: Display "Registration complete. Please log in."

[1854] User: Enter your email address and password on the login screen and click the "Login" button.

[1855] Device: Sends authentication information to the server.

[1856] Step 3:

[1857] Server: Retrieves user information from the database and verifies authentication information.

[1858] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[1859] On the device: Display a login success message and go to the home screen.

[1860] Step 4:

[1861] User: Press the "Start medical consultation" button on the home screen.

[1862] Device: Sends a request to the server to launch the generative AI model.

[1863] Server: Initializes the generative AI model and starts the chatbot session.

[1864] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[1865] Step 5:

[1866] User: Writes down symptoms and concerns in the chat interface and sends it.

[1867] Terminal: Sends the user's messages to the server.

[1868] Server: Passes messages to the generative AI model for analysis.

[1869] Chatbot: Generates appropriate response messages and returns the results to the server.

[1870] Server: Sends a response message to the terminal.

[1871] Terminal: Display the response message.

[1872] Step 6:

[1873] User: Enters additional information in response to the chatbot's questions and submits.

[1874] Terminal: Sends the user's messages to the server.

[1875] Server: Passes messages to the generative AI model.

[1876] Chatbot: Based on the analysis results, further questions and advice are generated and returned to the server.

[1877] Server: Sends a response message to the terminal.

[1878] Terminal: Display the response message.

[1879] Step 7:

[1880] User: Set up integration with the health management app on the app settings screen.

[1881] Device: Sends a request to connect with the health management app to the server.

[1882] Server: Accesses the health management app and retrieves the necessary data.

[1883] Server: Passes the acquired health data to the generative AI model.

[1884] Generative AI model: Analyzes data and assesses the user's health status.

[1885] Server: Sends the health assessment results to the device.

[1886] Device: Displays advice to the user based on health data.

[1887] Step 8:

[1888] Chatbot: A generative AI model analyzes the user's inquiry and determines whether it is urgent.

[1889] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[1890] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[1891] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[1892] Example 1

[1893] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1894] In remote medical consultations, there is a need for a system that allows users to easily receive medical consultations from home, efficiently manage their health status, and quickly connect with medical professionals in emergencies. Conventional systems have complicated user registration and login procedures, and the quality of responses from generative AI models is insufficient. Furthermore, the collection and analysis of health information through integration with health management software was not performed effectively, resulting in a lack of accurate advice for users. Furthermore, there were issues with smooth follow-up with medical professionals in emergencies.

[1895] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1896] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health information in cooperation with health management software; a means for providing the user with advice based on the acquired health information; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; a means for the user to enter authentication information such as their name, email address, and password to register and log in to the system; a means for transmitting the user's input symptoms and concerns to the generative AI model and generating an appropriate response; and a means for launching the generative AI model on the user's terminal and providing a chatbot interface that engages in natural conversation with the user. This allows users to conveniently receive medical consultations from home and receive high-quality responses through the generative AI model. Furthermore, through effective collection and analysis of health information in cooperation with health management software, the server can provide accurate advice to the user and quickly collaborate with medical professionals for follow-up in emergencies.

[1897] "User" refers to an individual who uses this system to conduct a remote medical consultation.

[1898] "Telemedical consultation" refers to medical consultation conducted remotely via the Internet.

[1899] A "generative AI model" refers to artificial intelligence that analyzes user input data and generates appropriate responses.

[1900] "User terminal" refers to the device (smartphone, PC, etc.) used by a user to access this system.

[1901] "Health management software" refers to applications and platforms for collecting, managing, and analyzing users' health data.

[1902] "Health information" refers to data related to the user's health condition (e.g., sleep time, amount of exercise, weight, etc.).

[1903] "Authentication Information" means the information, such as name, email address, and password, that a User uses to register and log in to the System.

[1904] "Chatbot interface" refers to the screen or application through which a user and a generative AI model interact.

[1905] "Medical Expert" refers to a doctor or other medical professional who provides follow-up care to the user based on the user's consultation.

[1906] "Follow-up" refers to providing additional diagnosis or advice to the user.

[1907] The present invention provides a system for users to receive remote medical consultations. This system is built around a chatbot using a generative AI model and can be accessed through a user terminal. Specific embodiments of the present invention are described below.

[1908] User Registration and Login

[1909] Users access the system through an app or web interface and register by entering their name, email address, and password, which then sends the authentication information to the server.

[1910] The server stores the entered user authentication information in a database using a database management system such as MySQL, and notifies the user when registration is complete.

[1911] The user logs in with the registered information and accesses the system. The server checks the user's authentication information against the database, and displays the home screen only if authentication is successful.

[1912] Initiating a medical consultation

[1913] When a user selects "Start medical consultation" on the home screen, the server launches a generative AI model (e.g., OpenAI's GPT-3).

[1914] The server initializes the generative AI model, starts the chatbot session, and displays the chatbot interface on the user's device, ready to begin a healthcare-related conversation.

[1915] Conversational exchange

[1916] Users can input their symptoms and concerns into the chatbot. For example, they can enter something like, "I've been coughing so much at night recently that I can't sleep."

[1917] The server sends this input to a generative AI model that generates a response, such as a question like, "How long has your cough lasted?"

[1918] The generative AI model digs deeper into the user's concerns through ongoing conversation and asks more detailed questions.

[1919] Health management data linkage

[1920] When a user connects the system to health management software (e.g., Google Fit or Apple Health) on the settings screen, the server obtains data from the health software.

[1921] The server sends the acquired health information to a generative AI model to analyze the user's health condition.

[1922] The chatbot will then provide appropriate advice to the user based on the analyzed data. For example, it could say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[1923] Follow-up with a medical professional

[1924] If the chatbot determines that the user's symptoms are urgent, it notifies the server of this information.

[1925] The server forwards the user's consultation to a medical professional for follow-up as needed.

[1926] Medical professionals can provide additional diagnosis and advice to users via video or text chat, allowing users to receive professional advice quickly.

[1927] Providing concrete examples

[1928] In a specific scenario in which a user is seeking medical advice, the following prompt sentences may be used:

[1929] Example prompt: "I've been coughing so much at night lately that I can't sleep. What should I do?"

[1930] This invention allows patients with mobility issues to easily receive medical consultations from home, receiving high-quality responses and accurate advice. It also enables prompt collaboration with medical professionals in emergencies, enabling appropriate follow-up. This system, utilizing generative AI models, further supports users' health management.

[1931] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1932] Step 1: User Registration

[1933] Input: User's name, email address, and password

[1934] Specific operation:

[1935] Through the app or web interface, users enter their name, email address, and password and click the "Register" button.

[1936] The server stores the information entered by the user in a database using MySQL.

[1937] The server will notify you, "Registration complete. Please log in."

[1938] Output: Registration completion notification

[1939] Step 2: Login authentication

[1940] Input: User's email address, password

[1941] Specific operation:

[1942] The user enters the registered email address and password and clicks the "Login" button.

[1943] The server checks the user's credentials against a database.

[1944] If the server successfully authenticates the user, it displays the home screen, but if it fails, it notifies the user that "Authentication failed."

[1945] Output: Home screen display or authentication failure notification

[1946] Step 3: Initiating a medical consultation

[1947] Input: Click on the "Start medical consultation" button

[1948] Specific operation:

[1949] The user selects "Start medical consultation" on the home screen.

[1950] The server initializes a generative AI model (e.g., OpenAI's GPT-3).

[1951] The server sends the chatbot interface to the user terminal and starts the session.

[1952] Output: Chatbot interface displayed

[1953] Step 4: Accepting User Input

[1954] Input: User's symptoms and concerns

[1955] Specific operation:

[1956] The user enters their symptoms and concerns in the text box and clicks the send button.

[1957] Output: Text data of input symptoms and anxieties

[1958] Step 5: Generate a response using a generative AI model

[1959] Input: Text data of user symptoms and anxieties

[1960] Specific operation:

[1961] The server sends the user's input text to the generative AI model.

[1962] The generative AI model analyzes the input and generates an appropriate response.

[1963] The server sends the generated response to the user terminal.

[1964] Output: The generated response text

[1965] Step 6: View the response

[1966] Input: Generated response text

[1967] Specific operation:

[1968] The terminal displays the response received from the server on the chatbot interface.

[1969] Output: The response displayed on the chatbot interface

[1970] Step 7: Link with health management app

[1971] Input: Select health management app and click link button

[1972] Specific operation:

[1973] The user configures the settings screen to link with a health management app (e.g., Google Fit or Apple Health).

[1974] The server calls the API of the selected health management app and sets up the connection.

[1975] Output: Notification that the connection with the health management app has been completed

[1976] Step 8: Acquire and analyze health data

[1977] Input: Data obtained from health management app

[1978] Specific operation:

[1979] The server uses the health management app's API to obtain the user's health data.

[1980] The server sends the acquired data to the generative AI model.

[1981] The generative AI model analyzes health data and generates advice about the user's health status.

[1982] The server transmits the generated advice to the user terminal.

[1983] Output: Health advice

[1984] Step 9: Providing advice based on health data

[1985] Input: Generated advice

[1986] Specific operation:

[1987] The terminal displays the advice received from the server on the chatbot interface.

[1988] Output: Display of health advice

[1989] Step 10: Emergency response decision and notification

[1990] Input: User symptoms and health data

[1991] Specific operation:

[1992] The chatbot analyzes the user's symptoms and health data to determine whether emergency response is required.

[1993] If the chatbot determines that an emergency response is required, it notifies the server.

[1994] Output: Notification that urgent action is required

[1995] Step 11: Consult a medical professional

[1996] Input: Notification that emergency action is required

[1997] Specific operation:

[1998] The server notifies the medical professional of the user's consultation and the urgency of the consultation, and performs follow-up.

[1999] Medical professionals will provide additional diagnosis and advice to users via video or text chat.

[2000] Output: Notification that a follow-up with a medical professional will be performed

[2001] The system allows users to easily receive medical consultations from home, receive high-quality responses, and is excellent for quickly connecting with medical professionals in emergencies.

[2002] (Application example 1)

[2003] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2004] While remote medical consultation systems allow users to easily receive medical consultations from home, providing prompt and appropriate support during health consultations at physical stores has been difficult. For this reason, new methods are needed for store staff to provide customers with appropriate health advice in real time and efficiently recommend products.

[2005] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2006] In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; and a means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses. This enables quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[2007] "User" refers to an individual who uses the system to receive remote medical consultations or in-store health consultations.

[2008] "Telemedical consultation" means a service that allows a user to consult with a medical professional online from the comfort of their own home or any other location.

[2009] A "generative AI model" refers to artificial intelligence technology that analyzes user input and generates appropriate responses and advice through natural conversation.

[2010] "User Terminal" means a device (e.g., smartphone, tablet, computer) through which a User accesses the System and conducts a remote medical consultation.

[2011] "Health management app" refers to an application that collects and manages a user's health data.

[2012] "Server" means the computer system that manages and operates the entire system and processes information from the generative AI model and database.

[2013] "Smart glasses" refer to wearable devices that provide information along the user's line of sight and assist staff in providing real-time health consultations in brick-and-mortar stores.

[2014] "Medical Professional" means a medical professional who is qualified to provide professional diagnosis and advice regarding the user's health inquiry.

[2015] The present invention provides a health consultation support system for use in a physical store, which allows users to receive health consultations in real time through store staff wearing smart glasses. Detailed embodiments of the present invention will be described below.

[2016] The server includes means for a user to initiate a remote medical consultation, means for analyzing the user's input using a generative AI model and generating a response, means for displaying the response generated by the generative AI model on the user's terminal, means for acquiring and analyzing the user's health data in cooperation with a health management app, means for providing advice to the user based on the acquired health data, means for transferring the user's consultation content to a medical professional and performing follow-up as necessary, and means for providing real-time health consultation support using the generative AI model in a physical store using smart glasses.

[2017] 1. Overall system configuration

[2018] The system consists of the following main components:

[2019] User devices: smartphones, tablets, computers, etc.

[2020] Server: Cloud-based computing services.

[2021] Generative AI models: such as OpenAI's GPT-4.

[2022] Smart glasses: Google Glass, etc.

[2023] Health management app: An application that collects and manages health data.

[2024] 2. System Operation

[2025] 2.1 User Registration and Login

[2026] The server provides a means for users to access the system through an app or web interface and enter the necessary registration information (name, email address, password, etc.) The server stores this information in a database and manages the authentication information.

[2027] 2.2 Starting a health consultation

[2028] When a user presses the "Start medical consultation" button, the server activates the generative AI model and displays a chatbot interface on the user's device. This chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation content and symptoms.

[2029] 2.3 Conversational exchanges

[2030] When a user inputs a question or symptom into the system, the generative AI model analyzes it and provides an appropriate answer, which is displayed on the user's device and the smart glasses' display.

[2031] 2.4 Health Management Data Linkage

[2032] When a user connects a health management app to the system, the server acquires this data and sends it to the generative AI model, which then analyzes the user's health condition and provides appropriate advice.

[2033] 2.5 Real-time health consultations at physical stores

[2034] When a store employee wearing smart glasses receives a health consultation from a customer, the server sends the information to the generative AI model, which generates a response. This response is displayed in real time on the employee's smart glasses, and the employee can then provide appropriate advice to the customer.

[2035] Specific examples

[2036] For example, if a customer asks how to choose vitamins, the following exchange might occur:

[2037] Staff: "What purpose are you looking for vitamins for?"

[2038] Customer: "I'm feeling tired and I'd like to have more energy."

[2039] Smart glasses display: "To increase your energy, we recommend supplements containing B and C vitamins. Iron may also be beneficial. Do you have any other questions? Learn more about these products."

[2040] Prompt Sentence Examples

[2041] "A client is asking about choosing vitamins. Can you recommend some vitamins or supplements to help them increase their energy?"

[2042] This will enable quick and appropriate health consultations even in physical stores, thereby increasing customer satisfaction.

[2043] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2044] Step 1:

[2045] A user accesses the system through an app or web interface and registers. The user enters the required information, such as name, email address, and password, and sends it to the server. The server stores this information in a database and notifies the user that registration is complete.

[2046] Input: Name, Email Address, Password

[2047] Data processing: Saving user information to a database

[2048] Output: Registration completion notification

[2049] Step 2:

[2050] A user logs in to the system. The user enters the registered email address and password and sends the authentication information to the server. The server authenticates the user and allows the user to access the system.

[2051] Input: Email address, Password

[2052] Data Calculation: Authentication Check

[2053] Output: System access permissions

[2054] Step 3:

[2055] The user presses the "Start medical consultation" button. The server launches the generative AI model and displays the chatbot interface on the user's device. The user's device then begins a dialogue with the user.

[2056] Input: Medical consultation start request

[2057] Data processing: Displaying the chatbot interface

[2058] Output: Medical consultation initiated

[2059] Step 4:

[2060] The user inputs symptoms or questions into the chatbot. The user's device sends the input text to the generative AI model. The server receives it, analyzes it with the generative AI model, and generates an appropriate response.

[2061] Input: User question or symptom

[2062] Data Computation: Question Analysis and Answer Generation

[2063] Output: The generated response

[2064] Step 5:

[2065] The server sends the generated response to the user terminal for display, which displays it to the user for further interaction.

[2066] Input: The generated response

[2067] Data processing: Sending and displaying responses

[2068] Output: Display response to user

[2069] Step 6:

[2070] The user configures the health management app to work with the server, which then acquires the user's health data from the app and sends it to the generative AI model for analysis.

[2071] Input: Health management app data

[2072] Data Computing: Health Data Analysis

[2073] Output: Advice based on health status

[2074] Step 7:

[2075] A user consults a staff member wearing smart glasses at a physical store. The staff member's smart glasses send the user's consultation to the generative AI model, which analyzes it on the server. The server then sends the answer to the staff member's smart glasses and displays it.

[2076] Input: Customer Question

[2077] Data Computation: Question Analysis and Answer Generation

[2078] Output: Response display on smart glasses

[2079] Step 8:

[2080] If necessary, the server will follow up by transferring the user's consultation to a medical professional who will provide further diagnosis and advice via video or text chat.

[2081] Input: User's inquiry

[2082] Data processing: Transfer of consultation details

[2083] Output: Follow-up with medical professionals

[2084] The above processing steps make it possible to provide prompt and appropriate health consultations even in physical stores.

[2085] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2086] The present invention combines an emotion engine with a remote medical consultation system, which can recognize the user's emotional state and provide appropriate responses and medical support through a generative AI model. Specific embodiments of the present invention are described below.

[2087] Overall system configuration

[2088] 1. User Registration and Login

[2089] User: Accesses the system through an application or web interface and creates a new account by entering the required registration information (name, email address, password, etc.).

[2090] Server: Saves the entered user information in the database and notifies the user that registration is complete. Once the user enters their authentication information on the login screen, they are granted access to the system.

[2091] 2. Initiating a medical consultation

[2092] User: When the user presses the "Start medical consultation" button within the application, the server launches the generative AI model and displays the chatbot interface on the user's device.

[2093] Chatbot: The chatbot uses generative AI models to have natural conversations with users and confirm their concerns and symptoms.

[2094] 3. Conversational exchanges

[2095] User: Enters symptoms and concerns into the chatbot. For example, "I've been coughing so much lately I can't sleep at night."

[2096] Chatbot: A generative AI model analyzes input and generates an appropriate response, such as, "How long have you had a cough?"

[2097] 4. Emotion Recognition by Emotion Engine

[2098] Server: Sends user input to the emotion engine and analyzes the emotional state.

[2099] Emotion engine: Recognizes user emotions (e.g., stress, anxiety, anger) and passes the results to a generative AI model.

[2100] Generative AI model: Tailors responses based on emotional state to provide appropriate support to users.

[2101] 5. Health Management Data Linkage

[2102] User: Health data is obtained by linking the health management app with the system from the settings screen.

[2103] Server: Sends data obtained from the health management app to the emotion engine and generative AI model to analyze the user's health condition.

[2104] Chatbot: Provides users with advice based on their health data and emotional state. For example, it might say, "Based on your recent sleep data, it appears that your average sleep time is low. We recommend that you get plenty of rest."

[2105] 6. Follow-up with a medical professional

[2106] Chatbot: If an emergency or specialized treatment is deemed necessary, the generative AI model will notify the system.

[2107] Server: Transfers the user's consultation to a medical professional, if necessary, and arranges for follow-up.

[2108] Medical Expert: Provides additional diagnosis and advice to users via video or text chat.

[2109] Specific examples

[2110] 1. User Registration

[2111] User: Launches the app, enters their name, email address, and password in the registration form, and clicks the "Register" button.

[2112] Server: Notify "Registration complete. Please log in."

[2113] 2. Medical consultation begins

[2114] User: Tap "Start Medical Consultation" on the app's home screen.

[2115] Server: The chatbot displays "Hello, how can we help you?"

[2116] 3. Conversational exchanges

[2117] User: Type "I've been coughing so much at night lately I can't sleep."

[2118] Chatbot: "That's terrible. How long have you had a cough?"

[2119] User: "About a week."

[2120] Chatbot: Ask a follow-up question: "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[2121] 4. Emotion Recognition by Emotion Engine

[2122] Server: Sends the user's input, "About a week" and "Are you experiencing any other symptoms such as fever or fatigue?" to the emotion engine.

[2123] Emotion engine: Recognizes user anxieties and passes that information to a generative AI model.

[2124] Generative AI model: Responds, "Okay, don't worry, this can get better with treatment."

[2125] 5. Health Management Data Linkage

[2126] User: Set up the connection to the health management app.

[2127] Server: Collects health management data and sends this information to the generative AI model and emotion engine.

[2128] Chatbot: "It seems like you've been getting less sleep lately. This may be contributing to your poor health."

[2129] 6. Follow-up with a medical professional

[2130] Chatbot: Notifies the patient, "If this condition persists, you may need to see a specialist."

[2131] Server: Send follow-up requests to medical professionals and schedule video chats.

[2132] Medical expert: Interacts with the user via video chat at a specified time to provide a detailed diagnosis.

[2133] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[2134] The processing flow will be explained below.

[2135] The present invention is a remote medical consultation system with an emotion engine built in. The specific processing flow will be explained below step by step.

[2136] Overall system configuration

[2137] Step 1:

[2138] User: Launches the application and clicks the "Sign Up" button.

[2139] On your device: Display a form for name, email address, password, etc.

[2140] User: Enter the required information and press the "Register" button.

[2141] Terminal: Sends the entered information to the server.

[2142] Step 2:

[2143] Server: Saves the user information in the database and sends a message to the device indicating that registration is complete.

[2144] On the device: Display "Registration successful. Please log in" to the user.

[2145] User: Enter your email address and password on the login screen and click the "Login" button.

[2146] Device: Sends authentication information to the server.

[2147] Step 3:

[2148] Server: Retrieves user information from the database and verifies authentication information.

[2149] Server: If authentication is successful, it sends an authentication token to the device and sends a login success message.

[2150] On the device: Display a login success message and go to the home screen.

[2151] Step 4:

[2152] User: Press the "Start medical consultation" button on the home screen.

[2153] Device: Sends a request to the server to launch the generative AI model.

[2154] Server: Initializes the generative AI model and starts the chatbot session.

[2155] Server: Deliver the chatbot interface to the device and display the message, "What would you like to consult about?"

[2156] Step 5:

[2157] User: Writes down symptoms and concerns in the chat interface and sends it.

[2158] Terminal: Sends the user's messages to the server.

[2159] Server: Passes messages to the generative AI model for analysis.

[2160] Chatbot: Generates appropriate response messages and returns the results to the server.

[2161] Server: Sends a response message to the terminal.

[2162] Terminal: Display the response message.

[2163] Step 6:

[2164] User: Enters additional information in response to the chatbot's questions and submits.

[2165] Terminal: Sends the user's messages to the server.

[2166] Server: Passes messages to the generative AI model and emotion engine.

[2167] Emotion engine: Analyzes user input and recognizes emotional states (e.g., stress, anxiety, relief, etc.).

[2168] Generative AI model: Adjusts responses based on the analysis results of the emotion engine.

[2169] Chatbot: Generates appropriate responses corresponding to emotions and returns them to the server.

[2170] Server: Sends a response message to the terminal.

[2171] Terminal: Display a response message such as "Don't worry, this may improve with treatment."

[2172] Step 7:

[2173] User: Set up integration with the health management app on the app settings screen.

[2174] Device: Sends a request to connect with the health management app to the server.

[2175] Server: Accesses the health management app and retrieves the necessary data.

[2176] Server: Passes acquired health data to the generative AI model and emotion engine.

[2177] Generative AI model: Analyzes health data and assesses the user's health status.

[2178] Emotion Engine: Integrates health data with your current emotional state to provide comprehensive analysis.

[2179] Server: Sends health assessment results and advice to the device.

[2180] Device: Display advice such as, "It appears you've been getting less sleep recently. This may be contributing to your poor health."

[2181] Step 8:

[2182] Chatbot: The generative AI model determines whether the user's inquiry is urgent or not based on the content of the inquiry and the results of emotional analysis.

[2183] Server: Sends requests to medical professionals and sets up follow-ups as needed.

[2184] Medical Professionals: Provide users with additional diagnosis and advice via video or text chat.

[2185] Server: Saves diagnosis results and advice in a database to help with future medical consultations.

[2186] The present invention allows patients with mobility issues to easily receive medical consultations from home, and allows for more personalized health management that takes into account the user's emotional state.

[2187] Example 2

[2188] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2189] In remote medical consultations, it can be difficult for users to accurately communicate their symptoms, and responses that ignore the user's emotional state can result in insufficient medical support.Furthermore, there is a lack of a system for appropriately utilizing users' health management data, making it difficult to provide appropriate advice and follow-up that is tailored to each individual user.

[2190] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2191] In this invention, the server includes: a means for a user to initiate a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user's terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation to a medical professional and performing follow-up as necessary; and a means for recognizing the user's emotional state and providing appropriate responses and medical support through the generative AI model. This enables accurate understanding of the user's symptoms and personalized medical support that takes their emotional state into account. Furthermore, by appropriately utilizing health management data, it is possible to provide advice tailored to each individual user and follow-up according to the level of urgency.

[2192] "Telemedical consultation" is a system that allows users to receive medical consultation through a communication network without physically visiting a medical institution.

[2193] A "generative AI model" is an algorithm or system that uses artificial intelligence to analyze and understand user input and generate an appropriate response.

[2194] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses to access the system.

[2195] A "health management app" is an application that collects and manages a user's health-related data (e.g., sleep data, exercise data, vital signs).

[2196] "Emotional state" refers to the user's emotional state (e.g., stress, anxiety, anger) as analyzed by the emotion engine.

[2197] The "emotion engine" is a component that analyzes emotions from user input and provides that information to the generative AI model.

[2198] "Medical professionals" are professionals with specialized medical knowledge, such as doctors, nurses, and pharmacists.

[2199] "Follow-up" refers to subsequent medical intervention to provide additional diagnosis or advice based on the user's consultation.

[2200] "Analysis" is the process of understanding input data and extracting or generating meaningful information based on it.

[2201] MODE FOR CARRYING OUT THE INVENTION

[2202] The present invention relates to a system for remote medical consultations. This system combines a generative AI model and an emotion engine to provide appropriate medical support based on the user's input and emotional state. Specific embodiments for implementing the present invention are described below.

[2203] System Configuration

[2204] This system mainly consists of the following components:

[2205] User devices (e.g. smartphones, tablets, PCs)

[2206] server

[2207] Generative AI Models

[2208] Emotion Engine

[2209] Health management app

[2210] Linking to medical professionals

[2211] The role of each component

[2212] 1. User Device

[2213] This is an electronic device that users use to conduct remote medical consultations. Through an application or web interface, users register and log in to an account and begin a medical consultation.

[2214] 2. Server

[2215] The server plays a central role in the entire system: it manages user authentication information, runs the generative AI model and emotion engine, sends responses to the user's device, and also coordinates data with the health management app and forwards follow-up information to medical professionals.

[2216] 3. Generative AI Models

[2217] It includes algorithms for analyzing user input and generating appropriate responses. The generative AI model generates natural conversations based on the user's symptoms and questions, providing accurate advice to the user.

[2218] 4. Emotion Engine

[2219] It analyzes user input and recognizes emotional states (e.g., stress, anxiety, anger). The emotion engine passes the results to a generative AI model, which then uses it to tailor responses.

[2220] 5. Health Management App

[2221] It is an application that collects and manages users' health data (e.g., sleep data, exercise data, vital signs). This data is sent to a server and analyzed by a generative AI model and emotion engine.

[2222] 6. Linking to medical professionals

[2223] If an emergency or specialized treatment is deemed necessary, the server will send a request to a medical professional who will provide the user with additional diagnosis and advice via video or text chat.

[2224] Example of operation

[2225] 1. User Registration and Login

[2226] A user launches the application, enters their name, email address, and password in the registration form, and clicks "Register."

[2227] The server saves the entered information in a database and notifies the user that "Registration is complete. Please log in."

[2228] 2. Initiating a medical consultation

[2229] The user taps "Start medical consultation" on the app's home screen.

[2230] The server launches an instance of the generative AI model and chatbot, which then displays "Hello, how can I help you?"

[2231] 3. Conversational exchanges

[2232] The user types, "Recently, I've been coughing so much at night that I can't sleep."

[2233] The chatbot responds, "That's terrible. How long have you had this cough?"

[2234] The user responds, "About a week."

[2235] The chatbot asks additional questions, such as, "Your persistent cough could be due to a cold or allergies. Do you also have a fever or fatigue?"

[2236] 4. Emotion Recognition by Emotion Engine

[2237] The server sends the user's input to the emotion engine.

[2238] The emotion engine recognizes the user's anxiety and passes that information to the generative AI model.

[2239] The generative AI model responds, "Okay, don't worry, this can get better with treatment."

[2240] 5. Health Management Data Linkage

[2241] The user configures the settings to link with the health management app.

[2242] A server collects health management data and sends this information to a generative AI model and emotion engine.

[2243] The chatbot advises, "It seems you've been sleeping less recently. This may be causing your health to deteriorate."

[2244] 6. Follow-up with a medical professional

[2245] The chatbot will notify the user, "If this condition persists, you may need to see a specialist."

[2246] The server sends follow-up requests to medical professionals and schedules video chats.

[2247] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[2248] Prompt Sentence Examples

[2249] "Provide advice to users based on their recent health data."

[2250] "Analyze the user's emotional state and generate a conversation to provide appropriate support."

[2251] "Please outline your follow-up procedures when urgent symptoms are reported."

[2252] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2253] A detailed explanation of the program's processing steps

[2254] Step 1: User registration and login

[2255] A user opens the application or web interface, enters their name, email address, and password in the new registration form, and clicks the "Register" button.

[2256] Input: Name, Email Address, Password

[2257] The server receives the entered information and stores it in a database.

[2258] Output: Notification of successful registration

[2259] The server notifies the user, "Registration complete. Please log in."

[2260] Step 2: Log in

[2261] The user enters their email address and password on the login screen and clicks the "Login" button.

[2262] Input: Email address, password

[2263] The server checks the entered credentials against its database.

[2264] Output: Notification of successful or failed login

[2265] If the authentication is successful, the server notifies the user that "Login was successful" and displays the dashboard screen.

[2266] Step 3: Initiating a medical consultation

[2267] The user taps "Start medical consultation" on the app's home screen.

[2268] Input: Request to start a medical consultation

[2269] The server launches an instance of the generated AI model and chatbot.

[2270] Output: Chatbot interface displayed

[2271] The server displays the chatbot interface on the user's device and asks the user, "Hello. What would you like to discuss with us?"

[2272] Step 4: Dialogue

[2273] The user inputs their symptoms into the chatbot (e.g., "I've been coughing so much at night lately that I can't sleep").

[2274] Input: User's symptoms

[2275] The chatbot uses a generative AI model to analyze user input.

[2276] Output: Generate an appropriate question (e.g., "How long has your cough lasted?")

[2277] The chatbot returns the generated question to the user.

[2278] Step 5: Emotion Recognition with the Emotion Engine

[2279] The server sends the user's input to the emotion engine.

[2280] Input: What the user types

[2281] The emotion engine analyzes the input and recognizes the user's emotional state (e.g., anxiety, stress, anger).

[2282] Output: Emotional state analysis results

[2283] The emotion engine passes the analysis results to the generative AI model.

[2284] The generative AI model adjusts its response based on the emotional state, saying, "Okay, don't worry, this can get better with treatment."

[2285] Step 6: Health management data integration

[2286] The user sets up linkage with the health management app on the settings screen.

[2287] Input: Health management app link information

[2288] The server acquires health data from the health management app.

[2289] Output: Health data collection

[2290] The server sends the collected health data to the generative AI model and emotion engine.

[2291] The generative AI model analyzes health data, and the chatbot advises, "It appears you've been sleeping less recently. This may be contributing to your poor health."

[2292] Step 7: Follow up with a medical professional

[2293] The chatbot analyzes the user's symptoms and input, and determines whether the condition is urgent or requires specialized treatment.

[2294] Input: User symptoms and input

[2295] The server sends a follow-up request to the medical professional.

[2296] Output: Send follow-up request

[2297] The server schedules the video chat and notifies the user.

[2298] A medical professional will meet with the user via video chat at the appointed time to provide a detailed diagnosis.

[2299] (Application example 2)

[2300] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2301] While remote medical consultation systems provide support for users regarding their health status, they face challenges in providing personalized advice that takes into account the user's emotional state and in providing insufficient security measures. Furthermore, because the user's emotional state can affect the overall response quality of the system, there is a need for an analysis of the user's emotional state and the generation of appropriate responses based on that state.

[2302] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to start a remote medical consultation; a means for analyzing the user's input using a generative AI model and generating a response; a means for displaying the response generated by the generative AI model on the user terminal; a means for acquiring and analyzing the user's health data in cooperation with a health management app; a means for providing advice to the user based on the acquired health data; a means for transferring the user's consultation content to a medical professional and performing follow-up as necessary; hardware including an emotion engine that analyzes and recognizes the user's emotional state; and a means for generating a response to the user based on the recognized emotional state and providing security advice. This enables personalized medical support and security measures that take the user's emotional state into consideration.

[2303] A "means for a user to initiate a remote medical consultation" is an interface that allows a user to initiate a consultation with a medical professional using communication technology.

[2304] "Means of using a generative AI model to analyze user input and generate a response" refers to the process of using artificial intelligence to analyze information entered by a user and generate an appropriate response based on that information.

[2305] "Means for displaying the response generated by the generative AI model on the user's device" refers to a method for visualizing the response created by the generative AI model on the user's device.

[2306] The "means for acquiring and analyzing user health data in cooperation with a health management application" is a method for collecting and analyzing user health-related data in cooperation with a health management application.

[2307] The "means for providing advice to the user based on the acquired health data" is a process of analyzing the collected health data and providing appropriate advice to the user based on the data.

[2308] "Means for transferring the user's consultation to a medical professional as needed and for carrying out follow-up" refers to a method for transferring the user's consultation to a medical professional when the user's consultation requires specialized treatment and for carrying out continuous follow-up.

[2309] "Hardware including an emotion engine that analyzes and recognizes a user's emotional state" is a specific hardware device that has the function of analyzing and recognizing a user's emotion.

[2310] The "means for generating a response to a user based on a recognized emotional state and providing security advice" is a method for generating an appropriate response based on the user's emotions recognized by the emotion engine and further providing security advice to the user.

[2311] The present invention combines an emotion engine with a remote medical consultation system, recognizing the user's emotional state and providing appropriate responses and security advice through a generative AI model.

[2312] A system for implementing the present invention uses the following major hardware and software components:

[2313] Hardware

[2314] Smartphone: The device on which the user operates the application.

[2315] Server: A device that hosts data processing and generative AI models, emotion engines, and databases.

[2316] Emotion engine: Dedicated hardware for analyzing user input and recognizing emotional states.

[2317] software

[2318] Application interface: An app that allows users to initiate medical consultations and input emotional and health data.

[2319] Generative AI model: An algorithm that analyzes user input and generates an appropriate response.

[2320] REST API server: Software that handles communication between the client (smartphone app) and the server.

[2321] Health management app: Software that collects and provides user health data.

[2322] Database: A storage system for storing user information, health data, and emotional state data.

[2323] Processing steps

[2324] 1. User Registration

[2325] The user launches the smartphone app and enters the required registration information (name, email address, password). The application interface sends this information to the REST API server, which stores it in a database. Once registration is complete, the server returns a confirmation message.

[2326] 2. Log in

[2327] A user logs in from a smartphone app using their email address and password. The information is sent back to the REST API server, which verifies the authentication information in the database. If authentication is successful, the server issues a session token.

[2328] 3. Initiating a medical consultation

[2329] The user presses the "Start medical consultation" button within the application, and their smartphone sends this request to the server, which then activates the generative AI model and displays the chatbot interface on the user's device.

[2330] 4. Conversational exchanges

[2331] The chatbot uses the generative AI model to have a natural conversation with the user and confirm the consultation details and symptoms. For example, if a user inputs "I've been coughing so much at night recently that I can't sleep," the generative AI model will respond with "How long has this cough been going on for?"

[2332] 5. Emotion Recognition by Emotion Engine

[2333] The server sends the user's conversation content to the emotion engine, which recognizes the user's emotional state. The recognized emotional state is passed to the generative AI model, which adjusts the response. For example, if the user's anxiety is recognized, the generative AI model responds, "Don't worry, this can get better with treatment."

[2334] 6. Health Management Data Linkage

[2335] The user connects their health management app to the system, and health data (e.g., sleep time, steps taken, heart rate, etc.) is sent to the server. This data is analyzed using a generative AI model and an emotion engine. The chatbot then advises, "It seems you've been getting less sleep recently. This may be contributing to your poor health."

[2336] 7. Follow-up with a medical professional

[2337] If advanced medical support is deemed necessary, the generative AI model notifies the system and transfers the user's consultation to a medical professional, who can provide additional diagnosis and advice via video or text chat.

[2338] Examples of concrete examples and prompts

[2339] 1. Example:

[2340] User registration: A user registers in the app by entering "test_user", "user@example.com", and "password123".

[2341] Login: Log in with the same user information.

[2342] Security Session: Tap the "Start Security Session" button to start the session.

[2343] Emotional data transmission: Data expressing "anxiety" is transmitted via the app.

[2344] Security advice: "There has been an increase in the number of accesses to your device recently. Please set up two-step authentication."

[2345] 2. Example prompt:

[2346] User: I've been feeling a bit uneasy with my devices lately. Any advice?

[2347] Chatbot: That's unfortunate. We've noticed some suspicious activity in your recent logs. We recommend you enable two-factor authentication and change your password. If you'd like more information, please type "Tell me more."

[2348] This will specifically specify the mode for carrying out the present invention, and serve as a reference for others to accurately understand and practice the invention.

[2349] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2350] Step 1: User Registration

[2351] Input: A user uses a smartphone app to enter their name, email address, and password.

[2352] Processing: The smartphone app sends the entered information to the REST API server, which saves the information in a database and completes the user registration.

[2353] Output: The server sends a successful registration notification back to the app, and the app displays "Registration successful" to the user.

[2354] Step 2: Log in

[2355] Input: The user enters the email address and password they registered on the smartphone app.

[2356] Processing: The smartphone app sends the entered authentication information to the REST API server. The server checks the authentication information by referencing the database, and if authentication is successful, issues a session token.

[2357] Output: The server sends a successful authentication response and a session token back to the app, and the app notifies the user that the login was successful.

[2358] Step 3: Initiating a medical consultation

[2359] Input: The user presses the "Start medical consultation" button in the smartphone app.

[2360] Processing: The smartphone app sends a request to start a consultation to the REST API server. The server starts the generative AI model and prepares to display the chatbot interface on the user's device.

[2361] Output: The server displays the chatbot interface on the user device and asks the user, "Hello, how can I help you?"

[2362] Step 4: Dialogue

[2363] Input: The user inputs their symptoms and concerns into the chatbot (e.g., "I've been having a bad cough lately and can't sleep at night").

[2364] Processing: The chatbot uses a generative AI model to analyze the user's input data and generate an appropriate response (e.g., "How long have you had a cough?").

[2365] Output: The server sends the generated response to the user's device, where the chatbot displays it to the user.

[2366] Step 5: Emotion Recognition with the Emotion Engine

[2367] Input: Text data entered by the user (e.g., "I've been coughing so much at night lately I can't sleep").

[2368] Processing: The server sends the text data to the emotion engine, which analyzes it and recognizes the user's emotional state (e.g., anxiety) as a result of the analysis.

[2369] Output: The emotion engine passes the recognized emotional state to the generative AI model, which tailors the response based on the emotion. The tailored response is displayed on the user's device.

[2370] Step 6: Health management data integration

[2371] Input: Health data collected from health management apps (e.g., sleep duration, steps, heart rate, etc.).

[2372] Processing: The server connects with the health management app to acquire health data, which is then analyzed using a generative AI model and emotion engine to evaluate the user's health status.

[2373] Output: Health advice generated based on the analysis results (e.g., "It appears you've been sleeping less recently. This may be causing your health condition to worsen") is displayed on the user's device.

[2374] Step 7: Follow up with a medical professional

[2375] Input: User consultation details and health data for any issues deemed urgent or requiring specialized care.

[2376] Processing: Based on the judgment of the generated AI model, the server forwards the user's consultation to a medical professional, who provides further diagnosis and advice to the user via video chat or text chat.

[2377] Output: The user is notified that the follow-up appointment has been scheduled and completed. The user will then have a video or text conversation with a medical professional at the designated time.

[2378] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2379] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2380] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2381] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2382] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2383] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2384] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2385] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the ...

Claims

1. a means for a user to initiate a telemedicine consultation; a means for analyzing a user's input using a generative AI model to generate a response; A means for displaying a response generated by the generative AI model on a user terminal; A means of acquiring and analyzing user health data in cooperation with a health management app; means for providing advice to the user based on the acquired health data; a means for forwarding the user's consultation to a medical professional and carrying out follow-up as necessary; A system including:

2. The system of claim 1, wherein the generative AI model is provided with a means for digging deeper into the user's consultation through continuous conversation with the user and asking detailed questions.

3. 2. The system according to claim 1, further comprising means for managing user authentication information and for allowing users to log into the system.

Images