System

Abstract

A system is provided.SOLUTION: A system comprising: an input unit for inputting text information about a symptom by a patient; a receiving unit for receiving the text information about the symptom input by the input unit; an analyzing unit for analyzing the text information received by the receiving unit and predicting a disease name; a diagnosis generating unit for generating the disease name predicted by the analyzing unit; and a notifying unit for notifying the patient of the examination date determined by the appointment optimizing unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Long waiting times at hospitals are a major issue for both medical institutions and patients. Long waiting times hinder efficient hospital operations and cause wasted time and stress for patients. To solve this problem, there is a need to optimize hospital reservation systems and improve the accuracy of pre-diagnosis, but current systems do not fully achieve this. Furthermore, if the information necessary for diagnosis is not properly provided to doctors, there is a risk of reduced medical efficiency and misdiagnosis. Furthermore, since some symptoms do not require a physical visit to a hospital, the introduction of online diagnosis is also an important issue. [Means for solving the problem]

[0005] The present invention is a medical support system that aims to reduce waiting times and improve the efficiency of medical treatment by having patients input text information about their symptoms as well as image and video data, and then using AI to perform highly accurate analysis based on that information. Specifically, it includes the following means.

[0006] The system includes an input means for a patient to input text information about symptoms, a receiving means for receiving the text information about symptoms input by the input means and image and video data, an analysis means for analyzing the text information and image and video data received by the receiving means and predicting the name of a disease, a diagnosis generation means for generating the disease name predicted by the analysis means and its basis, an appointment optimization means for determining the optimal appointment date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time, and a notification means for notifying the patient of the appointment date and time determined by the appointment optimization means.

[0007] Furthermore, by providing an online diagnosis suggestion means that suggests the option of online diagnosis to patients when it is determined that online diagnosis is possible, medical treatment can be carried out efficiently even when a physical hospital visit is not necessary. Also, by providing doctors with predicted disease names and diagnostic evidence and providing medical treatment support means to support medical treatment, the efficiency of doctors' medical treatment can be improved and the risk of misdiagnosis can be reduced. This can significantly improve the operational efficiency of hospitals and patient satisfaction.

[0008] "Patient" refers to a person who visits a healthcare facility or uses healthcare services.

[0009] "Text information about symptoms" refers to written information in which a patient describes their health condition or abnormalities in their own words.

[0010] "Image and video data" refers to still images and video data submitted by patients to demonstrate their symptoms or abnormalities.

[0011] "Input means" refers to a device or system used by a patient to enter information about their symptoms, such as a smartphone application or a computer web form.

[0012] The "receiving means" refers to a device or system that receives the text information and image and video data related to symptoms sent from the input means.

[0013] "Analysis means" refers to a device or algorithm that analyzes the text information and image or video data related to symptoms received by the receiving means and predicts the name of the disease.

[0014] The "diagnosis generating means" refers to a device or system that generates the disease name predicted by the analysis means and its basis as a single piece of diagnostic information.

[0015] "Reservation optimization means" refers to a device or system that determines the optimal consultation date and time based on the generated diagnostic information, taking into account existing reservation status and the patient's desired date and time.

[0016] "Notification means" refers to a device or system for notifying patients of the optimized appointment date and time, such as a smartphone push notification or email system.

[0017] "Online diagnosis suggestion means" refers to a device or system for suggesting the option to a patient when it is determined that online diagnosis is possible.

[0018] "Medical treatment support means" refers to devices and systems that provide doctors with predicted disease names and diagnostic evidence to support medical treatment. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The medical support system of the present invention allows patients to input their symptoms, and then uses artificial intelligence to perform highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. The elements of the system are as follows:

[0041] 1. Initial Setup and Login

[0042] User Login

[0043] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[0044] 2. Enter your symptoms

[0045] Symptom text entry

[0046] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[0047] 3. Symptom analysis

[0048] Data reception and analysis

[0049] The server receives text information and multimodal data (images and videos) sent from the device. The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and submits the most likely diagnosis.

[0050] 4. Generating diagnostic results

[0051] Creating diagnostic information

[0052] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[0053] 5. Optimizing appointment scheduling

[0054] Checking reservation status and deciding on consultation date and time

[0055] The server accesses the hospital's reservation system and checks the current reservation status. The server determines the optimal appointment date and time based on the user's desired date and time and available appointment times. The server then sends the determined appointment date and time to the terminal.

[0056] 6. Notifications and Online Diagnostics Options

[0057] Notification of appointment date and time

[0058] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[0059] 7. Start of consultation

[0060] User selection and consultation execution

[0061] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[0062] Specific examples

[0063] Symptom input and diagnosis

[0064] A user (say, Ichiro Tanaka) logs into the hospital's app and enters information about his sore throat. Tanaka enters the text "My sore throat has lasted for three days" and uploads an image of his throat. The device then sends the text information and image to the server.

[0065] Symptom analysis

[0066] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reason for this (text of symptoms, image analysis results).

[0067] Appointment optimization and notifications

[0068] The server references the hospital's reservation system to check whether there is an opening in the morning of the following day, which is Tanaka's desired time. It determines that the optimal consultation time is 11:00 AM the following day and notifies the terminal. The terminal then notifies Tanaka, "Please come in for a consultation at 11:00 AM tomorrow." On the day, Tanaka visits the hospital at 11:00 AM and is examined with a short wait.

[0069] Online Diagnostics Example

[0070] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Tanaka of the suggestion of a video call as an "online diagnosis option." Tanaka selects the online diagnosis and receives a consultation via video call at the specified time.

[0071] The present invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] The user accesses the hospital's app or website and enters their login information (user ID and password).

[0075] Step 2:

[0076] The terminal acquires the entered login information and sends it to the server.

[0077] Step 3:

[0078] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[0079] Step 4:

[0080] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[0081] Step 5:

[0082] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[0083] Step 6:

[0084] The terminal acquires the input text information and transmits it to the server.

[0085] Step 7:

[0086] If the user wishes, they can upload images or videos related to their symptoms.

[0087] Step 8:

[0088] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[0089] Step 9:

[0090] The server receives text information and multimodal data (images and videos) sent from the device.

[0091] Step 10:

[0092] The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology.

[0093] Step 11:

[0094] The AI ​​in the server uses computer vision technology to analyze image and video data.

[0095] Step 12:

[0096] The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[0097] Step 13:

[0098] The server compiles the disease name generated by the AI ​​and the basis for the diagnosis into a single piece of diagnostic information.

[0099] Step 14:

[0100] The server accesses the hospital's reservation system and checks the status of existing reservations.

[0101] Step 15:

[0102] The server determines the optimal consultation date and time taking into consideration the user's desired date and time and available appointment times.

[0103] Step 16:

[0104] The server transmits the determined consultation date and time to the terminal.

[0105] Step 17:

[0106] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[0107] Step 18:

[0108] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[0109] Step 19:

[0110] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[0111] Step 20:

[0112] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[0113] Step 21:

[0114] Doctors can conduct efficient examinations based on the information provided.

[0115] Example 1

[0116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] In modern healthcare, it is important for patients to quickly and accurately communicate their symptoms to medical institutions, but this has been difficult with conventional systems. Long wait times before seeing a doctor are also a factor in lower patient satisfaction. Furthermore, appointment scheduling systems are inefficient, often preventing patients from scheduling appointments at their desired dates and times. Therefore, there is a need for a system that can efficiently analyze patients' symptoms and determine the optimal appointment date and time, thereby improving the operational efficiency of medical institutions and increasing patient satisfaction.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0119] In this invention, the server includes an input means for the patient to input text information about symptoms, a receiving means for receiving the text information about symptoms and image and video data input by the input means, an analysis means for analyzing the text information and image and video data received by the receiving means to predict a disease name, a diagnosis generation means for generating the disease name predicted by the analysis means and its basis and compiling it into diagnostic information, an appointment optimization means for accessing the medical institution's appointment system and determining the optimal appointment date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time, and a notification means for notifying the patient of the appointment date and time determined by the appointment optimization means. This makes it possible to efficiently analyze the patient's symptoms to assist in diagnosis and quickly determine the optimal appointment date and time.

[0120] "Patient" refers to a person who receives medical examination or treatment from a medical institution.

[0121] "Text information about symptoms" refers to written information in which a patient describes in detail their symptoms or discomfort.

[0122] "Input means" refers to a device or application function that allows a patient to input text information about their symptoms.

[0123] The "receiving means" refers to a mechanism that acquires text information and image or video data input by the input means and transmits it to the server.

[0124] "Analysis means" refers to an algorithm or model that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease from the symptoms.

[0125] The "diagnosis generating means" refers to a mechanism that generates the disease name predicted by the analysis means and its basis, and compiles it as diagnostic information.

[0126] The "reservation optimization means" refers to a mechanism for accessing a medical institution's reservation system and determining the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[0127] The "notification means" refers to a mechanism that notifies the patient of the consultation date and time determined by the appointment optimization means.

[0128] The "online diagnosis suggestion means" refers to a mechanism for suggesting the option of online diagnosis to a patient when it is determined that online diagnosis is possible.

[0129] "Medical support tools" refer to mechanisms that provide medical professionals with predicted disease names and diagnostic evidence, thereby improving the efficiency of medical treatment.

[0130] "Medical institution" refers to a facility such as a hospital, clinic, or doctor's office that provides medical examinations and treatment.

[0131] MODE FOR CARRYING OUT THE INVENTION

[0132] The medical support system of the present invention is a system in which patients input their symptoms, and based on that, artificial intelligence performs highly accurate analysis to assist in diagnosis, and determines and notifies the patient of the optimal consultation date and time. Each element of this system will be described in detail.

[0133] Initial Setup and Login

[0134] Login Process

[0135] A user accesses a hospital's app or website and enters their user ID and password into the displayed login form. The device sends this information to the server via a secure protocol (e.g., HTTPS). The server receives the login information and authenticates it by checking it against the user information in its database. If authentication is successful, the server generates a message indicating successful authentication and session information and returns it to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[0136] Enter symptoms

[0137] Enter symptoms and submit data

[0138] The user enters detailed text about their symptoms, such as "I've had a sore throat for three days." The device collects this text information and uploads images and videos as needed. All information, including uploaded image and video data, is then sent to the server.

[0139] Symptom analysis

[0140] Data analysis

[0141] The server uses AI (for example, the Gemini model, which uses NLP technology) to analyze the received text information and image and video data. The text information is analyzed using natural language processing (NLP) technology to extract symptom characteristics and correlations. The image and video data is also analyzed using computer vision technology to recognize visual symptom characteristics. The AI ​​on the server integrates this information and suggests the most likely diagnosis.

[0142] Generating diagnostic results

[0143] Creating diagnostic information

[0144] The server compiles the AI-generated disease names and the diagnostic evidence, building diagnostic information that is saved in a format that is easy for patients to refer to later.

[0145] Optimizing medical appointments

[0146] Checking reservation status and deciding on consultation date and time

[0147] The server accesses the hospital's reservation system to check the current reservation status. Then, it determines the optimal appointment date and time based on the user's desired date and time and the available appointment times. It then sends the determined appointment date and time to the terminal.

[0148] Choosing between notifications and online diagnostics

[0149] Notification of appointment date and time and online diagnosis suggestions

[0150] The device notifies the user of the consultation date and time received from the server and displays the specific date and time. In some cases, confirmations and preparations for the consultation are displayed. If the server determines that an online diagnosis is sufficient, the device offers the option of an online diagnosis. If the user selects an online diagnosis, the device displays a link to a video call or provides instructions for the corresponding app.

[0151] Start of consultation

[0152] Conducting an examination

[0153] The user can either visit the hospital at the notified appointment date and time or choose online diagnosis. The server provides medical professionals with diagnostic information (disease name and diagnostic basis) to support them in providing efficient medical care.

[0154] Specific examples

[0155] Specific examples of symptom input and diagnosis

[0156] A user (e.g., a patient) logs into a hospital app and enters details about a sore throat. The user enters the text "I have had a sore throat for three days" and uploads an image of their throat. The device then sends this information to the server.

[0157] Specific examples of symptom analysis

[0158] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that the condition is likely to be "pharyngitis." The server then generates "pharyngitis" and the basis for this (text of symptoms and image analysis results) as diagnostic information.

[0159] Specific examples of appointments and notifications

[0160] The server checks the hospital's reservation system and confirms that there is availability for the morning of the following day, as desired by the patient. It determines that the optimal appointment time is 11:00 AM the following day and notifies the terminal of this. The terminal then notifies the patient, "Please come in for an appointment at 11:00 AM tomorrow." On the day of the appointment, the patient can visit the hospital at 11:00 AM and be seen with a short waiting time.

[0161] Examples of online diagnostics

[0162] The diagnosis results in "mild pharyngitis," and the server determines that online diagnosis is sufficient. The device notifies the patient of the "online diagnosis option" and suggests a video call. The patient selects online diagnosis and receives a consultation via video call at the specified time.

[0163] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.The following is also used as an example of a prompt sentence to input into the generative AI model:

[0164] "I've had a sore throat for three days. Please see the image below. What is the name of the disease that corresponds to this symptom?"

[0165] The above is a specific embodiment of the present invention.

[0166] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0167] Step 1:

[0168] Login Process

[0169] input:

[0170] The user accesses the hospital's app or website and enters their user ID and password into the login form that appears on the screen.

[0171] Operation:

[0172] The device sends this information to the server via a secure protocol (e.g., HTTPS).

[0173] output:

[0174] The server receives the login information, compares it with the user information in the database, and determines whether authentication is successful. If successful, it generates a message indicating successful authentication and session information, and returns them to the terminal.

[0175] Specific behavior:

[0176] The server queries the database to retrieve user information.

[0177] If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0178] The device will save the session ID and display a screen to proceed to the next step.

[0179] Step 2:

[0180] Enter symptoms and submit data

[0181] input:

[0182] Users enter detailed text about their symptoms and, if they wish, upload images or videos.

[0183] Operation:

[0184] The device acquires the entered text information and sends it to the server along with image and video data.

[0185] output:

[0186] The server receives the text information and image and video data.

[0187] Specific behavior:

[0188] The device temporarily stores the text information in its storage.

[0189] When a user uploads an image or video, the device checks the file format and size.

[0190] The device sends text, image, and video data to the server.

[0191] Step 3:

[0192] Data analysis

[0193] input:

[0194] Text information and image or video data received by the server.

[0195] Operation:

[0196] The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology to extract symptom characteristics and correlations, while image and video data is analyzed using computer vision technology.

[0197] output:

[0198] The AI ​​will suggest the most likely diagnosis based on the integrated information.

[0199] Specific behavior:

[0200] The server performs text analysis and extracts keywords and phrases.

[0201] Recognizing visual symptoms (e.g. redness or swelling of the throat) through image and video analysis.

[0202] Integrates NLP and computer vision results to implement AI algorithms for disease prediction.

[0203] Step 4:

[0204] Creating diagnostic information

[0205] input:

[0206] The disease name proposed through analysis and the supporting data.

[0207] Operation:

[0208] The server compiles the predicted disease name and the basis for the diagnosis as diagnostic information.

[0209] output:

[0210] Diagnostic information is generated and saved in a format that can be later referenced by the user.

[0211] Specific behavior:

[0212] The server stores the diagnostic information (disease name, reason) in a database as structured data.

[0213] -Include user-friendly explanations for diagnostic information.

[0214] Step 5:

[0215] Checking reservation status and deciding on consultation date and time

[0216] input:

[0217] The user's desired date and time and medical institution reservation status.

[0218] Operation:

[0219] The server accesses the hospital's reservation system to check the existing reservation status, and then determines the optimal appointment date and time based on the user's desired date and time.

[0220] output:

[0221] The determined consultation date and time is generated and transmitted to the terminal.

[0222] Specific behavior:

[0223] The server sends a query to the hospital's booking system through an API.

[0224] The server selects the optimal time based on the user's desired date and time and available reservation times.

[0225] The determined date and time is saved as session information and sent to the device.

[0226] Step 6:

[0227] Notification of appointment date and time and online diagnosis suggestions

[0228] input:

[0229] Decided appointment date and time and decision on online diagnosis.

[0230] Operation:

[0231] The terminal notifies the user of the consultation date and time received from the server and displays the specific date and time. If the server suggests online diagnosis, the terminal notifies the user of the online diagnosis option.

[0232] output:

[0233] The user is notified of appointment times and online diagnostic options.

[0234] Specific behavior:

[0235] The device will notify you of the appointment date and time and display a message on the screen saying, "Please come in for an appointment tomorrow at 11:00 AM."

[0236] -You will be presented with the option to do an online diagnostic and provided with a link if you wish.

[0237] Step 7:

[0238] Conducting an examination

[0239] input:

[0240] User-selected consultation date and time or online diagnosis

[0241] Operation:

[0242] The user visits the hospital at the designated appointment date and time, and the server provides the diagnosis information to the medical professional. In the case of online diagnosis, the consultation is conducted via video call.

[0243] output:

[0244] The examination is carried out efficiently.

[0245] Specific behavior:

[0246] The server provides medical professionals with diagnostic information (disease name and diagnostic basis).

[0247] -Medical professionals will examine the user based on the diagnostic information and provide appropriate treatment.

[0248] The above is the flow of processing of the program of this system.

[0249] (Application example 1)

[0250] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0251] In the modern healthcare system, the time and effort required for patients to access medical institutions and receive diagnosis and treatment is a problem. Furthermore, to receive an accurate diagnosis, a large amount of medical information must be provided to doctors, and efficient methods for doing this are needed. In particular, with the widespread use of online diagnosis, real-time diagnostic support and optimization of consultation appointments are important challenges.

[0252] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0253] In this invention, the server includes an input means for patients to input text information about their symptoms, a receiving means, an analyzing means, a diagnosis generating means, a reservation optimization means, a notification means, an online phone consultation means, and a prompting means. This allows patients to receive appropriate consultations efficiently and quickly. Furthermore, if an online consultation is recommended, real-time consultations can be conducted via video calls, improving the efficiency of the entire medical process. Furthermore, by guiding patients through the input of prompts, the patient can smoothly input information, which helps support accurate diagnoses.

[0254] "Input means" refers to a device or function that allows a patient to input text information about symptoms.

[0255] The "receiving means" is a device or function that receives text information and image and video data related to symptoms input by the input means.

[0256] The "analysis means" is a device or function that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease.

[0257] The "diagnosis generating means" is a device or function that generates the name of the disease predicted by the analysis means and the basis for that name.

[0258] The "reservation optimization means" is a device or function that determines the optimal consultation date and time taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[0259] The "notification means" is a device or function that notifies the patient of the consultation date and time determined by the reservation optimization means.

[0260] "Online call consultation means" refers to a device or function that conducts consultation via video call when online diagnosis is recommended.

[0261] The "prompt guide means" is a device or function that guides the user to input a prompt sentence.

[0262] This invention relates to a system in which patients input their symptoms, and artificial intelligence (AI) performs highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. This system includes the following elements.

[0263] System Overview

[0264] 1. Input Method

[0265] Patients use a smartphone app to enter text information about their symptoms, such as "I've had a sore throat for three days," and upload images and video data related to their symptoms if necessary.

[0266] 2. Receiving Method

[0267] The server receives the text information and image and video data input by the input means.

[0268] 3. Analysis method

[0269] The server analyzes the received text information using natural language processing (NLP) technology, and image and video data using computer vision technology. Based on this analysis, the AI ​​predicts the most likely diagnosis based on the patient's symptoms. Examples of AI technologies used include AI models such as Gemini.

[0270] 4. Diagnostic Generation Methods

[0271] The AI ​​generates a predicted diagnosis and diagnostic evidence, which includes the diagnosis, diagnostic evidence, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[0272] 5. Reservation optimization measures

[0273] The server works with existing reservation systems to determine the optimal appointment time, taking into account the patient's desired date and time and available appointment times. For example, if a patient requests an appointment the following morning, the server checks availability during that time slot and determines the optimal appointment time.

[0274] 6. Means of notification

[0275] The patient's device is notified of the determined consultation date and time. The notification includes the specific date and time, and the patient can follow the instructions to receive the consultation.

[0276] 7. Online consultation methods

[0277] If the server recommends online diagnosis, it provides an option for a video consultation, in which the patient will be consulted by a doctor via video call at a specified time.

[0278] 8. Prompt Navigation Methods

[0279] The server helps the patient to input the information smoothly by guiding the user through the prompts. For example, the app displays the following prompts: "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," and "Please tell us the date and time you would like to make an appointment."

[0280] Hardware and software used

[0281] Hardware: Smartphone

[0282] Software: Python, Django (server side), REST API, AI tools (Gemini, NLP, computer vision)

[0283] Specific processing flow

[0284] Patients enter their symptoms (e.g., "I've had a sore throat for three days") and upload an image.

[0285] The server receives this, analyzes it using AI, and predicts the name of the disease.

[0286] Based on the provided diagnostic information, the server works with the reservation system to determine the optimal consultation date and time and notify the patient.

[0287] If necessary, we will offer the option of online consultations via video call to conduct consultations in real time.

[0288] In this way, patients can receive appropriate medical attention efficiently and quickly, improving the efficiency of the entire medical process.

[0289] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0290] Step 1:

[0291] The patient logs into the smartphone app.

[0292] Input: User ID and password.

[0293] Processing: The terminal acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of authentication success and session information to the terminal.

[0294] Output: A successful authentication message and session information.

[0295] Step 2:

[0296] Patients enter text information about their symptoms and upload image and video data.

[0297] Input: Detailed textual information about the symptom (e.g., "Sore throat lasting 3 days"), along with any associated image and video data, if needed.

[0298] Processing: The terminal obtains the entered text information and uploaded multimedia data and sends them to the server.

[0299] Output: Text information about the symptoms and multimedia data are sent to the server.

[0300] Step 3:

[0301] The server analyzes the received text information and image and video data.

[0302] Input: Textual information and multimedia data about symptoms.

[0303] Processing: The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology. At the same time, it analyzes image and video data using computer vision technology. From these analyses, it estimates the most likely diagnosis.

[0304] Output: Presumed disease name and diagnostic basis.

[0305] Step 4:

[0306] The server generates diagnostic information.

[0307] Input: Presumed disease name and diagnostic basis.

[0308] Processing: The server compiles the AI-generated diagnosis and its rationale into a single diagnostic report, which includes the diagnosis, rationale, and recommended next steps.

[0309] Output: Diagnostic information.

[0310] Step 5:

[0311] The server works in conjunction with the existing reservation system to determine the optimal appointment date and time.

[0312] Input: Diagnosis information, patient's desired date and time.

[0313] Processing: The server accesses the hospital's reservation system, checks the existing reservation status, and determines the optimal appointment date and time based on the patient's desired date and time and available appointment times.

[0314] Output: Best appointment date and time.

[0315] Step 6:

[0316] The server notifies the patient of the determined consultation date and time.

[0317] Input: Best appointment date and time.

[0318] Processing: The server uses the notification means to notify the patient's terminal of the determined consultation date and time. The notification includes the specific date and time.

[0319] Output: A notification message such as "The appointment date and time has been confirmed."

[0320] Step 7:

[0321] If the server recommends online diagnosis, it will offer the option of a video call consultation.

[0322] Input: Diagnostic information.

[0323] Processing: The server uses the online consultation method to notify the patient of the option to have a consultation via video call. If the patient selects online consultation, the consultation will be conducted via video call at the specified time.

[0324] Output: Notification of options for online consultation and conducting consultation via video call.

[0325] Step 8:

[0326] The server will announce the prompt.

[0327] Input: User symptom input status.

[0328] Processing: The server uses a prompting means to display prompt statements (e.g., "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," "Please tell us the date and time you would like to schedule an appointment") to assist the user in entering information.

[0329] Output: Displays prompts and assists the user in entering input smoothly.

[0330] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0331] The medical support system of the present invention allows patients to input their symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. Furthermore, by combining it with an emotion engine, further support based on the user's emotional state is possible.

[0332] 1. Initial Setup and Login

[0333] User Login

[0334] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[0335] 2. Enter your symptoms

[0336] Symptom text entry

[0337] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[0338] 3. Emotion Recognition by Emotion Engine

[0339] Data reception and emotion recognition

[0340] The server receives text information and multimodal data (images and videos) sent from the device. The emotion engine on the server analyzes the text information and image / video data to recognize the user's emotions. Based on the results of emotion recognition, the server can grasp the user's stress level and emotional state.

[0341] 4. Symptom analysis

[0342] Symptom analysis

[0343] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[0344] 5. Generating diagnostic results

[0345] Creating diagnostic information

[0346] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[0347] 6. Optimizing appointment scheduling

[0348] Reservation optimization based on emotional information

[0349] The server accesses the hospital's reservation system and checks the status of existing appointments. Based on the user's emotional information recognized by the emotion engine, the server determines the optimal appointment date and time to reduce the user's stress. The server then sends the determined appointment date and time to the terminal.

[0350] 7. Notifications and Online Diagnostics Options

[0351] Notification of appointment date and time

[0352] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[0353] 8. Start of consultation

[0354] User selection and consultation execution

[0355] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[0356] Specific examples

[0357] Symptom input and diagnosis

[0358] A user (for example, Hanako Sato) logs into the hospital's app and enters information about her sore throat. Sato enters the text "My sore throat has lasted for three days" and uploads an image of her throat. The device then sends the text information and image to the server.

[0359] Emotion recognition

[0360] The emotion engine in the server analyzes Sato's emotions based on text information and image data, and recognizes that his stress level is high.

[0361] Analyzing symptoms and generating diagnostic results

[0362] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reasons for it (text of symptoms, image analysis results, and sentiment analysis results).

[0363] Appointment optimization and notifications

[0364] The server refers to the hospital's reservation system and, taking into account that Mr. Sato has a high stress level and needs to be examined urgently, checks whether there are any openings in the morning of the following day. It determines that the optimal consultation time is 9:00 AM the following day and notifies the terminal. The terminal then notifies Mr. Sato, "Please come in for a consultation at 9:00 AM tomorrow." On the day, Mr. Sato visits the hospital at 9:00 AM and is examined with a short wait.

[0365] Online Diagnostics Example

[0366] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Mr. Sato of a suggestion of a video call as an "online diagnosis option." Mr. Sato selects the online diagnosis and receives a consultation via video call at the specified time.

[0367] This invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. Furthermore, by combining it with an emotion engine, it becomes possible to respond to patients' emotional states in a way that takes them into account, allowing for the provision of more optimal medical services.

[0368] The processing flow will be explained below.

[0369] Step 1:

[0370] The user accesses the hospital's app or website and enters their login information (user ID and password).

[0371] Step 2:

[0372] The terminal acquires the entered login information and sends it to the server.

[0373] Step 3:

[0374] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[0375] Step 4:

[0376] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[0377] Step 5:

[0378] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[0379] Step 6:

[0380] The terminal acquires the input text information and transmits it to the server.

[0381] Step 7:

[0382] Users take and upload images and videos related to their symptoms.

[0383] Step 8:

[0384] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[0385] Step 9:

[0386] The server receives the text information and image and video data sent from the terminal.

[0387] Step 10:

[0388] The emotion engine in the server analyzes text information and image / video data to recognize the user's emotions. The emotion recognition results are saved.

[0389] Step 11:

[0390] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology.

[0391] Step 12:

[0392] The AI ​​in the server uses computer vision technology to analyze image and video data.

[0393] Step 13:

[0394] The AI ​​on the server integrates text information, image and video information, and emotional information to predict the most likely diagnosis.

[0395] Step 14:

[0396] The server compiles the disease name generated by the AI ​​and its diagnostic basis (text analysis results, image analysis results, and emotion analysis results) into a single diagnostic information.

[0397] Step 15:

[0398] The server accesses the hospital's reservation system to check the status of existing reservations and the patient's preferred date and time.

[0399] Step 16:

[0400] The server determines the optimal consultation date and time to reduce the user's stress based on the user's emotional information recognized by the emotion engine.

[0401] Step 17:

[0402] The server sends the determined consultation date and time to the terminal and notifies the user.

[0403] Step 18:

[0404] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[0405] Step 19:

[0406] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[0407] Step 20:

[0408] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[0409] Step 21:

[0410] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[0411] Step 22:

[0412] Doctors can conduct efficient examinations based on the information provided.

[0413] Specific examples

[0414] Symptom input and diagnosis

[0415] Following the detailed procedure from step 1 to step 8, a user (e.g., Hanako Sato) logs into the hospital's app and enters information about her sore throat. She also takes an image of her throat and sends it to the server along with text information.

[0416] Emotion recognition

[0417] Following steps 9 and 10, the emotion engine in the server analyzes Sato's emotions based on the text information and image data, and recognizes that his stress level is high.

[0418] Analyzing symptoms and generating diagnostic results

[0419] Following steps 11 to 14, the AI ​​on the server analyzes the text information and image data and diagnoses a high probability of "pharyngitis." It then generates diagnostic information including the basis for the diagnosis.

[0420] Appointment optimization and notifications

[0421] Following the procedures from step 15 to step 18, the server determines that Mr. Sato, who has a high stress level, needs an urgent medical examination, confirms an appointment for 9:00 AM the next day, and notifies the terminal. The terminal then notifies the user, "Please come in for a medical examination at 9:00 AM tomorrow."

[0422] Proposal for online diagnosis

[0423] Following step 19, the diagnosis is determined to be "mild pharyngitis," and it is determined that online diagnosis is sufficient. The device notifies the user of a video call suggestion as an "online diagnosis option."

[0424] Conducting an examination

[0425] Following steps 20 to 22, Mr. Sato can either visit the hospital at the notified appointment date and time or select online diagnosis. The server provides the diagnosis information to the doctor, who then performs the examination efficiently.

[0426] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. In addition, by combining it with an emotion engine, it is possible to provide optimal medical services that take into account the emotional state of the patient.

[0427] Example 2

[0428] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0429] Conventional medical support systems predict illnesses and determine consultation dates and times based solely on symptom information entered by the patient, making it difficult to provide optimal consultations that take into account the patient's emotional state and stress level.In addition, there are many cases where online diagnosis candidates are not presented and medical support for doctors is not provided adequately, making it difficult to improve patient satisfaction.

[0430] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0431] In this invention, the server includes an input device for inputting text information about the patient's symptoms, a receiving device for receiving the input text information about the symptoms and image and video data, an emotion recognition device for analyzing the received data and recognizing emotions, an analysis device for analyzing the text and image and video data based on the recognized emotion information to predict the most likely disease name, a diagnosis generation device for generating a predicted disease name and its basis, an appointment optimization device for determining the optimal consultation date and time based on the predicted disease name, existing appointment status, and the patient's emotional state, and a notification device for notifying the patient of the determined consultation date and time. This enables the provision of optimal consultation dates and times that take the patient's emotional state into consideration, which is expected to improve patient satisfaction and the operational efficiency of medical institutions. It also enables the provision of online diagnosis options and medical support to doctors, creating a system that provides comprehensive medical support.

[0432] An "input means" is a device or program that provides an interface for a patient to input textual information or other data about their symptoms.

[0433] The "receiving means" is a device or program that receives text information and image or video data input through the input means.

[0434] The "emotion recognition means" is a device or program for analyzing the data received by the receiving means and recognizing the emotional state of the patient.

[0435] The "analysis means" is a device or program that analyzes text information and image and video data related to symptoms based on the emotional state recognized by the emotion recognition means, and predicts the most likely name of the disease.

[0436] The "diagnosis generating means" is a device or program that generates a diagnosis based on the disease name predicted by the analysis means and the basis for that diagnosis.

[0437] The "reservation optimization means" is a device or program that determines the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's emotional state.

[0438] The "notification means" is a device or program that notifies the patient of the consultation date and time determined by the appointment optimization means.

[0439] The "online diagnosis suggestion means" is a device or program that suggests online diagnosis options to a patient when it is determined that online diagnosis is possible.

[0440] The "diagnosis support means" is a device or program that provides a doctor with a predicted disease name and diagnostic basis, and supports medical treatment.

[0441] The medical support system of the present invention allows users to input their own symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the user. Furthermore, by combining it with an emotion engine, appropriate support can be achieved based on the user's emotional state.

[0442] The system includes the following main components:

[0443] 1. Input method:

[0444] This is an interface for users to enter text information about their symptoms. Examples include an input screen using a smartphone app or a web browser. Users can enter details such as "I have had a sore throat for three days" and can also upload images and videos if necessary.

[0445] 2. Receiving means:

[0446] A device or program that receives input text information and image / video data. Data sent from a smartphone app or web browser is received by a server and securely transferred using encryption technology such as SSL / TLS.

[0447] 3. Emotion recognition means:

[0448] This is a device or program that recognizes the user's emotional state based on the data received by the receiving means. Specifically, it uses IBM Watson Tone Analyzer or similar emotion recognition software. This allows the user's stress level and emotional state to be analyzed and stored in a database.

[0449] 4. Analysis method:

[0450] This is a device or program that analyzes symptom-related text information and image and video data based on emotional information recognized by an emotion recognition means. Specifically, it uses the Google Cloud Natural Language API and TensorFlow model to analyze symptoms and predict disease names using natural language processing (NLP) and computer vision technologies.

[0451] 5. Diagnostic generation methods:

[0452] This is a device or program that generates a diagnosis result based on the disease name predicted by the analysis means and the basis for that diagnosis. The server generates diagnostic information including the disease name, the basis for the diagnosis, and recommended next steps (e.g., whether a doctor's consultation is necessary or whether online diagnosis is possible).

[0453] 6. Reservation optimization measures:

[0454] This is a device or program that determines the optimal appointment time by taking into account the predicted diagnosis, existing appointments, and the patient's emotional state. It accesses the hospital's reservation system (e.g., appointment management software) and determines the optimal appointment time.

[0455] 7. Means of notification:

[0456] This is a device or program that notifies patients of the appointment date and time determined by the appointment optimization means. This notification is done via a smartphone app, email, SMS, etc., and notifies the user of the specific appointment date and time.

[0457] For example, consider the following prompt:

[0458] We will implement a system where users log in to the hospital's app, input text and image data of their symptoms, and submit it. Based on that data, emotion recognition and symptom analysis will be performed to determine the optimal appointment time. What program code and AI tools will be used?

[0459] This system allows users to smoothly go through a series of processes, from entering symptoms to scheduling an appointment, providing emotion-based diagnostic assistance, optimizing consultation dates and times, receiving notifications, and even presenting online diagnostic options. Furthermore, by incorporating an emotion engine, it is possible to respond in a way that takes into account the patient's emotional state, thereby improving the quality of medical services and patient satisfaction.

[0460] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0461] Step 1: User Login

[0462] The user accesses the hospital's app or website and enters their user ID and password.

[0463] (Input) User ID and password

[0464] The terminal takes the entered login information and sends it to the server.

[0465] The server checks the login information it receives against information in its database.

[0466] (Output) Login success or failure message

[0467] If the login is successful, the server generates a session ID and returns it to the terminal.

[0468] The terminal stores the session ID in session storage and displays a screen for the user to proceed to the next step.

[0469] Step 2: Enter your symptoms

[0470] The user enters detailed text about the symptom on the symptom entry screen, for example, "I've had a sore throat for three days."

[0471] (Input) Text information about symptoms

[0472] The device checks the text information entered and retrieves any images or videos uploaded.

[0473] (Output) Symptom text, images, video data

[0474] The device converts this data into JSON format and sends it to the server.

[0475] Step 3: Emotion Recognition

[0476] The server receives the text information, images, and video data sent from the terminal.

[0477] (Input) Text information, image data, video data

[0478] The server inputs the received data into an emotion engine (e.g., IBM Watson Tone Analyzer).

[0479] The emotion engine analyzes text, images, and videos to recognize the user's emotional state.

[0480] (Output) Emotion recognition result (e.g., stress level, high or low)

[0481] The server stores the emotion recognition results in a database and proceeds to the next analysis step.

[0482] Step 4: Symptom analysis

[0483] The AI ​​in the server (e.g., Google Cloud Natural Language API, TensorFlow model) analyzes text, image, and video data related to symptoms based on emotional information obtained through emotion recognition methods.

[0484] (Input) Emotion recognition results, text information, image data, video data

[0485] The AI ​​engine uses natural language processing (NLP) technology to extract keywords from text information and predict disease names, and computer vision technology to analyze image and video data.

[0486] (Output) Predicted disease name and its reasoning

[0487] The server aggregates the analysis results and compiles them into a single diagnostic information.

[0488] Step 5: Generate diagnostic information

[0489] The server generates diagnostic information including the disease name predicted by the AI ​​and its rationale.

[0490] (Input) Predicted disease name and evidence

[0491] (Output) Diagnostic information (disease name, diagnostic basis, recommended next steps)

[0492] The server stores the diagnostic information in a database and prepares it for the scheduled optimization process.

[0493] Step 6: Optimize appointments

[0494] The server accesses the hospital's reservation system and checks the current reservation status.

[0495] (Input) Diagnosis information, existing appointment status, patient sentiment information

[0496] The server considers the user's emotional information (e.g., high stress level) and determines the optimal appointment time.

[0497] (Output) Best appointment date and time

[0498] The server inputs the determined consultation date and time into the reservation management system and confirms the reservation.

[0499] (Output) Confirmed appointment date and time

[0500] Step 7: Notification of appointment date and time

[0501] The terminal notifies the user of the confirmed consultation date and time.

[0502] (Input) Confirmed consultation date and time

[0503] (Output) Notification of appointment date and time (e.g. tomorrow 11:00 AM)

[0504] The device will send you appointment date and time notifications via app, email, or SMS.

[0505] Step 8: Start the consultation

[0506] The user can either visit the hospital at the specified date and time or choose to have an online diagnosis.

[0507] (Input) Specified consultation date and time

[0508] The server provides the doctor with the necessary diagnostic information for the examination.

[0509] (Output) Diagnostic information (disease name, diagnostic basis)

[0510] Doctors can conduct efficient consultations based on the information provided.

[0511] In this way, users can smoothly navigate through a series of processes, from symptom entry to appointment scheduling, emotion-based diagnostic assistance, optimization of appointment dates and times, notifications, and appointment selection.

[0512] (Application example 2)

[0513] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0514] In modern self-driving vehicles, if a passenger suddenly becomes ill, prompt and appropriate medical attention is required. However, current self-driving vehicles do not have a system that can monitor the passenger's physical condition in real time and provide the most appropriate medical attention when necessary. Furthermore, there is no mechanism in place to automatically contact a medical institution or change the route quickly. A system that can respond to such situations is needed.

[0515] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: input means for the patient to input text information about symptoms; receiving means for receiving the text information about symptoms and image and video data input by the input means; analysis means for analyzing the text information and image and video data received by the receiving means and predicting a disease name; diagnosis generation means for generating the disease name predicted by the analysis means and its basis; appointment optimization means for determining the optimal consultation date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time; notification means for notifying the patient of the consultation date and time determined by the appointment optimization means; and vehicle route change means for inputting symptoms and related data and changing the route to the nearest medical institution when a patient complains of feeling unwell inside the autonomously driven vehicle. This enables prompt medical attention when a passenger complains of feeling unwell, and automatically changes the route of the autonomously driven vehicle and contacts the nearest medical institution as necessary.

[0516] "Input means" refers to a device or interface that provides a function for patients to input text information, images, and video data related to their symptoms.

[0517] The "receiving means" is a device or system that has the function of receiving text information and image and video data related to symptoms input by the input means.

[0518] The "analysis means" refers to a device or software that has the function of analyzing the text information and image or video data received by the receiving means and predicting the name of the disease.

[0519] The "diagnosis generating means" is a device or system that has the function of generating diagnostic information based on the disease name predicted by the analysis means and the basis for that name.

[0520] An "appointment optimization tool" is a device or software that has the function of determining the optimal appointment date and time by taking into account the predicted diagnosis, existing appointment status, and the patient's desired date and time.

[0521] The "notification means" is a device or interface that has a function for notifying the patient of the consultation date and time determined by the appointment optimization means.

[0522] A "vehicle route change means" is a device or system that has the function of inputting symptoms and related data when a patient complains of feeling unwell inside an autonomous vehicle and changing the route to the nearest medical institution.

[0523] A specific embodiment of the in-vehicle medical support system of the present invention is described below. This system automatically provides appropriate medical care when a patient complains of poor health. This allows the patient to receive the most appropriate medical service quickly.

[0524] First, the user (patient) logs into the system using a tablet, smart glasses, or smartphone in the vehicle. The login information is sent to the server, which checks it against a database and, if authentication is successful, generates session information, allowing the user to proceed to the next step.

[0525] If a user feels unwell, they can enter specific symptoms using a tablet or smart glasses. For example, they can enter text information such as "I've had a stomachache for two days," and can also upload images or videos related to the symptoms. This information is then sent to the server via a receiving device.

[0526] The server analyzes the text information and image / video data received by the receiving means, using an emotion engine to analyze the patient's emotional state and stress level. It also uses medical AI to predict the name of the disease, and based on the results, generates diagnostic information using a diagnosis generation means. The diagnostic information includes the predicted name of the disease, its rationale, and recommended next steps.

[0527] Next, the server determines the optimal appointment date and time using the appointment optimization means, taking into account the existing appointment status and the patient's stress level. The determined appointment date and time is notified to the user by the notification means. For example, a specific date and time such as "Please come in for an appointment at 10:00 AM tomorrow" is displayed.

[0528] If it is determined that medical treatment is necessary, the server will change the route of the autonomous vehicle to the nearest medical institution using the vehicle route change means. In the event of an emergency, the server will automatically contact the nearest hospital and urge them to take appropriate action.

[0529] As a concrete example, suppose a user in a self-driving vehicle inputs "I've had stomach pain for two days" and uploads an image of their abdomen. The emotion engine detects high stress, and MedicalAI determines that there is a high possibility of acute gastroenteritis. Based on this information, the server determines the optimal date and time for an appointment, notifies the user, and changes the self-driving vehicle's route to the nearest hospital.

[0530] Example prompt sentence:

[0531] Enter "I have had abdominal pain for two days and my lower abdomen is swollen," upload an image of your abdomen, and run a diagnosis.

[0532] As described above, the in-vehicle medical support system of the present invention automatically performs a series of processes to respond to a patient's poor physical condition, providing prompt and appropriate medical care.

[0533] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0534] Step 1:

[0535] The terminal obtains the user's login information (user ID and password) and sends it to the server. The server authenticates the received login information and, if successful, returns session information to the terminal. The input is the user's login information and the output is session information from the server. Specifically, the server references the database and checks whether there is any data that matches the input information.

[0536] Step 2:

[0537] The user uses a device to input text information about their symptoms and related image and video data. The input data is received by the device and sent to the server. The input is text information about symptoms and image and video data, and the output is data transmission to the server. Specifically, the user enters symptoms in the text box and attaches related data using the file upload function.

[0538] Step 3:

[0539] The server receives the data sent from the device and uses an emotion engine to analyze the user's emotional state and stress level. The input is text information about symptoms and image / video data, and the output is the emotion analysis results. Specifically, the server performs text analysis and image / video analysis to evaluate the user's emotional state.

[0540] Step 4:

[0541] Based on the data received by MedicalAI on the server, it analyzes symptoms and predicts the name of the disease. The input is the analyzed emotional state and symptom data, and the output is the predicted name of the disease and its reasons. Specifically, the AI ​​uses natural language processing and image analysis technology to comprehensively analyze the data.

[0542] Step 5:

[0543] The server generates diagnostic information based on the predicted disease name and its rationale. This diagnostic information includes the disease name, diagnostic rationale, and recommended next steps. The input is the disease name and its rationale, and the output is the diagnostic information. Specifically, the system compiles each piece of information into a text format.

[0544] Step 6:

[0545] The server uses a reservation optimization method to determine the optimal appointment date and time, taking into account existing reservations and the user's stress level. The input is the predicted illness, existing reservations, and stress level, and the output is the optimal appointment date and time. Specifically, the server accesses the reservation system, checks availability, and calculates the optimal date and time.

[0546] Step 7:

[0547] The server notifies the user of the determined consultation date and time using a notification means. The input is the optimal consultation date and time, and the output is a notification message to the user. Specifically, the message is sent to the terminal via the notification system and displayed on the user's screen.

[0548] Step 8:

[0549] When a user in an autonomous vehicle complains of feeling unwell, the server inputs the symptoms and related data and uses the vehicle route change means to change the route to the nearest medical institution. The input is the user's declaration of poor health and its detailed data, and the output is the route to the most suitable medical institution. Specifically, the server sets a new destination in the vehicle's navigation system and changes the driving route.

[0550] Example prompt sentence:

[0551] Enter "I have had abdominal pain for two days and my lower abdomen is swollen," upload an image of your abdomen, and run a diagnosis.

[0552] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0553] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0554] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0555] [Second embodiment]

[0556] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0557] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0558] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0559] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0560] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0561] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0562] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0563] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0564] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0565] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0566] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0567] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0568] The medical support system of the present invention allows patients to input their symptoms, and then uses artificial intelligence to perform highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. The elements of the system are as follows:

[0569] 1. Initial Setup and Login

[0570] User Login

[0571] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[0572] 2. Enter your symptoms

[0573] Symptom text entry

[0574] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[0575] 3. Symptom analysis

[0576] Receiving and analyzing data

[0577] The server receives text information and multimodal data (images and videos) sent from the device. The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and submits the most likely diagnosis.

[0578] 4. Generating diagnostic results

[0579] Creating diagnostic information

[0580] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[0581] 5. Optimizing appointment scheduling

[0582] Checking reservation status and deciding on consultation date and time

[0583] The server accesses the hospital's reservation system and checks the current reservation status. The server determines the optimal appointment date and time based on the user's desired date and time and available appointment times. The server then sends the determined appointment date and time to the terminal.

[0584] 6. Notifications and Online Diagnostics Options

[0585] Notification of appointment date and time

[0586] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[0587] 7. Start of consultation

[0588] User selection and consultation execution

[0589] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[0590] Specific examples

[0591] Symptom input and diagnosis

[0592] A user (say, Ichiro Tanaka) logs into the hospital's app and enters information about his sore throat. Tanaka enters the text "My sore throat has lasted for three days" and uploads an image of his throat. The device then sends the text information and image to the server.

[0593] Symptom analysis

[0594] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reason for this (text of symptoms, image analysis results).

[0595] Appointment optimization and notifications

[0596] The server references the hospital's reservation system to check whether there is an opening in the morning of the following day, which is Tanaka's desired time. It determines that the optimal consultation time is 11:00 AM the following day and notifies the terminal. The terminal then notifies Tanaka, "Please come in for a consultation at 11:00 AM tomorrow." On the day, Tanaka visits the hospital at 11:00 AM and is examined with a short wait.

[0597] Online Diagnostics Example

[0598] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Tanaka of the suggestion of a video call as an "online diagnosis option." Tanaka selects the online diagnosis and receives a consultation via video call at the specified time.

[0599] The present invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.

[0600] The processing flow will be explained below.

[0601] Step 1:

[0602] The user accesses the hospital's app or website and enters their login information (user ID and password).

[0603] Step 2:

[0604] The terminal acquires the entered login information and sends it to the server.

[0605] Step 3:

[0606] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[0607] Step 4:

[0608] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[0609] Step 5:

[0610] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[0611] Step 6:

[0612] The terminal acquires the input text information and transmits it to the server.

[0613] Step 7:

[0614] If the user wishes, they can upload images or videos related to their symptoms.

[0615] Step 8:

[0616] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[0617] Step 9:

[0618] The server receives text information and multimodal data (images and videos) sent from the device.

[0619] Step 10:

[0620] The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology.

[0621] Step 11:

[0622] The AI ​​in the server uses computer vision technology to analyze image and video data.

[0623] Step 12:

[0624] The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[0625] Step 13:

[0626] The server compiles the disease name generated by the AI ​​and the basis for the diagnosis into a single piece of diagnostic information.

[0627] Step 14:

[0628] The server accesses the hospital's reservation system and checks the status of existing reservations.

[0629] Step 15:

[0630] The server determines the optimal consultation date and time taking into consideration the user's desired date and time and available appointment times.

[0631] Step 16:

[0632] The server transmits the determined consultation date and time to the terminal.

[0633] Step 17:

[0634] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[0635] Step 18:

[0636] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[0637] Step 19:

[0638] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[0639] Step 20:

[0640] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[0641] Step 21:

[0642] Doctors can conduct efficient examinations based on the information provided.

[0643] Example 1

[0644] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] In modern healthcare, it is important for patients to quickly and accurately communicate their symptoms to medical institutions, but this has been difficult with conventional systems. Long wait times before seeing a doctor are also a factor in lower patient satisfaction. Furthermore, appointment scheduling systems are inefficient, often preventing patients from scheduling appointments at their desired dates and times. Therefore, there is a need for a system that can efficiently analyze patients' symptoms and determine the optimal appointment date and time, thereby improving the operational efficiency of medical institutions and increasing patient satisfaction.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0647] In this invention, the server includes an input means for the patient to input text information about symptoms, a receiving means for receiving the text information about symptoms and image and video data input by the input means, an analysis means for analyzing the text information and image and video data received by the receiving means to predict a disease name, a diagnosis generation means for generating the disease name predicted by the analysis means and its basis and compiling it into diagnostic information, an appointment optimization means for accessing the medical institution's appointment system and determining the optimal appointment date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time, and a notification means for notifying the patient of the appointment date and time determined by the appointment optimization means. This makes it possible to efficiently analyze the patient's symptoms to assist in diagnosis and quickly determine the optimal appointment date and time.

[0648] "Patient" refers to a person who receives medical examination or treatment from a medical institution.

[0649] "Text information about symptoms" refers to written information in which a patient describes in detail their symptoms or discomfort.

[0650] "Input means" refers to a device or application function that allows a patient to input text information about their symptoms.

[0651] The "receiving means" refers to a mechanism that acquires text information and image or video data input by the input means and transmits it to the server.

[0652] "Analysis means" refers to an algorithm or model that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease from the symptoms.

[0653] The "diagnosis generating means" refers to a mechanism that generates the disease name predicted by the analysis means and its basis, and compiles it as diagnostic information.

[0654] The "reservation optimization means" refers to a mechanism for accessing a medical institution's reservation system and determining the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[0655] The "notification means" refers to a mechanism that notifies the patient of the consultation date and time determined by the appointment optimization means.

[0656] The "online diagnosis suggestion means" refers to a mechanism for suggesting the option of online diagnosis to a patient when it is determined that online diagnosis is possible.

[0657] "Medical support tools" refer to mechanisms that provide medical professionals with predicted disease names and diagnostic evidence, thereby improving the efficiency of medical treatment.

[0658] "Medical institution" refers to a facility such as a hospital, clinic, or doctor's office that provides medical examinations and treatment.

[0659] MODE FOR CARRYING OUT THE INVENTION

[0660] The medical support system of the present invention is a system in which patients input their symptoms, and based on that, artificial intelligence performs highly accurate analysis to assist in diagnosis, and determines and notifies the patient of the optimal consultation date and time. Each element of this system will be described in detail.

[0661] Initial Setup and Login

[0662] Login Process

[0663] A user accesses a hospital's app or website and enters their user ID and password into the displayed login form. The device sends this information to the server via a secure protocol (e.g., HTTPS). The server receives the login information and authenticates it by checking it against the user information in its database. If authentication is successful, the server generates a message indicating successful authentication and session information and returns it to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[0664] Enter symptoms

[0665] Enter symptoms and submit data

[0666] The user enters detailed text about their symptoms, such as "I've had a sore throat for three days." The device collects this text information and uploads images and videos as needed. All information, including uploaded image and video data, is then sent to the server.

[0667] Symptom analysis

[0668] Data analysis

[0669] The server uses AI (for example, the Gemini model, which uses NLP technology) to analyze the received text information and image and video data. The text information is analyzed using natural language processing (NLP) technology to extract symptom characteristics and correlations. The image and video data is also analyzed using computer vision technology to recognize visual symptom characteristics. The AI ​​on the server integrates this information and suggests the most likely diagnosis.

[0670] Generating diagnostic results

[0671] Creating diagnostic information

[0672] The server compiles the AI-generated disease names and the diagnostic evidence, building diagnostic information that is saved in a format that is easy for patients to refer to later.

[0673] Optimizing medical appointments

[0674] Checking reservation status and deciding on consultation date and time

[0675] The server accesses the hospital's reservation system to check the current reservation status. Then, it determines the optimal appointment date and time based on the user's desired date and time and the available appointment times. It then sends the determined appointment date and time to the terminal.

[0676] Choosing between notifications and online diagnostics

[0677] Notification of appointment date and time and online diagnosis suggestions

[0678] The device notifies the user of the consultation date and time received from the server and displays the specific date and time. In some cases, confirmations and preparations for the consultation are displayed. If the server determines that an online diagnosis is sufficient, the device offers the option of an online diagnosis. If the user selects an online diagnosis, the device displays a link to a video call or provides instructions for the corresponding app.

[0679] Start of consultation

[0680] Conducting an examination

[0681] The user can either visit the hospital at the notified appointment date and time or choose online diagnosis. The server provides medical professionals with diagnostic information (disease name and diagnostic basis) to support them in providing efficient medical care.

[0682] Specific examples

[0683] Specific examples of symptom input and diagnosis

[0684] A user (e.g., a patient) logs into a hospital app and enters details about a sore throat. The user enters the text "I have had a sore throat for three days" and uploads an image of their throat. The device then sends this information to the server.

[0685] Specific examples of symptom analysis

[0686] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that the condition is likely to be "pharyngitis." The server then generates "pharyngitis" and the basis for this (text of symptoms and image analysis results) as diagnostic information.

[0687] Specific examples of appointments and notifications

[0688] The server checks the hospital's reservation system and confirms that there is availability for the morning of the following day, as desired by the patient. It determines that the optimal appointment time is 11:00 AM the following day and notifies the terminal of this. The terminal then notifies the patient, "Please come in for an appointment at 11:00 AM tomorrow." On the day of the appointment, the patient can visit the hospital at 11:00 AM and be seen with a short waiting time.

[0689] Examples of online diagnostics

[0690] The diagnosis results in "mild pharyngitis," and the server determines that online diagnosis is sufficient. The device notifies the patient of the "online diagnosis option" and suggests a video call. The patient selects online diagnosis and receives a consultation via video call at the specified time.

[0691] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.The following is also used as an example of a prompt sentence to input into the generative AI model:

[0692] "I've had a sore throat for three days. Please see the image below. What is the name of the disease that corresponds to this symptom?"

[0693] The above is a specific embodiment of the present invention.

[0694] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0695] Step 1:

[0696] Login Process

[0697] input:

[0698] The user accesses the hospital's app or website and enters their user ID and password into the login form that appears on the screen.

[0699] Operation:

[0700] The device sends this information to the server via a secure protocol (e.g., HTTPS).

[0701] output:

[0702] The server receives the login information, compares it with the user information in the database, and determines whether authentication is successful. If successful, it generates a message indicating successful authentication and session information, and returns them to the terminal.

[0703] Specific behavior:

[0704] The server queries the database to retrieve user information.

[0705] If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0706] The device will save the session ID and display a screen to proceed to the next step.

[0707] Step 2:

[0708] Enter symptoms and submit data

[0709] input:

[0710] Users enter detailed text about their symptoms and, if they wish, upload images or videos.

[0711] Operation:

[0712] The device acquires the entered text information and sends it to the server along with image and video data.

[0713] output:

[0714] The server receives the text information and image and video data.

[0715] Specific behavior:

[0716] The device temporarily stores the text information in its storage.

[0717] When a user uploads an image or video, the device checks the file format and size.

[0718] The device sends text, image, and video data to the server.

[0719] Step 3:

[0720] Data analysis

[0721] input:

[0722] Text information and image or video data received by the server.

[0723] Operation:

[0724] The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology to extract symptom characteristics and correlations, while image and video data is analyzed using computer vision technology.

[0725] output:

[0726] The AI ​​will suggest the most likely diagnosis based on the integrated information.

[0727] Specific behavior:

[0728] The server performs text analysis and extracts keywords and phrases.

[0729] Recognizing visual symptoms (e.g. redness or swelling of the throat) through image and video analysis.

[0730] Integrates NLP and computer vision results to implement AI algorithms for disease prediction.

[0731] Step 4:

[0732] Creating diagnostic information

[0733] input:

[0734] The disease name proposed through analysis and the supporting data.

[0735] Operation:

[0736] The server compiles the predicted disease name and the basis for the diagnosis as diagnostic information.

[0737] output:

[0738] Diagnostic information is generated and saved in a format that can be later referenced by the user.

[0739] Specific behavior:

[0740] The server stores the diagnostic information (disease name, reason) in a database as structured data.

[0741] -Include user-friendly explanations for diagnostic information.

[0742] Step 5:

[0743] Checking reservation status and deciding on consultation date and time

[0744] input:

[0745] The user's desired date and time and medical institution reservation status.

[0746] Operation:

[0747] The server accesses the hospital's reservation system to check the existing reservation status, and then determines the optimal appointment date and time based on the user's desired date and time.

[0748] output:

[0749] The determined consultation date and time is generated and transmitted to the terminal.

[0750] Specific behavior:

[0751] The server sends a query to the hospital's booking system through an API.

[0752] The server selects the optimal time based on the user's desired date and time and available reservation times.

[0753] The determined date and time is saved as session information and sent to the device.

[0754] Step 6:

[0755] Notification of appointment date and time and online diagnosis suggestions

[0756] input:

[0757] Decided appointment date and time and decision on online diagnosis.

[0758] Operation:

[0759] The terminal notifies the user of the consultation date and time received from the server and displays the specific date and time. If the server suggests online diagnosis, the terminal notifies the user of the online diagnosis option.

[0760] output:

[0761] The user is notified of appointment times and online diagnostic options.

[0762] Specific behavior:

[0763] The device will notify you of the appointment date and time and display a message on the screen saying, "Please come in for an appointment tomorrow at 11:00 AM."

[0764] -You will be presented with the option to do an online diagnostic and provided with a link if you wish.

[0765] Step 7:

[0766] Conducting an examination

[0767] input:

[0768] User-selected consultation date and time or online diagnosis

[0769] Operation:

[0770] The user visits the hospital at the designated appointment date and time, and the server provides the diagnosis information to the medical professional. In the case of online diagnosis, the consultation is conducted via video call.

[0771] output:

[0772] The examination is carried out efficiently.

[0773] Specific behavior:

[0774] The server provides medical professionals with diagnostic information (disease name and diagnostic basis).

[0775] -Medical professionals will examine the user based on the diagnostic information and provide appropriate treatment.

[0776] The above is the flow of processing of the program of this system.

[0777] (Application example 1)

[0778] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0779] In the modern healthcare system, the time and effort required for patients to access medical institutions and receive diagnosis and treatment is a problem. Furthermore, to receive an accurate diagnosis, a large amount of medical information must be provided to doctors, and efficient methods for doing this are needed. In particular, with the widespread use of online diagnosis, real-time diagnostic support and optimization of consultation appointments are important challenges.

[0780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0781] In this invention, the server includes an input means for patients to input text information about their symptoms, a receiving means, an analyzing means, a diagnosis generating means, a reservation optimization means, a notification means, an online phone consultation means, and a prompting means. This allows patients to receive appropriate consultations efficiently and quickly. Furthermore, if an online consultation is recommended, real-time consultations can be conducted via video calls, improving the efficiency of the entire medical process. Furthermore, by guiding patients through the input of prompts, the patient can smoothly input information, which helps support accurate diagnoses.

[0782] "Input means" refers to a device or function that allows a patient to input text information about symptoms.

[0783] The "receiving means" is a device or function that receives text information and image and video data related to symptoms input by the input means.

[0784] The "analysis means" is a device or function that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease.

[0785] The "diagnosis generating means" is a device or function that generates the name of the disease predicted by the analysis means and the basis for that name.

[0786] The "reservation optimization means" is a device or function that determines the optimal consultation date and time taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[0787] The "notification means" is a device or function that notifies the patient of the consultation date and time determined by the reservation optimization means.

[0788] "Online call consultation means" refers to a device or function that conducts consultation via video call when online diagnosis is recommended.

[0789] The "prompt guide means" is a device or function that guides the user to input a prompt sentence.

[0790] This invention relates to a system in which patients input their symptoms, and artificial intelligence (AI) performs highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. This system includes the following elements.

[0791] System Overview

[0792] 1. Input Method

[0793] Patients use a smartphone app to enter text information about their symptoms, such as "I've had a sore throat for three days," and upload images and video data related to their symptoms if necessary.

[0794] 2. Receiving Method

[0795] The server receives the text information and image and video data input by the input means.

[0796] 3. Analysis method

[0797] The server analyzes the received text information using natural language processing (NLP) technology, and image and video data using computer vision technology. Based on this analysis, the AI ​​predicts the most likely diagnosis based on the patient's symptoms. Examples of AI technologies used include AI models such as Gemini.

[0798] 4. Diagnostic Generation Methods

[0799] The AI ​​generates a predicted diagnosis and diagnostic evidence, which includes the diagnosis, diagnostic evidence, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[0800] 5. Reservation optimization measures

[0801] The server works with existing reservation systems to determine the optimal appointment time, taking into account the patient's desired date and time and available appointment times. For example, if a patient requests an appointment the following morning, the server checks availability during that time slot and determines the optimal appointment time.

[0802] 6. Means of notification

[0803] The patient's device is notified of the determined consultation date and time. The notification includes the specific date and time, and the patient can follow the instructions to receive the consultation.

[0804] 7. Online consultation methods

[0805] If the server recommends online diagnosis, it provides an option for a video consultation, in which the patient will be consulted by a doctor via video call at a specified time.

[0806] 8. Prompt Navigation Methods

[0807] The server helps the patient to input the information smoothly by guiding the user through the prompts. For example, the app displays the following prompts: "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," and "Please tell us the date and time you would like to make an appointment."

[0808] Hardware and software used

[0809] Hardware: Smartphone

[0810] Software: Python, Django (server side), REST API, AI tools (Gemini, NLP, computer vision)

[0811] Specific processing flow

[0812] Patients enter their symptoms (e.g., "I've had a sore throat for three days") and upload an image.

[0813] The server receives this, analyzes it using AI, and predicts the name of the disease.

[0814] Based on the provided diagnostic information, the server works with the reservation system to determine the optimal consultation date and time and notify the patient.

[0815] If necessary, we will offer the option of online consultations via video call to conduct consultations in real time.

[0816] In this way, patients can receive appropriate medical attention efficiently and quickly, improving the efficiency of the entire medical process.

[0817] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0818] Step 1:

[0819] The patient logs into the smartphone app.

[0820] Input: User ID and password.

[0821] Processing: The terminal acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of authentication success and session information to the terminal.

[0822] Output: A successful authentication message and session information.

[0823] Step 2:

[0824] Patients enter text information about their symptoms and upload image and video data.

[0825] Input: Detailed textual information about the symptom (e.g., "Sore throat lasting 3 days"), along with any associated image and video data, if needed.

[0826] Processing: The terminal obtains the entered text information and uploaded multimedia data and sends them to the server.

[0827] Output: Text information about the symptoms and multimedia data are sent to the server.

[0828] Step 3:

[0829] The server analyzes the received text information and image and video data.

[0830] Input: Textual information and multimedia data about symptoms.

[0831] Processing: The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology. At the same time, it analyzes image and video data using computer vision technology. From these analyses, it estimates the most likely diagnosis.

[0832] Output: Presumed disease name and diagnostic basis.

[0833] Step 4:

[0834] The server generates diagnostic information.

[0835] Input: Presumed disease name and diagnostic basis.

[0836] Processing: The server compiles the AI-generated diagnosis and its rationale into a single diagnostic report, which includes the diagnosis, rationale, and recommended next steps.

[0837] Output: Diagnostic information.

[0838] Step 5:

[0839] The server works in conjunction with the existing reservation system to determine the optimal appointment date and time.

[0840] Input: Diagnosis information, patient's desired date and time.

[0841] Processing: The server accesses the hospital's reservation system, checks the existing reservation status, and determines the optimal appointment date and time based on the patient's desired date and time and available appointment times.

[0842] Output: Best appointment date and time.

[0843] Step 6:

[0844] The server notifies the patient of the determined consultation date and time.

[0845] Input: Best appointment date and time.

[0846] Processing: The server uses the notification means to notify the patient's terminal of the determined consultation date and time. The notification includes the specific date and time.

[0847] Output: A notification message such as "The appointment date and time has been confirmed."

[0848] Step 7:

[0849] If the server recommends online diagnosis, it will offer the option of a video call consultation.

[0850] Input: Diagnostic information.

[0851] Processing: The server uses the online consultation method to notify the patient of the option to have a consultation via video call. If the patient selects online consultation, the consultation will be conducted via video call at the specified time.

[0852] Output: Notification of options for online consultation and conducting consultation via video call.

[0853] Step 8:

[0854] The server will announce the prompt.

[0855] Input: User symptom input status.

[0856] Processing: The server uses a prompting means to display prompt statements (e.g., "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," "Please tell us the date and time you would like to schedule an appointment") to assist the user in entering information.

[0857] Output: Displays prompts and assists the user in entering input smoothly.

[0858] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0859] The medical support system of the present invention allows patients to input their symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. Furthermore, by combining it with an emotion engine, further support based on the user's emotional state is possible.

[0860] 1. Initial Setup and Login

[0861] User Login

[0862] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[0863] 2. Enter your symptoms

[0864] Symptom text entry

[0865] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[0866] 3. Emotion Recognition by Emotion Engine

[0867] Data reception and emotion recognition

[0868] The server receives text information and multimodal data (images and videos) sent from the device. The emotion engine on the server analyzes the text information and image / video data to recognize the user's emotions. Based on the results of emotion recognition, the server can grasp the user's stress level and emotional state.

[0869] 4. Symptom analysis

[0870] Symptom analysis

[0871] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[0872] 5. Generating diagnostic results

[0873] Creating diagnostic information

[0874] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[0875] 6. Optimizing appointment scheduling

[0876] Reservation optimization based on emotional information

[0877] The server accesses the hospital's reservation system and checks the status of existing appointments. Based on the user's emotional information recognized by the emotion engine, the server determines the optimal appointment date and time to reduce the user's stress. The server then sends the determined appointment date and time to the terminal.

[0878] 7. Notifications and Online Diagnostics Options

[0879] Notification of appointment date and time

[0880] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[0881] 8. Start of consultation

[0882] User selection and consultation execution

[0883] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[0884] Specific examples

[0885] Symptom input and diagnosis

[0886] A user (for example, Hanako Sato) logs into the hospital's app and enters information about her sore throat. Sato enters the text "My sore throat has lasted for three days" and uploads an image of her throat. The device then sends the text information and image to the server.

[0887] Emotion recognition

[0888] The emotion engine in the server analyzes Sato's emotions based on text information and image data, and recognizes that his stress level is high.

[0889] Analyzing symptoms and generating diagnostic results

[0890] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reasons for it (text of symptoms, image analysis results, and sentiment analysis results).

[0891] Appointment optimization and notifications

[0892] The server refers to the hospital's reservation system and, taking into account that Mr. Sato has a high stress level and needs to be examined urgently, checks whether there are any openings in the morning of the following day. It determines that the optimal consultation time is 9:00 AM the following day and notifies the terminal. The terminal then notifies Mr. Sato, "Please come in for a consultation at 9:00 AM tomorrow." On the day, Mr. Sato visits the hospital at 9:00 AM and is examined with a short wait.

[0893] Online Diagnostics Example

[0894] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Mr. Sato of a suggestion of a video call as an "online diagnosis option." Mr. Sato selects the online diagnosis and receives a consultation via video call at the specified time.

[0895] This invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. Furthermore, by combining it with an emotion engine, it becomes possible to respond to patients' emotional states in a way that takes them into account, allowing for the provision of more optimal medical services.

[0896] The processing flow will be explained below.

[0897] Step 1:

[0898] The user accesses the hospital's app or website and enters their login information (user ID and password).

[0899] Step 2:

[0900] The terminal acquires the entered login information and sends it to the server.

[0901] Step 3:

[0902] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[0903] Step 4:

[0904] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[0905] Step 5:

[0906] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[0907] Step 6:

[0908] The terminal acquires the input text information and transmits it to the server.

[0909] Step 7:

[0910] Users take and upload images and videos related to their symptoms.

[0911] Step 8:

[0912] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[0913] Step 9:

[0914] The server receives the text information and image and video data sent from the terminal.

[0915] Step 10:

[0916] The emotion engine in the server analyzes text information and image / video data to recognize the user's emotions. The emotion recognition results are saved.

[0917] Step 11:

[0918] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology.

[0919] Step 12:

[0920] The AI ​​in the server uses computer vision technology to analyze image and video data.

[0921] Step 13:

[0922] The AI ​​on the server integrates text information, image and video information, and emotional information to predict the most likely diagnosis.

[0923] Step 14:

[0924] The server compiles the disease name generated by the AI ​​and its diagnostic basis (text analysis results, image analysis results, and emotion analysis results) into a single diagnostic information.

[0925] Step 15:

[0926] The server accesses the hospital's reservation system to check the status of existing reservations and the patient's preferred date and time.

[0927] Step 16:

[0928] The server determines the optimal consultation date and time to reduce the user's stress based on the user's emotional information recognized by the emotion engine.

[0929] Step 17:

[0930] The server sends the determined consultation date and time to the terminal and notifies the user.

[0931] Step 18:

[0932] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[0933] Step 19:

[0934] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[0935] Step 20:

[0936] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[0937] Step 21:

[0938] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[0939] Step 22:

[0940] Doctors can conduct efficient examinations based on the information provided.

[0941] Specific examples

[0942] Symptom input and diagnosis

[0943] Following the detailed procedure from step 1 to step 8, a user (e.g., Hanako Sato) logs into the hospital's app and enters information about her sore throat. She also takes an image of her throat and sends it to the server along with text information.

[0944] Emotion recognition

[0945] Following steps 9 and 10, the emotion engine in the server analyzes Sato's emotions based on the text information and image data, and recognizes that his stress level is high.

[0946] Analyzing symptoms and generating diagnostic results

[0947] Following steps 11 to 14, the AI ​​on the server analyzes the text information and image data and diagnoses a high probability of "pharyngitis." It then generates diagnostic information including the basis for the diagnosis.

[0948] Appointment optimization and notifications

[0949] Following the procedures from step 15 to step 18, the server determines that Mr. Sato, who has a high stress level, needs an urgent medical examination, confirms an appointment for 9:00 AM the next day, and notifies the terminal. The terminal then notifies the user, "Please come in for a medical examination at 9:00 AM tomorrow."

[0950] Proposal for online diagnosis

[0951] Following step 19, the diagnosis is determined to be "mild pharyngitis," and it is determined that online diagnosis is sufficient. The device notifies the user of a video call suggestion as an "online diagnosis option."

[0952] Conducting an examination

[0953] Following steps 20 to 22, Mr. Sato can either visit the hospital at the notified appointment date and time or select online diagnosis. The server provides the diagnosis information to the doctor, who then performs the examination efficiently.

[0954] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. In addition, by combining it with an emotion engine, it is possible to provide optimal medical services that take into account the emotional state of the patient.

[0955] Example 2

[0956] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0957] Conventional medical support systems predict illnesses and determine consultation dates and times based solely on symptom information entered by the patient, making it difficult to provide optimal consultations that take into account the patient's emotional state and stress level.In addition, there are many cases where online diagnosis candidates are not presented and medical support for doctors is not provided adequately, making it difficult to improve patient satisfaction.

[0958] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0959] In this invention, the server includes an input device for inputting text information about the patient's symptoms, a receiving device for receiving the input text information about the symptoms and image and video data, an emotion recognition device for analyzing the received data and recognizing emotions, an analysis device for analyzing the text and image and video data based on the recognized emotion information to predict the most likely disease name, a diagnosis generation device for generating a predicted disease name and its basis, an appointment optimization device for determining the optimal consultation date and time based on the predicted disease name, existing appointment status, and the patient's emotional state, and a notification device for notifying the patient of the determined consultation date and time. This enables the provision of optimal consultation dates and times that take the patient's emotional state into consideration, which is expected to improve patient satisfaction and the operational efficiency of medical institutions. It also enables the provision of online diagnosis options and medical support to doctors, creating a system that provides comprehensive medical support.

[0960] An "input means" is a device or program that provides an interface for a patient to input textual information or other data about their symptoms.

[0961] The "receiving means" is a device or program that receives text information and image or video data input through the input means.

[0962] The "emotion recognition means" is a device or program for analyzing the data received by the receiving means and recognizing the emotional state of the patient.

[0963] The "analysis means" is a device or program that analyzes text information and image and video data related to symptoms based on the emotional state recognized by the emotion recognition means, and predicts the most likely name of the disease.

[0964] The "diagnosis generating means" is a device or program that generates a diagnosis based on the disease name predicted by the analysis means and the basis for that diagnosis.

[0965] The "reservation optimization means" is a device or program that determines the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's emotional state.

[0966] The "notification means" is a device or program that notifies the patient of the consultation date and time determined by the appointment optimization means.

[0967] The "online diagnosis suggestion means" is a device or program that suggests online diagnosis options to a patient when it is determined that online diagnosis is possible.

[0968] The "diagnosis support means" is a device or program that provides a doctor with a predicted disease name and diagnostic basis, and supports medical treatment.

[0969] The medical support system of the present invention allows users to input their own symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the user. Furthermore, by combining it with an emotion engine, appropriate support can be achieved based on the user's emotional state.

[0970] The system includes the following main components:

[0971] 1. Input method:

[0972] This is an interface for users to enter text information about their symptoms. Examples include an input screen using a smartphone app or a web browser. Users can enter details such as "I have had a sore throat for three days" and can also upload images and videos if necessary.

[0973] 2. Receiving means:

[0974] A device or program that receives input text information and image / video data. Data sent from a smartphone app or web browser is received by a server and securely transferred using encryption technology such as SSL / TLS.

[0975] 3. Emotion recognition means:

[0976] This is a device or program that recognizes the user's emotional state based on the data received by the receiving means. Specifically, it uses IBM Watson Tone Analyzer or similar emotion recognition software. This allows the user's stress level and emotional state to be analyzed and stored in a database.

[0977] 4. Analysis method:

[0978] This is a device or program that analyzes symptom-related text information and image and video data based on emotional information recognized by an emotion recognition means. Specifically, it uses the Google Cloud Natural Language API and TensorFlow model to analyze symptoms and predict disease names using natural language processing (NLP) and computer vision technologies.

[0979] 5. Diagnostic generation methods:

[0980] This is a device or program that generates a diagnosis result based on the disease name predicted by the analysis means and the basis for that diagnosis. The server generates diagnostic information including the disease name, the basis for the diagnosis, and recommended next steps (e.g., whether a doctor's consultation is necessary or whether online diagnosis is possible).

[0981] 6. Reservation optimization measures:

[0982] This is a device or program that determines the optimal appointment time by taking into account the predicted diagnosis, existing appointments, and the patient's emotional state. It accesses the hospital's reservation system (e.g., reservation management software) and determines the optimal appointment time.

[0983] 7. Means of notification:

[0984] This is a device or program that notifies patients of the appointment date and time determined by the appointment optimization means. This notification is made via a smartphone app, email, SMS, etc., and informs the user of the specific appointment date and time.

[0985] For example, consider the following prompt:

[0986] We will implement a system where users log in to the hospital's app, input text and image data of their symptoms, and submit it. Based on that data, emotion recognition and symptom analysis will be performed to determine the optimal appointment time. What program code and AI tools will be used?

[0987] This system allows users to smoothly go through a series of processes, from entering symptoms to scheduling an appointment, providing emotion-based diagnostic assistance, optimizing consultation dates and times, receiving notifications, and even presenting online diagnostic options. Furthermore, by incorporating an emotion engine, it is possible to respond in a way that takes into account the patient's emotional state, thereby improving the quality of medical services and patient satisfaction.

[0988] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0989] Step 1: User Login

[0990] The user accesses the hospital's app or website and enters their user ID and password.

[0991] (Input) User ID and password

[0992] The terminal takes the entered login information and sends it to the server.

[0993] The server checks the login information it receives against information in its database.

[0994] (Output) Login success or failure message

[0995] If the login is successful, the server generates a session ID and returns it to the terminal.

[0996] The terminal stores the session ID in session storage and displays a screen for the user to proceed to the next step.

[0997] Step 2: Enter your symptoms

[0998] The user enters detailed text about the symptom on the symptom entry screen, for example, "I've had a sore throat for three days."

[0999] (Input) Text information about symptoms

[1000] The device checks the text information entered and retrieves any images or videos uploaded.

[1001] (Output) Symptom text, images, video data

[1002] The device converts this data into JSON format and sends it to the server.

[1003] Step 3: Emotion Recognition

[1004] The server receives the text information, images, and video data sent from the terminal.

[1005] (Input) Text information, image data, video data

[1006] The server inputs the received data into an emotion engine (e.g., IBM Watson Tone Analyzer).

[1007] The emotion engine analyzes text, images, and videos to recognize the user's emotional state.

[1008] (Output) Emotion recognition result (e.g., stress level, high or low)

[1009] The server stores the emotion recognition results in a database and proceeds to the next analysis step.

[1010] Step 4: Symptom analysis

[1011] The AI ​​in the server (e.g., Google Cloud Natural Language API, TensorFlow model) analyzes text, image, and video data related to symptoms based on emotional information obtained through emotion recognition methods.

[1012] (Input) Emotion recognition results, text information, image data, video data

[1013] The AI ​​engine uses natural language processing (NLP) technology to extract keywords from text information and predict disease diagnoses, and computer vision technology to analyze image and video data.

[1014] (Output) Predicted disease name and its reasoning

[1015] The server aggregates the analysis results and compiles them into a single diagnostic information.

[1016] Step 5: Generate diagnostic information

[1017] The server generates diagnostic information including the disease name predicted by the AI ​​and its rationale.

[1018] (Input) Predicted disease name and evidence

[1019] (Output) Diagnostic information (disease name, diagnostic basis, recommended next steps)

[1020] The server stores the diagnostic information in a database and prepares it for the scheduled optimization process.

[1021] Step 6: Optimize appointments

[1022] The server accesses the hospital's reservation system and checks the current reservation status.

[1023] (Input) Diagnosis information, existing appointment status, patient sentiment information

[1024] The server considers the user's emotional information (e.g., high stress level) and determines the optimal appointment time.

[1025] (Output) Best appointment date and time

[1026] The server inputs the determined consultation date and time into the reservation management system and confirms the reservation.

[1027] (Output) Confirmed appointment date and time

[1028] Step 7: Notification of appointment date and time

[1029] The terminal notifies the user of the confirmed consultation date and time.

[1030] (Input) Confirmed consultation date and time

[1031] (Output) Notification of appointment date and time (e.g. tomorrow 11:00 AM)

[1032] The device will send you appointment date and time notifications via app, email, or SMS.

[1033] Step 8: Start the consultation

[1034] The user can either visit the hospital at the specified date and time or choose to have an online diagnosis.

[1035] (Input) Specified consultation date and time

[1036] The server provides the doctor with the necessary diagnostic information for the examination.

[1037] (Output) Diagnostic information (disease name, diagnostic basis)

[1038] Doctors can conduct efficient consultations based on the information provided.

[1039] In this way, users can smoothly navigate through a series of processes, from symptom entry to appointment scheduling, emotion-based diagnostic assistance, optimization of appointment dates and times, notifications, and appointment selection.

[1040] (Application example 2)

[1041] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1042] In modern self-driving vehicles, if a passenger suddenly becomes ill, prompt and appropriate medical attention is required. However, current self-driving vehicles do not have a system that can monitor the passenger's physical condition in real time and provide the most appropriate medical attention when necessary. Furthermore, there is no mechanism in place to automatically contact a medical institution or change the route quickly. A system that can respond to such situations is needed.

[1043] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: input means for the patient to input text information about symptoms; receiving means for receiving the text information about symptoms and image and video data input by the input means; analysis means for analyzing the text information and image and video data received by the receiving means and predicting a disease name; diagnosis generation means for generating the disease name predicted by the analysis means and its basis; appointment optimization means for determining the optimal consultation date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time; notification means for notifying the patient of the consultation date and time determined by the appointment optimization means; and vehicle route change means for inputting symptoms and related data and changing the route to the nearest medical institution when a patient complains of feeling unwell inside the autonomously driven vehicle. This enables prompt medical attention when a passenger complains of feeling unwell, and automatically changes the route of the autonomously driven vehicle and contacts the nearest medical institution as necessary.

[1044] "Input means" refers to a device or interface that provides a function for patients to input text information, images, and video data related to their symptoms.

[1045] The "receiving means" is a device or system that has the function of receiving text information and image and video data related to symptoms input by the input means.

[1046] The "analysis means" refers to a device or software that has the function of analyzing the text information and image or video data received by the receiving means and predicting the name of the disease.

[1047] The "diagnosis generating means" is a device or system that has the function of generating diagnostic information based on the disease name predicted by the analysis means and the basis for that name.

[1048] An "appointment optimization tool" is a device or software that has the function of determining the optimal appointment date and time by taking into account the predicted diagnosis, existing appointment status, and the patient's desired date and time.

[1049] The "notification means" is a device or interface that has a function for notifying the patient of the consultation date and time determined by the appointment optimization means.

[1050] A "vehicle route change means" is a device or system that has the function of inputting symptoms and related data when a patient complains of feeling unwell inside an autonomous vehicle and changing the route to the nearest medical institution.

[1051] A specific embodiment of the in-vehicle medical support system of the present invention is described below. This system automatically provides appropriate medical care when a patient complains of poor health. This allows the patient to receive the most appropriate medical service quickly.

[1052] First, the user (patient) logs into the system using a tablet, smart glasses, or smartphone in the vehicle. The login information is sent to the server, which checks it against a database and, if authentication is successful, generates session information, allowing the user to proceed to the next step.

[1053] If a user feels unwell, they can enter specific symptoms using a tablet or smart glasses. For example, they can enter text information such as "I've had a stomachache for two days," and can also upload images or videos related to the symptoms. This information is then sent to the server via a receiving device.

[1054] The server analyzes the text information and image / video data received by the receiving means, using an emotion engine to analyze the patient's emotional state and stress level. It also uses medical AI to predict the name of the disease, and based on the results, generates diagnostic information using a diagnosis generation means. The diagnostic information includes the predicted name of the disease, its rationale, and recommended next steps.

[1055] Next, the server determines the optimal appointment date and time using the appointment optimization means, taking into account the existing appointment status and the patient's stress level. The determined appointment date and time is notified to the user by the notification means. For example, a specific date and time such as "Please come in for an appointment tomorrow at 10:00 AM" is displayed.

[1056] If it is determined that medical treatment is necessary, the server will use the vehicle route change method to change the route of the autonomous vehicle to the nearest medical institution. In the event of an emergency, the server will automatically contact the nearest hospital and urge them to take appropriate action.

[1057] As a concrete example, suppose a user in a self-driving vehicle inputs "I've had stomach pain for two days" and uploads an image of their abdomen. The emotion engine detects high stress, and MedicalAI determines that there is a high possibility of acute gastroenteritis. Based on this information, the server determines the optimal date and time for an appointment, notifies the user, and changes the self-driving vehicle's route to the nearest hospital.

[1058] Example prompt sentence:

[1059] Enter "I have had abdominal pain for two days and my lower abdomen is swollen," upload an image of your abdomen, and run a diagnosis.

[1060] As described above, the in-vehicle medical support system of the present invention automatically performs a series of processes to respond to a patient's poor physical condition, providing prompt and appropriate medical care.

[1061] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1062] Step 1:

[1063] The terminal obtains the user's login information (user ID and password) and sends it to the server. The server authenticates the received login information and, if successful, returns session information to the terminal. The input is the user's login information and the output is session information from the server. Specifically, the server references the database and checks whether there is any data that matches the input information.

[1064] Step 2:

[1065] The user uses a device to input text information about their symptoms and related image and video data. The input data is received by the device and sent to the server. The input is text information about symptoms and image and video data, and the output is data transmission to the server. Specifically, the user enters symptoms in the text box and attaches related data using the file upload function.

[1066] Step 3:

[1067] The server receives the data sent from the device and uses an emotion engine to analyze the user's emotional state and stress level. The input is text information about symptoms and image / video data, and the output is the emotion analysis results. Specifically, the server performs text analysis and image / video analysis to evaluate the user's emotional state.

[1068] Step 4:

[1069] Based on the data received by MedicalAI on the server, it analyzes symptoms and predicts the name of the disease. The input is the analyzed emotional state and symptom data, and the output is the predicted name of the disease and its reasons. Specifically, the AI ​​uses natural language processing and image analysis technology to comprehensively analyze the data.

[1070] Step 5:

[1071] The server generates diagnostic information based on the predicted disease name and its rationale. This diagnostic information includes the disease name, diagnostic rationale, and recommended next steps. The input is the disease name and its rationale, and the output is the diagnostic information. Specifically, the system compiles each piece of information into a text format.

[1072] Step 6:

[1073] The server uses a reservation optimization method to determine the optimal appointment date and time, taking into account existing reservations and the user's stress level. The input is the predicted illness, existing reservations, and stress level, and the output is the optimal appointment date and time. Specifically, the server accesses the reservation system, checks availability, and calculates the optimal date and time.

[1074] Step 7:

[1075] The server notifies the user of the determined consultation date and time using a notification means. The input is the optimal consultation date and time, and the output is a notification message to the user. Specifically, the message is sent to the terminal via the notification system and displayed on the user's screen.

[1076] Step 8:

[1077] When a user in an autonomous vehicle complains of feeling unwell, the server inputs the symptoms and related data and uses the vehicle route change means to change the route to the nearest medical institution. The input is the user's declaration of poor health and its detailed data, and the output is the route to the most suitable medical institution. Specifically, the server sets a new destination in the vehicle's navigation system and changes the driving route.

[1078] Example prompt sentence:

[1079] Enter "I have had abdominal pain for two days and my lower abdomen is swollen," upload an image of your abdomen, and run a diagnosis.

[1080] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1081] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1082] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1083] [Third embodiment]

[1084] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1085] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1086] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1087] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1088] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1089] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1090] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1091] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1092] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1093] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1094] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1095] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1096] The medical support system of the present invention allows patients to input their symptoms, and then uses artificial intelligence to perform highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. The elements of the system are as follows:

[1097] 1. Initial Setup and Login

[1098] User Login

[1099] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[1100] 2. Enter your symptoms

[1101] Symptom text entry

[1102] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[1103] 3. Symptom analysis

[1104] Receiving and analyzing data

[1105] The server receives text information and multimodal data (images and videos) sent from the device. The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and submits the most likely diagnosis.

[1106] 4. Generating diagnostic results

[1107] Creating diagnostic information

[1108] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[1109] 5. Optimizing appointment scheduling

[1110] Checking reservation status and deciding on consultation date and time

[1111] The server accesses the hospital's reservation system and checks the current reservation status. The server determines the optimal appointment date and time based on the user's desired date and time and available appointment times. The server then sends the determined appointment date and time to the terminal.

[1112] 6. Notifications and Online Diagnostics Options

[1113] Notification of appointment date and time

[1114] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[1115] 7. Start of consultation

[1116] User selection and consultation execution

[1117] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[1118] Specific examples

[1119] Symptom input and diagnosis

[1120] A user (say, Ichiro Tanaka) logs into the hospital's app and enters information about his sore throat. Tanaka enters the text "My sore throat has lasted for three days" and uploads an image of his throat. The device then sends the text information and image to the server.

[1121] Symptom analysis

[1122] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reason for this (text of symptoms, image analysis results).

[1123] Appointment optimization and notifications

[1124] The server references the hospital's reservation system to check whether there is an opening in the morning of the following day, which is Tanaka's desired time. It determines that the optimal consultation time is 11:00 AM the following day and notifies the terminal. The terminal then notifies Tanaka, "Please come in for a consultation at 11:00 AM tomorrow." On the day, Tanaka visits the hospital at 11:00 AM and is examined with a short wait.

[1125] Online Diagnostics Example

[1126] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Tanaka of the suggestion of a video call as an "online diagnosis option." Tanaka selects the online diagnosis and receives a consultation via video call at the specified time.

[1127] The present invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.

[1128] The processing flow will be explained below.

[1129] Step 1:

[1130] The user accesses the hospital's app or website and enters their login information (user ID and password).

[1131] Step 2:

[1132] The terminal acquires the entered login information and sends it to the server.

[1133] Step 3:

[1134] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[1135] Step 4:

[1136] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[1137] Step 5:

[1138] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[1139] Step 6:

[1140] The terminal acquires the input text information and transmits it to the server.

[1141] Step 7:

[1142] If the user wishes, they can upload images or videos related to their symptoms.

[1143] Step 8:

[1144] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[1145] Step 9:

[1146] The server receives text information and multimodal data (images and videos) sent from the device.

[1147] Step 10:

[1148] The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology.

[1149] Step 11:

[1150] The AI ​​in the server uses computer vision technology to analyze image and video data.

[1151] Step 12:

[1152] The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[1153] Step 13:

[1154] The server compiles the disease name generated by the AI ​​and the basis for the diagnosis into a single piece of diagnostic information.

[1155] Step 14:

[1156] The server accesses the hospital's reservation system and checks the status of existing reservations.

[1157] Step 15:

[1158] The server determines the optimal consultation date and time taking into consideration the user's desired date and time and available appointment times.

[1159] Step 16:

[1160] The server transmits the determined consultation date and time to the terminal.

[1161] Step 17:

[1162] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[1163] Step 18:

[1164] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[1165] Step 19:

[1166] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[1167] Step 20:

[1168] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[1169] Step 21:

[1170] Doctors can conduct efficient examinations based on the information provided.

[1171] Example 1

[1172] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1173] In modern healthcare, it is important for patients to quickly and accurately communicate their symptoms to medical institutions, but this has been difficult with conventional systems. Long wait times before seeing a doctor are also a factor in lower patient satisfaction. Furthermore, appointment scheduling systems are inefficient, often preventing patients from scheduling appointments at their desired dates and times. Therefore, there is a need for a system that can efficiently analyze patients' symptoms and determine the optimal appointment date and time, thereby improving the operational efficiency of medical institutions and increasing patient satisfaction.

[1174] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1175] In this invention, the server includes an input means for the patient to input text information about symptoms, a receiving means for receiving the text information about symptoms and image and video data input by the input means, an analysis means for analyzing the text information and image and video data received by the receiving means to predict a disease name, a diagnosis generation means for generating the disease name predicted by the analysis means and its basis and compiling it into diagnostic information, an appointment optimization means for accessing the medical institution's appointment system and determining the optimal appointment date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time, and a notification means for notifying the patient of the appointment date and time determined by the appointment optimization means. This makes it possible to efficiently analyze the patient's symptoms to assist in diagnosis and quickly determine the optimal appointment date and time.

[1176] "Patient" refers to a person who receives medical examination or treatment from a medical institution.

[1177] "Text information about symptoms" refers to written information in which a patient describes in detail their symptoms or discomfort.

[1178] "Input means" refers to a device or application function that allows a patient to input text information about their symptoms.

[1179] The "receiving means" refers to a mechanism that acquires text information and image or video data input by the input means and transmits it to the server.

[1180] "Analysis means" refers to an algorithm or model that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease from the symptoms.

[1181] The "diagnosis generating means" refers to a mechanism that generates the disease name predicted by the analysis means and its basis, and compiles it as diagnostic information.

[1182] The "reservation optimization means" refers to a mechanism for accessing a medical institution's reservation system and determining the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[1183] The "notification means" refers to a mechanism that notifies the patient of the consultation date and time determined by the appointment optimization means.

[1184] The "online diagnosis suggestion means" refers to a mechanism for suggesting the option of online diagnosis to a patient when it is determined that online diagnosis is possible.

[1185] "Medical support tools" refer to mechanisms that provide medical professionals with predicted disease names and diagnostic evidence, thereby improving the efficiency of medical treatment.

[1186] "Medical institution" refers to a facility such as a hospital, clinic, or doctor's office that provides medical examinations and treatment.

[1187] MODE FOR CARRYING OUT THE INVENTION

[1188] The medical support system of the present invention is a system in which patients input their symptoms, and based on that, artificial intelligence performs highly accurate analysis to assist in diagnosis, and determines and notifies the patient of the optimal consultation date and time. Each element of this system will be described in detail.

[1189] Initial Setup and Login

[1190] Login Process

[1191] A user accesses a hospital's app or website and enters their user ID and password into the displayed login form. The device sends this information to the server via a secure protocol (e.g., HTTPS). The server receives the login information and authenticates it by checking it against the user information in its database. If authentication is successful, the server generates a message indicating successful authentication and session information and returns it to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[1192] Enter symptoms

[1193] Enter symptoms and submit data

[1194] The user enters detailed text about their symptoms, such as "I've had a sore throat for three days." The device collects this text information and uploads images and videos as needed. All information, including uploaded image and video data, is then sent to the server.

[1195] Symptom analysis

[1196] Data analysis

[1197] The server uses AI (for example, the Gemini model, which uses NLP technology) to analyze the received text information and image and video data. The text information is analyzed using natural language processing (NLP) technology to extract symptom characteristics and correlations. The image and video data is also analyzed using computer vision technology to recognize visual symptom characteristics. The AI ​​on the server integrates this information and suggests the most likely diagnosis.

[1198] Generating diagnostic results

[1199] Creating diagnostic information

[1200] The server compiles the AI-generated disease names and the diagnostic evidence, building diagnostic information that is saved in a format that is easy for patients to refer to later.

[1201] Optimizing medical appointments

[1202] Checking reservation status and deciding on consultation date and time

[1203] The server accesses the hospital's reservation system to check the current reservation status. Then, it determines the optimal appointment date and time based on the user's desired date and time and the available appointment times. It then sends the determined appointment date and time to the terminal.

[1204] Choosing between notifications and online diagnostics

[1205] Notification of appointment date and time and online diagnosis suggestions

[1206] The device notifies the user of the consultation date and time received from the server and displays the specific date and time. In some cases, confirmations and preparations for the consultation are displayed. If the server determines that an online diagnosis is sufficient, the device offers the option of an online diagnosis. If the user selects an online diagnosis, the device displays a link to a video call or provides instructions for the corresponding app.

[1207] Start of consultation

[1208] Conducting an examination

[1209] The user can either visit the hospital at the notified appointment date and time or choose online diagnosis. The server provides medical professionals with diagnostic information (disease name and diagnostic basis) to support them in providing efficient medical care.

[1210] Specific examples

[1211] Specific examples of symptom input and diagnosis

[1212] A user (e.g., a patient) logs into a hospital app and enters details about a sore throat. The user enters the text "I have had a sore throat for three days" and uploads an image of their throat. The device then sends this information to the server.

[1213] Specific examples of symptom analysis

[1214] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that the condition is likely to be "pharyngitis." The server then generates "pharyngitis" and the basis for this (text of symptoms and image analysis results) as diagnostic information.

[1215] Specific examples of appointments and notifications

[1216] The server checks the hospital's reservation system and confirms that there is availability for the morning of the following day, as desired by the patient. It determines that the optimal appointment time is 11:00 AM the following day and notifies the terminal of this. The terminal then notifies the patient, "Please come in for an appointment at 11:00 AM tomorrow." On the day of the appointment, the patient can visit the hospital at 11:00 AM and be seen with a short waiting time.

[1217] Examples of online diagnostics

[1218] The diagnosis results in "mild pharyngitis," and the server determines that online diagnosis is sufficient. The device notifies the patient of the "online diagnosis option" and suggests a video call. The patient selects online diagnosis and receives a consultation via video call at the specified time.

[1219] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.The following is also used as an example of a prompt sentence to input into the generative AI model:

[1220] "I've had a sore throat for three days. Please see the image below. What is the name of the disease that corresponds to this symptom?"

[1221] The above is a specific embodiment of the present invention.

[1222] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1223] Step 1:

[1224] Login Process

[1225] input:

[1226] The user accesses the hospital's app or website and enters their user ID and password into the login form that appears on the screen.

[1227] Operation:

[1228] The device sends this information to the server via a secure protocol (e.g., HTTPS).

[1229] output:

[1230] The server receives the login information, compares it with the user information in the database, and determines whether authentication is successful. If successful, it generates a message indicating successful authentication and session information, and returns them to the terminal.

[1231] Specific behavior:

[1232] The server queries the database to retrieve user information.

[1233] If authentication is successful, the server generates a session ID and sends that information to the terminal.

[1234] The device will save the session ID and display a screen to proceed to the next step.

[1235] Step 2:

[1236] Enter symptoms and submit data

[1237] input:

[1238] Users enter detailed text about their symptoms and, if they wish, upload images or videos.

[1239] Operation:

[1240] The device acquires the entered text information and sends it to the server along with image and video data.

[1241] output:

[1242] The server receives the text information and image and video data.

[1243] Specific behavior:

[1244] The device temporarily stores the text information in its storage.

[1245] When a user uploads an image or video, the device checks the file format and size.

[1246] The device sends text, image, and video data to the server.

[1247] Step 3:

[1248] Data analysis

[1249] input:

[1250] Text information and image or video data received by the server.

[1251] Operation:

[1252] The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology to extract symptom characteristics and correlations, while image and video data is analyzed using computer vision technology.

[1253] output:

[1254] The AI ​​will suggest the most likely diagnosis based on the integrated information.

[1255] Specific behavior:

[1256] The server performs text analysis and extracts keywords and phrases.

[1257] Recognizing visual symptoms (e.g. redness or swelling of the throat) through image and video analysis.

[1258] Integrates NLP and computer vision results to implement AI algorithms for disease prediction.

[1259] Step 4:

[1260] Creating diagnostic information

[1261] input:

[1262] The disease name proposed through analysis and the supporting data.

[1263] Operation:

[1264] The server compiles the predicted disease name and the basis for the diagnosis as diagnostic information.

[1265] output:

[1266] Diagnostic information is generated and saved in a format that can be later referenced by the user.

[1267] Specific behavior:

[1268] The server stores the diagnostic information (disease name, reason) in a database as structured data.

[1269] -Include user-friendly explanations for diagnostic information.

[1270] Step 5:

[1271] Checking reservation status and deciding on consultation date and time

[1272] input:

[1273] The user's desired date and time and medical institution reservation status.

[1274] Operation:

[1275] The server accesses the hospital's reservation system to check the existing reservation status, and then determines the optimal appointment date and time based on the user's desired date and time.

[1276] output:

[1277] The determined consultation date and time is generated and transmitted to the terminal.

[1278] Specific behavior:

[1279] The server sends a query to the hospital's booking system through an API.

[1280] The server selects the optimal time based on the user's desired date and time and available reservation times.

[1281] The determined date and time is saved as session information and sent to the device.

[1282] Step 6:

[1283] Notification of appointment date and time and online diagnosis suggestions

[1284] input:

[1285] Decided appointment date and time and decision on online diagnosis.

[1286] Operation:

[1287] The terminal notifies the user of the consultation date and time received from the server and displays the specific date and time. If the server suggests online diagnosis, the terminal notifies the user of the online diagnosis option.

[1288] output:

[1289] The user is notified of appointment times and online diagnostic options.

[1290] Specific behavior:

[1291] The device will notify you of the appointment date and time and display a message on the screen saying, "Please come in for an appointment tomorrow at 11:00 AM."

[1292] -You will be presented with the option to do an online diagnostic and provided with a link if you wish.

[1293] Step 7:

[1294] Conducting an examination

[1295] input:

[1296] User-selected consultation date and time or online diagnosis

[1297] Operation:

[1298] The user visits the hospital at the designated appointment date and time, and the server provides the diagnosis information to the medical professional. In the case of online diagnosis, the consultation is conducted via video call.

[1299] output:

[1300] The examination is carried out efficiently.

[1301] Specific behavior:

[1302] The server provides medical professionals with diagnostic information (disease name and diagnostic basis).

[1303] -Medical professionals will examine the user based on the diagnostic information and provide appropriate treatment.

[1304] The above is the flow of processing of the program of this system.

[1305] (Application example 1)

[1306] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1307] In the modern healthcare system, the time and effort required for patients to access medical institutions and receive diagnosis and treatment is a problem. Furthermore, to receive an accurate diagnosis, a large amount of medical information must be provided to doctors, and efficient methods for doing this are needed. In particular, with the widespread use of online diagnosis, real-time diagnostic support and optimization of consultation appointments are important challenges.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1309] In this invention, the server includes an input means for patients to input text information about their symptoms, a receiving means, an analyzing means, a diagnosis generating means, a reservation optimization means, a notification means, an online phone consultation means, and a prompting means. This allows patients to receive appropriate consultations efficiently and quickly. Furthermore, if an online consultation is recommended, real-time consultations can be conducted via video calls, improving the efficiency of the entire medical process. Furthermore, by guiding patients through the input of prompts, the patient can smoothly input information, which helps support accurate diagnoses.

[1310] "Input means" refers to a device or function that allows a patient to input text information about symptoms.

[1311] The "receiving means" is a device or function that receives text information and image and video data related to symptoms input by the input means.

[1312] The "analysis means" is a device or function that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease.

[1313] The "diagnosis generating means" is a device or function that generates the name of the disease predicted by the analysis means and the basis for that name.

[1314] The "reservation optimization means" is a device or function that determines the optimal consultation date and time taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[1315] The "notification means" is a device or function that notifies the patient of the consultation date and time determined by the reservation optimization means.

[1316] "Online call consultation means" refers to a device or function that conducts consultation via video call when online diagnosis is recommended.

[1317] The "prompt guide means" is a device or function that guides the user to input a prompt sentence.

[1318] This invention relates to a system in which patients input their symptoms, and artificial intelligence (AI) performs highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. This system includes the following elements.

[1319] System Overview

[1320] 1. Input Method

[1321] Patients use a smartphone app to enter text information about their symptoms, such as "I've had a sore throat for three days," and upload images and video data related to their symptoms if necessary.

[1322] 2. Receiving Method

[1323] The server receives the text information and image and video data input by the input means.

[1324] 3. Analysis method

[1325] The server analyzes the received text information using natural language processing (NLP) technology, and image and video data using computer vision technology. Based on this analysis, the AI ​​predicts the most likely diagnosis based on the patient's symptoms. Examples of AI technologies used include AI models such as Gemini.

[1326] 4. Diagnostic Generation Methods

[1327] The AI ​​generates a predicted diagnosis and diagnostic evidence, which includes the diagnosis, diagnostic evidence, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[1328] 5. Reservation optimization measures

[1329] The server works with existing reservation systems to determine the optimal appointment time, taking into account the patient's desired date and time and available appointment times. For example, if a patient requests an appointment the following morning, the server checks availability during that time slot and determines the optimal appointment time.

[1330] 6. Means of notification

[1331] The patient's device is notified of the determined consultation date and time. The notification includes the specific date and time, and the patient can follow the instructions to receive the consultation.

[1332] 7. Online consultation methods

[1333] If the server recommends online diagnosis, it provides an option for a video consultation, in which the patient will be consulted by a doctor via video call at a specified time.

[1334] 8. Prompt Navigation Methods

[1335] The server helps the patient to input the information smoothly by guiding the user through the prompts. For example, the app displays the following prompts: "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," and "Please tell us the date and time you would like to make an appointment."

[1336] Hardware and software used

[1337] Hardware: Smartphone

[1338] Software: Python, Django (server side), REST API, AI tools (Gemini, NLP, computer vision)

[1339] Specific processing flow

[1340] Patients enter their symptoms (e.g., "I've had a sore throat for three days") and upload an image.

[1341] The server receives this, analyzes it using AI, and predicts the name of the disease.

[1342] Based on the provided diagnostic information, the server works with the reservation system to determine the optimal consultation date and time and notify the patient.

[1343] If necessary, we will offer the option of online consultations via video call to conduct consultations in real time.

[1344] In this way, patients can receive appropriate medical attention efficiently and quickly, improving the efficiency of the entire medical process.

[1345] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1346] Step 1:

[1347] The patient logs into the smartphone app.

[1348] Input: User ID and password.

[1349] Processing: The terminal acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of authentication success and session information to the terminal.

[1350] Output: A successful authentication message and session information.

[1351] Step 2:

[1352] Patients enter text information about their symptoms and upload image and video data.

[1353] Input: Detailed textual information about the symptom (e.g., "Sore throat lasting 3 days"), along with any associated image and video data, if needed.

[1354] Processing: The terminal obtains the entered text information and uploaded multimedia data and sends them to the server.

[1355] Output: Text information about the symptoms and multimedia data are sent to the server.

[1356] Step 3:

[1357] The server analyzes the received text information and image and video data.

[1358] Input: Textual information and multimedia data about symptoms.

[1359] Processing: The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology. At the same time, it analyzes image and video data using computer vision technology. From these analyses, it estimates the most likely diagnosis.

[1360] Output: Presumed disease name and diagnostic basis.

[1361] Step 4:

[1362] The server generates diagnostic information.

[1363] Input: Presumed disease name and diagnostic basis.

[1364] Processing: The server compiles the AI-generated diagnosis and its rationale into a single diagnostic report, which includes the diagnosis, rationale, and recommended next steps.

[1365] Output: Diagnostic information.

[1366] Step 5:

[1367] The server works in conjunction with the existing reservation system to determine the optimal appointment date and time.

[1368] Input: Diagnosis information, patient's desired date and time.

[1369] Processing: The server accesses the hospital's reservation system, checks the existing reservation status, and determines the optimal appointment date and time based on the patient's desired date and time and available appointment times.

[1370] Output: Best appointment date and time.

[1371] Step 6:

[1372] The server notifies the patient of the determined consultation date and time.

[1373] Input: Best appointment date and time.

[1374] Processing: The server uses the notification means to notify the patient's terminal of the determined consultation date and time. The notification includes the specific date and time.

[1375] Output: A notification message such as "The appointment date and time has been confirmed."

[1376] Step 7:

[1377] If the server recommends online diagnosis, it will offer the option of a video call consultation.

[1378] Input: Diagnostic information.

[1379] Processing: The server uses the online consultation method to notify the patient of the option to have a consultation via video call. If the patient selects online consultation, the consultation will be conducted via video call at the specified time.

[1380] Output: Notification of options for online consultation and conducting consultation via video call.

[1381] Step 8:

[1382] The server will announce the prompt.

[1383] Input: User symptom input status.

[1384] Processing: The server uses a prompting means to display prompt statements (e.g., "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," "Please tell us the date and time you would like to schedule an appointment") to assist the user in entering information.

[1385] Output: Displays prompts and assists the user in entering input smoothly.

[1386] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1387] The medical support system of the present invention allows patients to input their symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. Furthermore, by combining it with an emotion engine, further support based on the user's emotional state is possible.

[1388] 1. Initial Setup and Login

[1389] User Login

[1390] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[1391] 2. Enter your symptoms

[1392] Symptom text entry

[1393] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[1394] 3. Emotion Recognition by Emotion Engine

[1395] Data reception and emotion recognition

[1396] The server receives text information and multimodal data (images and videos) sent from the device. The emotion engine on the server analyzes the text information and image / video data to recognize the user's emotions. Based on the results of emotion recognition, the server can grasp the user's stress level and emotional state.

[1397] 4. Symptom analysis

[1398] Symptom analysis

[1399] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[1400] 5. Generating diagnostic results

[1401] Creating diagnostic information

[1402] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[1403] 6. Optimizing appointment scheduling

[1404] Reservation optimization based on emotional information

[1405] The server accesses the hospital's reservation system and checks the status of existing appointments. Based on the user's emotional information recognized by the emotion engine, the server determines the optimal appointment date and time to reduce the user's stress. The server then sends the determined appointment date and time to the terminal.

[1406] 7. Notifications and Online Diagnostics Options

[1407] Notification of appointment date and time

[1408] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[1409] 8. Start of consultation

[1410] User selection and consultation execution

[1411] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[1412] Specific examples

[1413] Symptom input and diagnosis

[1414] A user (for example, Hanako Sato) logs into the hospital's app and enters information about her sore throat. Sato enters the text "My sore throat has lasted for three days" and uploads an image of her throat. The device then sends the text information and image to the server.

[1415] Emotion recognition

[1416] The emotion engine in the server analyzes Sato's emotions based on text information and image data, and recognizes that his stress level is high.

[1417] Analyzing symptoms and generating diagnostic results

[1418] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reasons for it (text of symptoms, image analysis results, and sentiment analysis results).

[1419] Appointment optimization and notifications

[1420] The server refers to the hospital's reservation system and, taking into account that Mr. Sato has a high stress level and needs to be examined urgently, checks whether there are any openings in the morning of the following day. It determines that the optimal consultation time is 9:00 AM the following day and notifies the terminal. The terminal then notifies Mr. Sato, "Please come in for a consultation at 9:00 AM tomorrow." On the day, Mr. Sato visits the hospital at 9:00 AM and is examined with a short wait.

[1421] Online Diagnostics Example

[1422] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Mr. Sato of a suggestion of a video call as an "online diagnosis option." Mr. Sato selects the online diagnosis and receives a consultation via video call at the specified time.

[1423] This invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. Furthermore, by combining it with an emotion engine, it becomes possible to respond to patients' emotional states in a way that takes them into account, allowing for the provision of more optimal medical services.

[1424] The processing flow will be explained below.

[1425] Step 1:

[1426] The user accesses the hospital's app or website and enters their login information (user ID and password).

[1427] Step 2:

[1428] The terminal acquires the entered login information and sends it to the server.

[1429] Step 3:

[1430] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[1431] Step 4:

[1432] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[1433] Step 5:

[1434] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[1435] Step 6:

[1436] The terminal acquires the input text information and transmits it to the server.

[1437] Step 7:

[1438] Users take and upload images and videos related to their symptoms.

[1439] Step 8:

[1440] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[1441] Step 9:

[1442] The server receives the text information and image and video data sent from the terminal.

[1443] Step 10:

[1444] The emotion engine in the server analyzes text information and image / video data to recognize the user's emotions. The emotion recognition results are saved.

[1445] Step 11:

[1446] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology.

[1447] Step 12:

[1448] The AI ​​in the server uses computer vision technology to analyze image and video data.

[1449] Step 13:

[1450] The AI ​​on the server integrates text information, image and video information, and emotional information to predict the most likely diagnosis.

[1451] Step 14:

[1452] The server compiles the disease name generated by the AI ​​and its diagnostic basis (text analysis results, image analysis results, and emotion analysis results) into a single diagnostic information.

[1453] Step 15:

[1454] The server accesses the hospital's reservation system to check the status of existing reservations and the patient's preferred date and time.

[1455] Step 16:

[1456] The server determines the optimal consultation date and time to reduce the user's stress based on the user's emotional information recognized by the emotion engine.

[1457] Step 17:

[1458] The server sends the determined consultation date and time to the terminal and notifies the user.

[1459] Step 18:

[1460] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[1461] Step 19:

[1462] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[1463] Step 20:

[1464] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[1465] Step 21:

[1466] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[1467] Step 22:

[1468] Doctors can conduct efficient examinations based on the information provided.

[1469] Specific examples

[1470] Symptom input and diagnosis

[1471] Following the detailed procedure from step 1 to step 8, a user (e.g., Hanako Sato) logs into the hospital's app and enters information about her sore throat. She also takes an image of her throat and sends it to the server along with text information.

[1472] Emotion recognition

[1473] Following steps 9 and 10, the emotion engine in the server analyzes Sato's emotions based on the text information and image data, and recognizes that his stress level is high.

[1474] Analyzing symptoms and generating diagnostic results

[1475] Following steps 11 to 14, the AI ​​on the server analyzes the text information and image data and diagnoses a high probability of "pharyngitis." It then generates diagnostic information including the basis for the diagnosis.

[1476] Appointment optimization and notifications

[1477] Following the procedures from step 15 to step 18, the server determines that Mr. Sato, who has a high stress level, needs an urgent medical examination, confirms an appointment for 9:00 AM the next day, and notifies the terminal. The terminal then notifies the user, "Please come in for a medical examination at 9:00 AM tomorrow."

[1478] Proposal for online diagnosis

[1479] Following step 19, the diagnosis is determined to be "mild pharyngitis," and it is determined that online diagnosis is sufficient. The device notifies the user of a video call suggestion as an "online diagnosis option."

[1480] Conducting an examination

[1481] Following steps 20 to 22, Mr. Sato can either visit the hospital at the notified appointment date and time or select online diagnosis. The server provides the diagnosis information to the doctor, who then performs the examination efficiently.

[1482] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. In addition, by combining it with an emotion engine, it is possible to provide optimal medical services that take into account the emotional state of the patient.

[1483] Example 2

[1484] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1485] Conventional medical support systems predict illnesses and determine consultation dates and times based solely on symptom information entered by the patient, making it difficult to provide optimal consultations that take into account the patient's emotional state and stress level.In addition, there are many cases where online diagnosis candidates are not presented and medical support for doctors is not provided adequately, making it difficult to improve patient satisfaction.

[1486] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1487] In this invention, the server includes an input device for inputting text information about the patient's symptoms, a receiving device for receiving the input text information about the symptoms and image and video data, an emotion recognition device for analyzing the received data and recognizing emotions, an analysis device for analyzing the text and image and video data based on the recognized emotion information to predict the most likely disease name, a diagnosis generation device for generating a predicted disease name and its basis, an appointment optimization device for determining the optimal consultation date and time based on the predicted disease name, existing appointment status, and the patient's emotional state, and a notification device for notifying the patient of the determined consultation date and time. This enables the provision of optimal consultation dates and times that take the patient's emotional state into consideration, which is expected to improve patient satisfaction and the operational efficiency of medical institutions. It also enables the provision of online diagnosis options and medical support to doctors, creating a system that provides comprehensive medical support.

[1488] An "input means" is a device or program that provides an interface for a patient to input textual information or other data about their symptoms.

[1489] The "receiving means" is a device or program that receives text information and image or video data input through the input means.

[1490] The "emotion recognition means" is a device or program for analyzing the data received by the receiving means and recognizing the emotional state of the patient.

[1491] The "analysis means" is a device or program that analyzes text information and image and video data related to symptoms based on the emotional state recognized by the emotion recognition means, and predicts the most likely name of the disease.

[1492] The "diagnosis generating means" is a device or program that generates a diagnosis based on the disease name predicted by the analysis means and the basis for that diagnosis.

[1493] The "reservation optimization means" is a device or program that determines the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's emotional state.

[1494] The "notification means" is a device or program that notifies the patient of the consultation date and time determined by the appointment optimization means.

[1495] The "online diagnosis suggestion means" is a device or program that suggests online diagnosis options to a patient when it is determined that online diagnosis is possible.

[1496] The "diagnosis support means" is a device or program that provides a doctor with a predicted disease name and diagnostic basis, and supports medical treatment.

[1497] The medical support system of the present invention allows users to input their own symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the user. Furthermore, by combining it with an emotion engine, appropriate support can be achieved based on the user's emotional state.

[1498] The system includes the following main components:

[1499] 1. Input method:

[1500] This is an interface for users to enter text information about their symptoms. Examples include an input screen using a smartphone app or a web browser. Users can enter details such as "I have had a sore throat for three days" and can also upload images and videos if necessary.

[1501] 2. Receiving means:

[1502] A device or program that receives input text information and image / video data. Data sent from a smartphone app or web browser is received by a server and securely transferred using encryption technology such as SSL / TLS.

[1503] 3. Emotion recognition means:

[1504] This is a device or program that recognizes the user's emotional state based on the data received by the receiving means. Specifically, it uses IBM Watson Tone Analyzer or similar emotion recognition software. This allows the user's stress level and emotional state to be analyzed and stored in a database.

[1505] 4. Analysis method:

[1506] This is a device or program that analyzes symptom-related text information and image and video data based on emotional information recognized by an emotion recognition means. Specifically, it uses the Google Cloud Natural Language API and TensorFlow model to analyze symptoms and predict disease names using natural language processing (NLP) and computer vision technologies.

[1507] 5. Diagnostic generation methods:

[1508] This is a device or program that generates a diagnosis result based on the disease name predicted by the analysis means and the basis for that diagnosis. The server generates diagnostic information including the disease name, the basis for the diagnosis, and recommended next steps (e.g., whether a doctor's consultation is necessary or whether online diagnosis is possible).

[1509] 6. Reservation optimization measures:

[1510] This is a device or program that determines the optimal appointment time by taking into account the predicted diagnosis, existing appointments, and the patient's emotional state. It accesses the hospital's reservation system (e.g., reservation management software) and determines the optimal appointment time.

[1511] 7. Means of notification:

[1512] This is a device or program that notifies patients of the appointment date and time determined by the appointment optimization means. This notification is made via a smartphone app, email, SMS, etc., and informs the user of the specific appointment date and time.

[1513] For example, consider the following prompt:

[1514] We will implement a system where users log in to the hospital's app, input text and image data of their symptoms, and submit it. Based on that data, emotion recognition and symptom analysis will be performed to determine the optimal appointment time. What program code and AI tools will be used?

[1515] This system allows users to smoothly go through a series of processes, from entering symptoms to scheduling an appointment, providing emotion-based diagnostic assistance, optimizing consultation dates and times, receiving notifications, and even presenting online diagnostic options. Furthermore, by incorporating an emotion engine, it is possible to respond in a way that takes into account the patient's emotional state, thereby improving the quality of medical services and patient satisfaction.

[1516] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1517] Step 1: User Login

[1518] The user accesses the hospital's app or website and enters their user ID and password.

[1519] (Input) User ID and password

[1520] The terminal takes the entered login information and sends it to the server.

[1521] The server checks the login information it receives against information in its database.

[1522] (Output) Login success or failure message

[1523] If the login is successful, the server generates a session ID and returns it to the terminal.

[1524] The terminal stores the session ID in session storage and displays a screen for the user to proceed to the next step.

[1525] Step 2: Enter your symptoms

[1526] The user enters detailed text about the symptom on the symptom entry screen, for example, "I've had a sore throat for three days."

[1527] (Input) Text information about symptoms

[1528] The device checks the text information entered and retrieves any images or videos uploaded.

[1529] (Output) Symptom text, images, video data

[1530] The device converts this data into JSON format and sends it to the server.

[1531] Step 3: Emotion Recognition

[1532] The server receives the text information, images, and video data sent from the terminal.

[1533] (Input) Text information, image data, video data

[1534] The server inputs the received data into an emotion engine (e.g., IBM Watson Tone Analyzer).

[1535] The emotion engine analyzes text, images, and videos to recognize the user's emotional state.

[1536] (Output) Emotion recognition result (e.g., stress level, high or low)

[1537] The server stores the emotion recognition results in a database and proceeds to the next analysis step.

[1538] Step 4: Symptom analysis

[1539] The AI ​​in the server (e.g., Google Cloud Natural Language API, TensorFlow model) analyzes text, image, and video data related to symptoms based on emotional information obtained through emotion recognition methods.

[1540] (Input) Emotion recognition results, text information, image data, video data

[1541] The AI ​​engine uses natural language processing (NLP) technology to extract keywords from text information and predict disease diagnoses, and computer vision technology to analyze image and video data.

[1542] (Output) Predicted disease name and its reasoning

[1543] The server aggregates the analysis results and compiles them into a single diagnostic information.

[1544] Step 5: Generate diagnostic information

[1545] The server generates diagnostic information including the disease name predicted by the AI ​​and its rationale.

[1546] (Input) Predicted disease name and evidence

[1547] (Output) Diagnostic information (disease name, diagnostic basis, recommended next steps)

[1548] The server stores the diagnostic information in a database and prepares it for the scheduled optimization process.

[1549] Step 6: Optimize appointments

[1550] The server accesses the hospital's reservation system and checks the current reservation status.

[1551] (Input) Diagnosis information, existing appointment status, patient sentiment information

[1552] The server considers the user's emotional information (e.g., high stress level) and determines the optimal appointment time.

[1553] (Output) Best appointment date and time

[1554] The server inputs the determined consultation date and time into the reservation management system and confirms the reservation.

[1555] (Output) Confirmed appointment date and time

[1556] Step 7: Notification of appointment date and time

[1557] The terminal notifies the user of the confirmed consultation date and time.

[1558] (Input) Confirmed consultation date and time

[1559] (Output) Notification of appointment date and time (e.g. tomorrow 11:00 AM)

[1560] The device will send you appointment date and time notifications via app, email, or SMS.

[1561] Step 8: Start the consultation

[1562] The user can either visit the hospital at the specified date and time or choose to have an online diagnosis.

[1563] (Input) Specified consultation date and time

[1564] The server provides the doctor with the necessary diagnostic information for the examination.

[1565] (Output) Diagnostic information (disease name, diagnostic basis)

[1566] Doctors can conduct efficient consultations based on the information provided.

[1567] In this way, users can smoothly navigate through a series of processes, from symptom entry to appointment scheduling, emotion-based diagnostic assistance, optimization of appointment dates and times, notifications, and appointment selection.

[1568] (Application example 2)

[1569] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1570] In modern self-driving vehicles, if a passenger suddenly becomes ill, prompt and appropriate medical attention is required. However, current self-driving vehicles do not have a system that can monitor the passenger's physical condition in real time and provide the most appropriate medical attention when necessary. Furthermore, there is no mechanism in place to automatically contact a medical institution or change the route quickly. A system that can respond to such situations is needed.

[1571] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: input means for the patient to input text information about symptoms; receiving means for receiving the text information about symptoms and image and video data input by the input means; analysis means for analyzing the text information and image and video data received by the receiving means and predicting a disease name; diagnosis generation means for generating the disease name predicted by the analysis means and its basis; appointment optimization means for determining the optimal consultation date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time; notification means for notifying the patient of the consultation date and time determined by the appointment optimization means; and vehicle route change means for inputting symptoms and related data and changing the route to the nearest medical institution when a patient complains of feeling unwell inside the autonomously driven vehicle. This enables prompt medical attention when a passenger complains of feeling unwell, and automatically changes the route of the autonomously driven vehicle and contacts the nearest medical institution as necessary.

[1572] "Input means" refers to a device or interface that provides a function for patients to input text information, images, and video data related to their symptoms.

[1573] The "receiving means" is a device or system that has the function of receiving text information and image and video data related to symptoms input by the input means.

[1574] The "analysis means" refers to a device or software that has the function of analyzing the text information and image or video data received by the receiving means and predicting the name of the disease.

[1575] The "diagnosis generating means" is a device or system that has the function of generating diagnostic information based on the disease name predicted by the analysis means and the basis for that name.

[1576] An "appointment optimization tool" is a device or software that has the function of determining the optimal appointment date and time by taking into account the predicted diagnosis, existing appointment status, and the patient's desired date and time.

[1577] The "notification means" is a device or interface that has a function for notifying the patient of the consultation date and time determined by the appointment optimization means.

[1578] A "vehicle route change means" is a device or system that has the function of inputting symptoms and related data when a patient complains of feeling unwell inside an autonomous vehicle and changing the route to the nearest medical institution.

[1579] A specific embodiment of the in-vehicle medical support system of the present invention is described below. This system automatically provides appropriate medical care when a patient complains of poor health. This allows the patient to receive the most appropriate medical service quickly.

[1580] First, the user (patient) logs into the system using a tablet, smart glasses, or smartphone in the vehicle. The login information is sent to the server, which checks it against a database and, if authentication is successful, generates session information, allowing the user to proceed to the next step.

[1581] If a user feels unwell, they can enter specific symptoms using a tablet or smart glasses. For example, they can enter text information such as "I've had a stomachache for two days," and can also upload images or videos related to the symptoms. This information is then sent to the server via a receiving device.

[1582] The server analyzes the text information and image / video data received by the receiving means, using an emotion engine to analyze the patient's emotional state and stress level. It also uses medical AI to predict the name of the disease, and based on the results, generates diagnostic information using a diagnosis generation means. The diagnostic information includes the predicted name of the disease, its rationale, and recommended next steps.

[1583] Next, the server determines the optimal appointment date and time using the appointment optimization means, taking into account the existing appointment status and the patient's stress level. The determined appointment date and time is notified to the user by the notification means. For example, a specific date and time such as "Please come in for an appointment tomorrow at 10:00 AM" is displayed.

[1584] If it is determined that medical treatment is necessary, the server will use the vehicle route change method to change the route of the autonomous vehicle to the nearest medical institution. In the event of an emergency, the server will automatically contact the nearest hospital and urge them to take appropriate action.

[1585] As a concrete example, suppose a user in a self-driving vehicle inputs "I've had stomach pain for two days" and uploads an image of their abdomen. The emotion engine detects high stress, and MedicalAI determines that there is a high possibility of acute gastroenteritis. Based on this information, the server determines the optimal date and time for an appointment, notifies the user, and changes the self-driving vehicle's route to the nearest hospital.

[1586] Example prompt sentence:

[1587] Enter "I have had abdominal pain for two days and my lower abdomen is swollen," upload an image of your abdomen, and run a diagnosis.

[1588] As described above, the in-vehicle medical support system of the present invention automatically performs a series of processes to respond to a patient's poor physical condition, providing prompt and appropriate medical care.

[1589] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1590] Step 1:

[1591] The terminal obtains the user's login information (user ID and password) and sends it to the server. The server authenticates the received login information and, if successful, returns session information to the terminal. The input is the user's login information and the output is session information from the server. Specifically, the server references the database and checks whether there is any data that matches the input information.

[1592] Step 2:

[1593] The user uses a device to input text information about their symptoms and related image and video data. The input data is received by the device and sent to the server. The input is text information about symptoms and image and video data, and the output is data transmission to the server. Specifically, the user enters symptoms in the text box and attaches related data using the file upload function.

[1594] Step 3:

[1595] The server receives the data sent from the device and uses an emotion engine to analyze the user's emotional state and stress level. The input is text information about symptoms and image / video data, and the output is the emotion analysis results. Specifically, the server performs text analysis and image / video analysis to evaluate the user's emotional state.

[1596] Step 4:

[1597] Based on the data received by MedicalAI on the server, it analyzes symptoms and predicts the name of the disease. The input is the analyzed emotional state and symptom data, and the output is the predicted name of the disease and its reasons. Specifically, the AI ​​uses natural language processing and image analysis technology to comprehensively analyze the data.

[1598] Step 5:

[1599] The server generates diagnostic information based on the predicted disease name and its rationale. This diagnostic information includes the disease name, diagnostic rationale, and recommended next steps. The input is the disease name and its rationale, and the output is the diagnostic information. Specifically, the system compiles each piece of information into a text format.

[1600] Step 6:

[1601] The server uses a reservation optimization method to determine the optimal appointment date and time, taking into account existing reservations and the user's stress level. The input is the predicted illness, existing reservations, and stress level, and the output is the optimal appointment date and time. Specifically, the server accesses the reservation system, checks availability, and calculates the optimal date and time.

[1602] Step 7:

[1603] The server notifies the user of the determined consultation date and time using a notification means. The input is the optimal consultation date and time, and the output is a notification message to the user. Specifically, the message is sent to the terminal via the notification system and displayed on the user's screen.

[1604] Step 8:

[1605] When a user in an autonomous vehicle complains of feeling unwell, the server inputs the symptoms and related data and uses the vehicle route change means to change the route to the nearest medical institution. The input is the user's declaration of poor health and its detailed data, and the output is the route to the most suitable medical institution. Specifically, the server sets a new destination in the vehicle's navigation system and changes the driving route.

[1606] Example prompt sentence:

[1607] Enter "I have had abdominal pain for two days and my lower abdomen is swollen," upload an image of your abdomen, and run a diagnosis.

[1608] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1609] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1610] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1611] [Fourth embodiment]

[1612] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1613] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1614] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1615] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1616] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1618] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1619] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1620] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1623] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1624] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1625] The medical support system of the present invention allows patients to input their symptoms, and then uses artificial intelligence to perform highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. The elements of the system are as follows:

[1626] 1. Initial Setup and Login

[1627] User Login

[1628] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[1629] 2. Enter your symptoms

[1630] Symptom text entry

[1631] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[1632] 3. Symptom analysis

[1633] Receiving and analyzing data

[1634] The server receives text information and multimodal data (images and videos) sent from the device. The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and submits the most likely diagnosis.

[1635] 4. Generating diagnostic results

[1636] Creating diagnostic information

[1637] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[1638] 5. Optimizing appointment scheduling

[1639] Checking reservation status and deciding on consultation date and time

[1640] The server accesses the hospital's reservation system and checks the current reservation status. The server determines the optimal appointment date and time based on the user's desired date and time and available appointment times. The server then sends the determined appointment date and time to the terminal.

[1641] 6. Notifications and Online Diagnostics Options

[1642] Notification of appointment date and time

[1643] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[1644] 7. Start of consultation

[1645] User selection and consultation execution

[1646] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[1647] Specific examples

[1648] Symptom input and diagnosis

[1649] A user (say, Ichiro Tanaka) logs into the hospital's app and enters information about his sore throat. Tanaka enters the text "My sore throat has lasted for three days" and uploads an image of his throat. The device then sends the text information and image to the server.

[1650] Symptom analysis

[1651] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reason for this (text of symptoms, image analysis results).

[1652] Appointment optimization and notifications

[1653] The server references the hospital's reservation system to check whether there is an opening in the morning of the following day, which is Tanaka's desired time. It determines that the optimal consultation time is 11:00 AM the following day and notifies the terminal. The terminal then notifies Tanaka, "Please come in for a consultation at 11:00 AM tomorrow." On the day, Tanaka visits the hospital at 11:00 AM and is examined with a short wait.

[1654] Online Diagnostics Example

[1655] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Tanaka of the suggestion of a video call as an "online diagnosis option." Tanaka selects the online diagnosis and receives a consultation via video call at the specified time.

[1656] The present invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.

[1657] The processing flow will be explained below.

[1658] Step 1:

[1659] The user accesses the hospital's app or website and enters their login information (user ID and password).

[1660] Step 2:

[1661] The terminal acquires the entered login information and sends it to the server.

[1662] Step 3:

[1663] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[1664] Step 4:

[1665] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[1666] Step 5:

[1667] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[1668] Step 6:

[1669] The terminal acquires the input text information and transmits it to the server.

[1670] Step 7:

[1671] If the user wishes, they can upload images or videos related to their symptoms.

[1672] Step 8:

[1673] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[1674] Step 9:

[1675] The server receives text information and multimodal data (images and videos) sent from the device.

[1676] Step 10:

[1677] The AI ​​(Gemini) on the server performs text analysis and analyzes symptoms using natural language processing (NLP) technology.

[1678] Step 11:

[1679] The AI ​​in the server uses computer vision technology to analyze image and video data.

[1680] Step 12:

[1681] The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[1682] Step 13:

[1683] The server compiles the disease name generated by the AI ​​and the basis for the diagnosis into a single piece of diagnostic information.

[1684] Step 14:

[1685] The server accesses the hospital's reservation system and checks the status of existing reservations.

[1686] Step 15:

[1687] The server determines the optimal consultation date and time taking into consideration the user's desired date and time and available appointment times.

[1688] Step 16:

[1689] The server transmits the determined consultation date and time to the terminal.

[1690] Step 17:

[1691] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[1692] Step 18:

[1693] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[1694] Step 19:

[1695] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[1696] Step 20:

[1697] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[1698] Step 21:

[1699] Doctors can conduct efficient examinations based on the information provided.

[1700] Example 1

[1701] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1702] In modern healthcare, it is important for patients to quickly and accurately communicate their symptoms to medical institutions, but this has been difficult with conventional systems. Long wait times before seeing a doctor are also a factor in lower patient satisfaction. Furthermore, appointment scheduling systems are inefficient, often preventing patients from scheduling appointments at their desired dates and times. Therefore, there is a need for a system that can efficiently analyze patients' symptoms and determine the optimal appointment date and time, thereby improving the operational efficiency of medical institutions and increasing patient satisfaction.

[1703] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1704] In this invention, the server includes an input means for the patient to input text information about symptoms, a receiving means for receiving the text information about symptoms and image and video data input by the input means, an analysis means for analyzing the text information and image and video data received by the receiving means to predict a disease name, a diagnosis generation means for generating the disease name predicted by the analysis means and its basis and compiling it into diagnostic information, an appointment optimization means for accessing the medical institution's appointment system and determining the optimal appointment date and time taking into account the predicted disease name, existing appointment status, and the patient's desired date and time, and a notification means for notifying the patient of the appointment date and time determined by the appointment optimization means. This makes it possible to efficiently analyze the patient's symptoms to assist in diagnosis and quickly determine the optimal appointment date and time.

[1705] "Patient" refers to a person who receives medical examination or treatment from a medical institution.

[1706] "Text information about symptoms" refers to written information in which a patient describes in detail their symptoms or discomfort.

[1707] "Input means" refers to a device or application function that allows a patient to input text information about their symptoms.

[1708] The "receiving means" refers to a mechanism that acquires text information and image or video data input by the input means and transmits it to the server.

[1709] "Analysis means" refers to an algorithm or model that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease from the symptoms.

[1710] The "diagnosis generating means" refers to a mechanism that generates the disease name predicted by the analysis means and its basis, and compiles it as diagnostic information.

[1711] The "reservation optimization means" refers to a mechanism for accessing a medical institution's reservation system and determining the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[1712] The "notification means" refers to a mechanism that notifies the patient of the consultation date and time determined by the appointment optimization means.

[1713] The "online diagnosis suggestion means" refers to a mechanism for suggesting the option of online diagnosis to a patient when it is determined that online diagnosis is possible.

[1714] "Medical support tools" refer to mechanisms that provide medical professionals with predicted disease names and diagnostic evidence, thereby improving the efficiency of medical treatment.

[1715] "Medical institution" refers to a facility such as a hospital, clinic, or doctor's office that provides medical examinations and treatment.

[1716] MODE FOR CARRYING OUT THE INVENTION

[1717] The medical support system of the present invention is a system in which patients input their symptoms, and based on that, artificial intelligence performs highly accurate analysis to assist in diagnosis, and determines and notifies the patient of the optimal consultation date and time. Each element of this system will be described in detail.

[1718] Initial Setup and Login

[1719] Login Process

[1720] A user accesses a hospital's app or website and enters their user ID and password into the displayed login form. The device sends this information to the server via a secure protocol (e.g., HTTPS). The server receives the login information and authenticates it by checking it against the user information in its database. If authentication is successful, the server generates a message indicating successful authentication and session information and returns it to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[1721] Enter symptoms

[1722] Enter symptoms and submit data

[1723] The user enters detailed text about their symptoms, such as "I've had a sore throat for three days." The device collects this text information and uploads images and videos as needed. All information, including uploaded image and video data, is then sent to the server.

[1724] Symptom analysis

[1725] Data analysis

[1726] The server uses AI (for example, the Gemini model, which uses NLP technology) to analyze the received text information and image and video data. The text information is analyzed using natural language processing (NLP) technology to extract symptom characteristics and correlations. The image and video data is also analyzed using computer vision technology to recognize visual symptom characteristics. The AI ​​on the server integrates this information and suggests the most likely diagnosis.

[1727] Generating diagnostic results

[1728] Creating diagnostic information

[1729] The server compiles the AI-generated disease names and the diagnostic evidence, building diagnostic information that is saved in a format that is easy for patients to refer to later.

[1730] Optimizing medical appointments

[1731] Checking reservation status and deciding on consultation date and time

[1732] The server accesses the hospital's reservation system to check the current reservation status. Then, it determines the optimal appointment date and time based on the user's desired date and time and the available appointment times. It then sends the determined appointment date and time to the terminal.

[1733] Choosing between notifications and online diagnostics

[1734] Notification of appointment date and time and online diagnosis suggestions

[1735] The device notifies the user of the consultation date and time received from the server and displays the specific date and time. In some cases, confirmations and preparations for the consultation are displayed. If the server determines that an online diagnosis is sufficient, the device offers the option of an online diagnosis. If the user selects an online diagnosis, the device displays a link to a video call or provides instructions for the corresponding app.

[1736] Start of consultation

[1737] Conducting an examination

[1738] The user can either visit the hospital at the notified appointment date and time or choose online diagnosis. The server provides medical professionals with diagnostic information (disease name and diagnostic basis) to support them in providing efficient medical care.

[1739] Specific examples

[1740] Specific examples of symptom input and diagnosis

[1741] A user (e.g., a patient) logs into a hospital app and enters details about a sore throat. The user enters the text "I have had a sore throat for three days" and uploads an image of their throat. The device then sends this information to the server.

[1742] Specific examples of symptom analysis

[1743] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that the condition is likely to be "pharyngitis." The server then generates "pharyngitis" and the basis for this (text of symptoms and image analysis results) as diagnostic information.

[1744] Specific examples of appointments and notifications

[1745] The server checks the hospital's reservation system and confirms that there is availability for the morning of the following day, as desired by the patient. It determines that the optimal appointment time is 11:00 AM the following day and notifies the terminal of this. The terminal then notifies the patient, "Please come in for an appointment at 11:00 AM tomorrow." On the day of the appointment, the patient can visit the hospital at 11:00 AM and be seen with a short waiting time.

[1746] Examples of online diagnostics

[1747] The diagnosis results in "mild pharyngitis," and the server determines that online diagnosis is sufficient. The device notifies the patient of the "online diagnosis option" and suggests a video call. The patient selects online diagnosis and receives a consultation via video call at the specified time.

[1748] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions.The following is also used as an example of a prompt sentence to input into the generative AI model:

[1749] "I've had a sore throat for three days. Please see the image below. What is the name of the disease that corresponds to this symptom?"

[1750] The above is a specific embodiment of the present invention.

[1751] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1752] Step 1:

[1753] Login Process

[1754] input:

[1755] The user accesses the hospital's app or website and enters their user ID and password into the login form that appears on the screen.

[1756] Operation:

[1757] The device sends this information to the server via a secure protocol (e.g., HTTPS).

[1758] output:

[1759] The server receives the login information, compares it with the user information in the database, and determines whether authentication is successful. If successful, it generates a message indicating successful authentication and session information, and returns them to the terminal.

[1760] Specific behavior:

[1761] The server queries the database to retrieve user information.

[1762] If authentication is successful, the server generates a session ID and sends that information to the terminal.

[1763] The device will save the session ID and display a screen to proceed to the next step.

[1764] Step 2:

[1765] Enter symptoms and submit data

[1766] input:

[1767] Users enter detailed text about their symptoms and, if they wish, upload images or videos.

[1768] Operation:

[1769] The device acquires the entered text information and sends it to the server along with image and video data.

[1770] output:

[1771] The server receives the text information and image and video data.

[1772] Specific behavior:

[1773] The device temporarily stores the text information in its storage.

[1774] When a user uploads an image or video, the device checks the file format and size.

[1775] The device sends text, image, and video data to the server.

[1776] Step 3:

[1777] Data analysis

[1778] input:

[1779] Text information and image or video data received by the server.

[1780] Operation:

[1781] The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology to extract symptom characteristics and correlations, while image and video data is analyzed using computer vision technology.

[1782] output:

[1783] The AI ​​will suggest the most likely diagnosis based on the integrated information.

[1784] Specific behavior:

[1785] The server performs text analysis and extracts keywords and phrases.

[1786] Recognizing visual symptoms (e.g. redness or swelling of the throat) through image and video analysis.

[1787] Integrates NLP and computer vision results to implement AI algorithms for disease prediction.

[1788] Step 4:

[1789] Creating diagnostic information

[1790] input:

[1791] The disease name proposed through analysis and the supporting data.

[1792] Operation:

[1793] The server compiles the predicted disease name and the basis for the diagnosis as diagnostic information.

[1794] output:

[1795] Diagnostic information is generated and saved in a format that can be later referenced by the user.

[1796] Specific behavior:

[1797] The server stores the diagnostic information (disease name, reason) in a database as structured data.

[1798] -Include user-friendly explanations for diagnostic information.

[1799] Step 5:

[1800] Checking reservation status and deciding on consultation date and time

[1801] input:

[1802] The user's desired date and time and medical institution reservation status.

[1803] Operation:

[1804] The server accesses the hospital's reservation system to check the existing reservation status, and then determines the optimal appointment date and time based on the user's desired date and time.

[1805] output:

[1806] The determined consultation date and time is generated and transmitted to the terminal.

[1807] Specific behavior:

[1808] The server sends a query to the hospital's booking system through an API.

[1809] The server selects the optimal time based on the user's desired date and time and available reservation times.

[1810] The determined date and time is saved as session information and sent to the device.

[1811] Step 6:

[1812] Notification of appointment date and time and online diagnosis suggestions

[1813] input:

[1814] Decided appointment date and time and decision on online diagnosis.

[1815] Operation:

[1816] The terminal notifies the user of the consultation date and time received from the server and displays the specific date and time. If the server suggests online diagnosis, the terminal notifies the user of the online diagnosis option.

[1817] output:

[1818] The user is notified of appointment times and online diagnostic options.

[1819] Specific behavior:

[1820] The device will notify you of the appointment date and time and display a message on the screen saying, "Please come in for an appointment tomorrow at 11:00 AM."

[1821] -You will be presented with the option to do an online diagnostic and provided with a link if you wish.

[1822] Step 7:

[1823] Conducting an examination

[1824] input:

[1825] User-selected consultation date and time or online diagnosis

[1826] Operation:

[1827] The user visits the hospital at the designated appointment date and time, and the server provides the diagnosis information to the medical professional. In the case of online diagnosis, the consultation is conducted via video call.

[1828] output:

[1829] The examination is carried out efficiently.

[1830] Specific behavior:

[1831] The server provides medical professionals with diagnostic information (disease name and diagnostic basis).

[1832] -Medical professionals will examine the user based on the diagnostic information and provide appropriate treatment.

[1833] The above is the flow of processing of the program of this system.

[1834] (Application example 1)

[1835] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1836] In the modern healthcare system, the time and effort required for patients to access medical institutions and receive diagnosis and treatment is a problem. Furthermore, to receive an accurate diagnosis, a large amount of medical information must be provided to doctors, and efficient methods for doing this are needed. In particular, with the widespread use of online diagnosis, real-time diagnostic support and optimization of consultation appointments are important challenges.

[1837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1838] In this invention, the server includes an input means for patients to input text information about their symptoms, a receiving means, an analyzing means, a diagnosis generating means, a reservation optimization means, a notification means, an online phone consultation means, and a prompting means. This allows patients to receive appropriate consultations efficiently and quickly. Furthermore, if an online consultation is recommended, real-time consultations can be conducted via video calls, improving the efficiency of the entire medical process. Furthermore, by guiding patients through the input of prompts, the patient can smoothly input information, which helps support accurate diagnoses.

[1839] "Input means" refers to a device or function that allows a patient to input text information about symptoms.

[1840] The "receiving means" is a device or function that receives text information and image and video data related to symptoms input by the input means.

[1841] The "analysis means" is a device or function that analyzes the text information and image or video data received by the receiving means and predicts the name of the disease.

[1842] The "diagnosis generating means" is a device or function that generates the name of the disease predicted by the analysis means and the basis for that name.

[1843] The "reservation optimization means" is a device or function that determines the optimal consultation date and time taking into consideration the predicted disease name, existing reservation status, and the patient's desired date and time.

[1844] The "notification means" is a device or function that notifies the patient of the consultation date and time determined by the reservation optimization means.

[1845] "Online call consultation means" refers to a device or function that conducts consultation via video call when online diagnosis is recommended.

[1846] The "prompt guide means" is a device or function that guides the user to input a prompt sentence.

[1847] This invention relates to a system in which patients input their symptoms, and artificial intelligence (AI) performs highly accurate analysis based on the input to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. This system includes the following elements.

[1848] System Overview

[1849] 1. Input Method

[1850] Patients use a smartphone app to enter text information about their symptoms, such as "I've had a sore throat for three days," and upload images and video data related to their symptoms if necessary.

[1851] 2. Receiving Method

[1852] The server receives the text information and image and video data input by the input means.

[1853] 3. Analysis method

[1854] The server analyzes the received text information using natural language processing (NLP) technology, and image and video data using computer vision technology. Based on this analysis, the AI ​​predicts the most likely diagnosis based on the patient's symptoms. Examples of AI technologies used include AI models such as Gemini.

[1855] 4. Diagnostic Generation Methods

[1856] The AI ​​generates a predicted diagnosis and diagnostic evidence, which includes the diagnosis, diagnostic evidence, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[1857] 5. Reservation optimization measures

[1858] The server works with existing reservation systems to determine the optimal appointment time, taking into account the patient's desired date and time and available appointment times. For example, if a patient requests an appointment the following morning, the server checks availability during that time slot and determines the optimal appointment time.

[1859] 6. Means of notification

[1860] The patient's device is notified of the determined consultation date and time. The notification includes the specific date and time, and the patient can follow the instructions to receive the consultation.

[1861] 7. Online consultation methods

[1862] If the server recommends online diagnosis, it provides an option for a video consultation, in which the patient will be consulted by a doctor via video call at a specified time.

[1863] 8. Prompt Navigation Methods

[1864] The server helps the patient to input the information smoothly by guiding the user through the prompts. For example, the app displays the following prompts: "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," and "Please tell us the date and time you would like to make an appointment."

[1865] Hardware and software used

[1866] Hardware: Smartphone

[1867] Software: Python, Django (server side), REST API, AI tools (Gemini, NLP, computer vision)

[1868] Specific processing flow

[1869] Patients enter their symptoms (e.g., "I've had a sore throat for three days") and upload an image.

[1870] The server receives this, analyzes it using AI, and predicts the name of the disease.

[1871] Based on the provided diagnostic information, the server works with the reservation system to determine the optimal consultation date and time and notify the patient.

[1872] If necessary, we will offer the option of online consultations via video call to conduct consultations in real time.

[1873] In this way, patients can receive appropriate medical attention efficiently and quickly, improving the efficiency of the entire medical process.

[1874] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1875] Step 1:

[1876] The patient logs into the smartphone app.

[1877] Input: User ID and password.

[1878] Processing: The terminal acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of authentication success and session information to the terminal.

[1879] Output: A successful authentication message and session information.

[1880] Step 2:

[1881] Patients enter text information about their symptoms and upload image and video data.

[1882] Input: Detailed textual information about the symptom (e.g., "Sore throat lasting 3 days"), along with any associated image and video data, if needed.

[1883] Processing: The terminal obtains the entered text information and uploaded multimedia data and sends them to the server.

[1884] Output: Text information about the symptoms and multimedia data are sent to the server.

[1885] Step 3:

[1886] The server analyzes the received text information and image and video data.

[1887] Input: Textual information and multimedia data about symptoms.

[1888] Processing: The AI ​​(Gemini) on the server analyzes text information using natural language processing (NLP) technology. At the same time, it analyzes image and video data using computer vision technology. From these analyses, it estimates the most likely diagnosis.

[1889] Output: Presumed disease name and diagnostic basis.

[1890] Step 4:

[1891] The server generates diagnostic information.

[1892] Input: Presumed disease name and diagnostic basis.

[1893] Processing: The server compiles the AI-generated diagnosis and its rationale into a single diagnostic report, which includes the diagnosis, rationale, and recommended next steps.

[1894] Output: Diagnostic information.

[1895] Step 5:

[1896] The server works in conjunction with the existing reservation system to determine the optimal appointment date and time.

[1897] Input: Diagnosis information, patient's desired date and time.

[1898] Processing: The server accesses the hospital's reservation system, checks the existing reservation status, and determines the optimal appointment date and time based on the patient's desired date and time and available appointment times.

[1899] Output: Best appointment date and time.

[1900] Step 6:

[1901] The server notifies the patient of the determined consultation date and time.

[1902] Input: Best appointment date and time.

[1903] Processing: The server uses the notification means to notify the patient's terminal of the determined consultation date and time. The notification includes the specific date and time.

[1904] Output: A notification message such as "The appointment date and time has been confirmed."

[1905] Step 7:

[1906] If the server recommends online diagnosis, it will offer the option of a video call consultation.

[1907] Input: Diagnostic information.

[1908] Processing: The server uses the online consultation method to notify the patient of the option to have a consultation via video call. If the patient selects online consultation, the consultation will be conducted via video call at the specified time.

[1909] Output: Notification of options for online consultation and conducting consultation via video call.

[1910] Step 8:

[1911] The server will announce the prompt.

[1912] Input: User symptom input status.

[1913] Processing: The server uses a prompting means to display prompt statements (e.g., "Please enter your symptoms. For example, you have had a sore throat for three days," "Please select an image to upload," "Please tell us the date and time you would like to schedule an appointment") to assist the user in entering information.

[1914] Output: Displays prompts and assists the user in entering input smoothly.

[1915] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1916] The medical support system of the present invention allows patients to input their symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the patient. Furthermore, by combining it with an emotion engine, further support based on the user's emotional state is possible.

[1917] 1. Initial Setup and Login

[1918] User Login

[1919] The user accesses the hospital's app or website and enters their login information (user ID and password). The device acquires the entered login information and sends it to the server. The server performs authentication by comparing the received login information with the user information in its database. If authentication is successful, the server returns a message of successful authentication and session information to the device. The device saves the session information and displays a screen that allows the user to proceed to the next step.

[1920] 2. Enter your symptoms

[1921] Symptom text entry

[1922] The user enters detailed text about the symptoms. For example, "I have had a sore throat for three days." The device acquires the entered text information. If the user wishes, they can upload images or videos related to the symptoms. The device acquires the uploaded image and video data and sends it to the server along with the text information.

[1923] 3. Emotion Recognition by Emotion Engine

[1924] Data reception and emotion recognition

[1925] The server receives text information and multimodal data (images and videos) sent from the device. The emotion engine on the server analyzes the text information and image / video data to recognize the user's emotions. Based on the results of emotion recognition, the server grasps the user's stress level and emotional state.

[1926] 4. Symptom analysis

[1927] Symptom analysis

[1928] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology. It also analyzes image and video data using computer vision technology. The AI ​​on the server integrates this information and predicts the most likely diagnosis.

[1929] 5. Generating diagnostic results

[1930] Creating diagnostic information

[1931] The server compiles the AI-generated disease name and its diagnostic basis into a single diagnostic information set, which includes the disease name, diagnostic basis, and recommended next steps (e.g., whether a doctor's consultation is required or whether online diagnosis is possible).

[1932] 6. Optimizing appointment scheduling

[1933] Reservation optimization based on emotional information

[1934] The server accesses the hospital's reservation system and checks the status of existing appointments. Based on the user's emotional information recognized by the emotion engine, the server determines the optimal appointment date and time to reduce the user's stress. The server then sends the determined appointment date and time to the terminal.

[1935] 7. Notifications and Online Diagnostics Options

[1936] Notification of appointment date and time

[1937] The terminal notifies the user that "The consultation date and time has been confirmed" and displays the specific date and time. Example: "Tomorrow at 11:00 AM." If the server determines that online diagnosis is sufficient, it also notifies the terminal of the option for online diagnosis. Example: "Do you wish to have an online diagnosis?"

[1938] 8. Start of consultation

[1939] User selection and consultation execution

[1940] The user can either visit the hospital at the notified appointment date and time, or choose online diagnosis. The server provides the doctor with diagnosis information (disease name and diagnostic basis). The doctor can then perform an efficient examination based on the information provided.

[1941] Specific examples

[1942] Symptom input and diagnosis

[1943] A user (for example, Hanako Sato) logs into the hospital's app and enters information about her sore throat. She enters the text "My sore throat has lasted for three days" and uploads an image of her throat. The device then sends the text information and image to the server.

[1944] Emotion recognition

[1945] The emotion engine in the server analyzes Sato's emotions based on text information and image data, and recognizes that his stress level is high.

[1946] Analyzing symptoms and generating diagnostic results

[1947] The server analyzes the text "Sore throat for 3 days" and the image, and the AI ​​determines that "pharyngitis" is likely. The server generates "pharyngitis" and the reasons for it (text of symptoms, image analysis results, and sentiment analysis results).

[1948] Appointment optimization and notifications

[1949] The server refers to the hospital's reservation system and, taking into account that Mr. Sato has a high stress level and needs to be examined urgently, checks whether there are any openings in the morning of the following day. It determines that the optimal consultation time is 9:00 AM the following day and notifies the terminal. The terminal then notifies Mr. Sato, "Please come in for a consultation at 9:00 AM tomorrow." On the day, Mr. Sato visits the hospital at 9:00 AM and is examined with a short wait.

[1950] Online Diagnostics Example

[1951] The diagnosis results in "mild pharyngitis," and the server determines that an online diagnosis is sufficient. The device notifies Mr. Sato of a suggestion of a video call as an "online diagnosis option." Mr. Sato selects the online diagnosis and receives a consultation via video call at the specified time.

[1952] This invention can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. Furthermore, by combining it with an emotion engine, it becomes possible to respond to patients' emotional states in a way that takes them into account, allowing for the provision of more optimal medical services.

[1953] The processing flow will be explained below.

[1954] Step 1:

[1955] The user accesses the hospital's app or website and enters their login information (user ID and password).

[1956] Step 2:

[1957] The terminal acquires the entered login information and sends it to the server.

[1958] Step 3:

[1959] The server authenticates the received login information by checking it against the user information in its database. If authentication is successful, the server returns a message indicating successful authentication and the session information to the terminal.

[1960] Step 4:

[1961] The terminal saves the session information and displays a screen that allows the user to proceed to the next step.

[1962] Step 5:

[1963] The user enters detailed text about the symptom, e.g., "I've had a sore throat for three days."

[1964] Step 6:

[1965] The terminal acquires the input text information and transmits it to the server.

[1966] Step 7:

[1967] Users take and upload images and videos related to their symptoms.

[1968] Step 8:

[1969] The device retrieves the uploaded image and video data and sends it to the server along with the text information.

[1970] Step 9:

[1971] The server receives the text information and image and video data sent from the terminal.

[1972] Step 10:

[1973] The emotion engine in the server analyzes text information and image / video data to recognize the user's emotions. The emotion recognition results are saved.

[1974] Step 11:

[1975] The AI ​​(Gemini) on the server performs text analysis based on the emotional information recognized by the emotion engine, and analyzes symptoms using natural language processing (NLP) technology.

[1976] Step 12:

[1977] The AI ​​in the server uses computer vision technology to analyze image and video data.

[1978] Step 13:

[1979] The AI ​​on the server integrates text information, image and video information, and emotional information to predict the most likely diagnosis.

[1980] Step 14:

[1981] The server compiles the disease name generated by the AI ​​and its diagnostic basis (text analysis results, image analysis results, and emotion analysis results) into a single diagnostic information.

[1982] Step 15:

[1983] The server accesses the hospital's reservation system to check the status of existing reservations and the patient's preferred date and time.

[1984] Step 16:

[1985] The server determines the optimal consultation date and time to reduce the user's stress based on the user's emotional information recognized by the emotion engine.

[1986] Step 17:

[1987] The server sends the determined consultation date and time to the terminal and notifies the user.

[1988] Step 18:

[1989] The device notifies the user that the appointment date and time has been confirmed, and displays the specific date and time, e.g., "Tomorrow at 11:00 AM."

[1990] Step 19:

[1991] If the server determines that online diagnostics are sufficient, it will also notify the terminal of the online diagnostics option. Example: "Do you want online diagnostics?"

[1992] Step 20:

[1993] The user can either visit the hospital on the notified date and time or choose to have an online diagnosis.

[1994] Step 21:

[1995] The server provides the doctor with diagnostic information (disease name and diagnostic basis).

[1996] Step 22:

[1997] Doctors can conduct efficient examinations based on the information provided.

[1998] Specific examples

[1999] Symptom input and diagnosis

[2000] Following the detailed procedure from step 1 to step 8, a user (e.g., Hanako Sato) logs into the hospital's app and enters information about her sore throat. She also takes an image of her throat and sends it to the server along with text information.

[2001] Emotion recognition

[2002] Following steps 9 and 10, the emotion engine in the server analyzes Sato's emotions based on the text information and image data, and recognizes that his stress level is high.

[2003] Analyzing symptoms and generating diagnostic results

[2004] Following steps 11 to 14, the AI ​​on the server analyzes the text information and image data and diagnoses a high probability of "pharyngitis." It then generates diagnostic information including the basis for the diagnosis.

[2005] Appointment optimization and notifications

[2006] Following the procedures from step 15 to step 18, the server determines that Mr. Sato, who has a high stress level, needs an urgent medical examination, confirms an appointment for 9:00 AM the next day, and notifies the terminal. The terminal then notifies the user, "Please come in for a medical examination at 9:00 AM tomorrow."

[2007] Proposal for online diagnosis

[2008] Following step 19, the diagnosis is determined to be "mild pharyngitis," and it is determined that online diagnosis is sufficient. The device notifies the user of a video call suggestion as an "online diagnosis option."

[2009] Conducting an examination

[2010] Following steps 20 to 22, Mr. Sato can either visit the hospital at the notified appointment date and time or select online diagnosis. The server provides the diagnosis information to the doctor, who then performs the examination efficiently.

[2011] This system can significantly reduce waiting times at hospitals, improve patient satisfaction, and increase the operational efficiency of medical institutions. In addition, by combining it with an emotion engine, it is possible to provide optimal medical services that take into account the emotional state of the patient.

[2012] Example 2

[2013] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2014] Conventional medical support systems predict illnesses and determine consultation dates and times based solely on symptom information entered by the patient, making it difficult to provide optimal consultations that take into account the patient's emotional state and stress level.In addition, there are many cases where online diagnosis candidates are not presented and medical support for doctors is not provided adequately, making it difficult to improve patient satisfaction.

[2015] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2016] In this invention, the server includes an input device for inputting text information about the patient's symptoms, a receiving device for receiving the input text information about the symptoms and image and video data, an emotion recognition device for analyzing the received data and recognizing emotions, an analysis device for analyzing the text and image and video data based on the recognized emotion information to predict the most likely disease name, a diagnosis generation device for generating a predicted disease name and its basis, an appointment optimization device for determining the optimal consultation date and time based on the predicted disease name, existing appointment status, and the patient's emotional state, and a notification device for notifying the patient of the determined consultation date and time. This enables the provision of optimal consultation dates and times that take the patient's emotional state into consideration, which is expected to improve patient satisfaction and the operational efficiency of medical institutions. It also enables the provision of online diagnosis options and medical support to doctors, creating a system that provides comprehensive medical support.

[2017] An "input means" is a device or program that provides an interface for a patient to input textual information or other data about their symptoms.

[2018] The "receiving means" is a device or program that receives text information and image or video data input through the input means.

[2019] The "emotion recognition means" is a device or program for analyzing the data received by the receiving means and recognizing the emotional state of the patient.

[2020] The "analysis means" is a device or program that analyzes text information and image and video data related to symptoms based on the emotional state recognized by the emotion recognition means, and predicts the most likely name of the disease.

[2021] The "diagnosis generating means" is a device or program that generates a diagnosis based on the disease name predicted by the analysis means and the basis for that diagnosis.

[2022] The "reservation optimization means" is a device or program that determines the optimal consultation date and time by taking into consideration the predicted disease name, existing reservation status, and the patient's emotional state.

[2023] The "notification means" is a device or program that notifies the patient of the consultation date and time determined by the appointment optimization means.

[2024] The "online diagnosis suggestion means" is a device or program that suggests online diagnosis options to a patient when it is determined that online diagnosis is possible.

[2025] The "diagnosis support means" is a device or program that provides a doctor with a predicted disease name and diagnostic basis, and supports medical treatment.

[2026] The medical support system of the present invention allows users to input their own symptoms, and then uses AI to perform highly accurate analysis to assist in diagnosis, determine the optimal consultation date and time, and notify the user. Furthermore, by combining it with an emotion engine, appropriate support can be achieved based on the user's emotional state.

[2027] The system includes the following main components:

[2028] 1. Input method:

[2029] This is an interface for users to enter text information about their symptoms. Examples include an input screen using a smartphone app or a web browser. Users can enter details such as "I have had a sore throat for three days" and can also upload images and videos if necessary.

[2030] 2. Receiving means:

[2031] A device or program that receives input text information and image / video data. Data sent from a smartphone app or web browser is received by a server and transferred securely using encryption technology such as SSL / TLS.

[2032] 3. Emotion recognition means:

[2033] This is a device or program that recognizes the user's emotional state based on the data received by the receiving means. Specifically, it uses IBM Watson Tone Analyzer or similar emotion recognition software. This allows the user's stress level and emotional state to be analyzed and stored in a datab...

Claims

1. an input means for a patient to input text information about their symptoms; a receiving means for receiving text information and image and video data relating to the symptoms input by the input means; an analysis means for analyzing the text information and image or video data received by the receiving means and predicting the name of a disease; a diagnosis generating means for generating a disease name predicted by the analyzing means and its basis; A reservation optimization means for determining the optimal consultation date and time in consideration of the predicted disease name, existing reservation status, and the patient's desired date and time; a notification means for notifying the patient of the examination date and time determined by the appointment optimization means; A system including:

2. 2. The system according to claim 1, further comprising an online diagnosis suggestion means for suggesting the option of online diagnosis to the patient when it is determined that online diagnosis is possible.

3. The system according to claim 1, further comprising a medical support means for providing a predicted disease name and diagnostic basis to a doctor and supporting medical treatment.

Images