system

Abstract

We provide the system. [Solution] A system that provides medical second opinions using a generative artificial intelligence device, A means for inputting blood test results using an input device, A means for inputting the results of the medical interview and physical examination, A means of integrating the above blood test results with the results of the medical interview and physical examination, A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans, A means for providing the generated treatment plan to healthcare professionals and patients via a display means, A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the medical field, it is important to incorporate various perspectives in the determination of treatment policies. However, in many medical institutions, it is difficult to quickly propose an optimal treatment policy with limited resources. In particular, to obtain a second opinion, it is necessary to incorporate the opinions of multiple doctors, which is time-consuming and costly. In addition, for the general public, since medical information is highly specialized, it is difficult to select a treatment policy with sufficient understanding. Therefore, there is a need for a system that can efficiently and effectively determine an optimal treatment policy for patients and doctors.

Means for Solving the Problems

[0005] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. This system includes the following means:

[0006] 1. A means for inputting blood test results using an input device.

[0007] 2. Means for inputting the results of the medical interview and physical examination using input means,

[0008] 3. Means for integrating the above blood test results with the results of the medical interview and physical examination,

[0009] 4. A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans.

[0010] 5. Means for providing the generated treatment plan to healthcare professionals and patients via display means.

[0011] 6. A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism.

[0012] This system allows healthcare professionals to quickly propose and consider multiple treatment options within limited time and resources. Furthermore, because the generative artificial intelligence model generates treatment plans based on learned datasets, it enables highly accurate suggestions that comply with the latest medical information. In addition, multiple treatment plans are provided to patients in an easy-to-understand format, promoting treatment choices that are more based on informed consent.

[0013] A "generative artificial intelligence device" is a device that utilizes artificial intelligence technology to generate new information or suggestions based on input data.

[0014] "Blood test results" are medical data obtained by analyzing blood, and are used to evaluate a patient's health status and diseases.

[0015] "The results of consultation and palpation" refer to medical data obtained by medical staff through questioning patients and conducting physical examinations.

[0016] "Input means" refers to a device or interface for inputting data into a system, including, for example, a keyboard and a touch screen.

[0017] "Integrating means" refers to means for summarizing different types of data into one unified format.

[0018] "Generative artificial intelligence model" refers to a model of artificial intelligence technology that has the ability to generate new information and proposals by learning a large amount of data in advance.

[0019] "Treatment plan" refers to a plan that details treatment policies and procedures based on the patient's condition.

[0020] "Display means" refers to a device or interface for visually presenting the information generated by the system to the user, including, for example, a monitor and a projector.

[0021] "Medical staff" refers to professionals engaged in medical care, such as doctors, nurses, and pharmacists.

[0022] "Patient" refers to a person who receives diagnosis and treatment.

[0023] "Selecting means" refers to means for selecting the optimal option from a plurality of provided options.

[0024] "Final decision-making means" refers to means for finalizing, recording, and storing the selected options and plans.

Brief Description of the Drawings

[0025] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0026] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0027] First, let's explain the terminology used in the following explanation.

[0028] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0029] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0030] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0031] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0033] [First Embodiment]

[0034] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0035] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0038] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0041] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0045] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0046] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[0047] Data entry

[0048] The user first enters blood test results using a terminal. These results include data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into the system's input form or imported electronically. The user also enters the results of the patient's medical history and physical examination. This includes the patient's symptoms (e.g., fever, fatigue) and physical examination results (e.g., heart rate, blood pressure).

[0049] Data integration and processing

[0050] The server receives blood test results and interview / palpation results sent from the terminal. The received data is integrated and compiled into a single dataset. Specifically, the blood test results and interview / palpation results are integrated and converted into a format suitable for subsequent processing.

[0051] Treatment plan generation

[0052] Based on the integrated data, the server uses a generative artificial intelligence model to generate multiple treatment plans. This model is pre-tuned and trained on past performance and the latest medical research data. This generates candidate treatment plans that are optimal for the patient's specific health condition.

[0053] Display and selection of treatment plans

[0054] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). On the terminal, multiple treatment plans are displayed in a list format, including a detailed description of each treatment plan and its expected outcome.

[0055] Users (patients and doctors) evaluate these treatment plans and select the optimal one. Options include highly effective treatments, cost-effective treatments, and treatments best suited to the patient's lifestyle and preferences.

[0056] Final decision and implementation

[0057] The selected treatment plan is communicated to the server via the user's device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[0058] Specific example

[0059] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, a generative artificial intelligence model generates multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[0060] The generated treatment plan is displayed on the terminal, allowing users (doctor and patient) to review the details of each plan and select a specific course of action. They choose the plan they deem most appropriate (for example, "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[0061] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] Users (medical staff) use a terminal to input blood test results. Blood test results consist of numerical data such as red blood cell count, white blood cell count, and platelet count. This data is either manually entered into an input form or imported into the system using an electronic import method.

[0065] Step 2:

[0066] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. The results of the medical interview and physical examination include the symptoms reported by the patient (e.g., fatigue, fever, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.). This information is also entered into a dedicated form.

[0067] Step 3:

[0068] The server receives blood test results and interview / physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[0069] Step 4:

[0070] The server integrates the received blood test results and the results of the medical interview and physical examination. The integrated data is converted into a format suitable for later analysis and compiled into a single dataset.

[0071] Step 5:

[0072] The server inputs the integrated data into a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. Using this model, multiple treatment plans are generated, tailored to the patient's specific health condition.

[0073] Step 6:

[0074] The server sends multiple treatment plans it has generated to the terminal. The terminal displays the candidate treatment plans in a list format, along with a detailed description of each plan and its expected outcome.

[0075] Step 7:

[0076] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the patient's lifestyle and preferences.

[0077] Step 8:

[0078] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[0079] Step 9:

[0080] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[0081] (Example 1)

[0082] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0083] In modern medicine, there is a need for systems that enable healthcare professionals to propose multiple treatment plans quickly and accurately within limited time and resources, allowing patients to choose the optimal treatment. However, conventional systems do not adequately provide mechanisms for integrating multiple data sources, automatically generating appropriate treatment plans, and ultimately enabling patients and doctors to efficiently select a treatment plan. This increases the burden on healthcare professionals and makes it difficult for patients to make appropriate treatment choices based on informed consent.

[0084] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0085] In this invention, the server includes means for a user to input patient health data using a terminal, means for transmitting data from the terminal to the server, means for the server to integrate the received data and convert it into a format, means for generating multiple treatment plans using a generation AI model based on the integrated data, means for transmitting the generated treatment plans to the terminal and displaying them to the user, means for the user to select the optimal plan from the displayed treatment plans, and means for notifying the server of the selected treatment plan and saving it. This enables healthcare professionals to efficiently generate and present multiple treatment plans within limited time and resources, and allows patients to make the optimal treatment choice based on informed consent.

[0086] A "user" is the person who inputs patient health data and operates the system.

[0087] A "terminal" is an electronic device used by the user to perform input tasks or select treatment plans.

[0088] A "server" is a central processing unit that receives data transmitted from terminals and performs data integration, transformation, analysis, and generation of treatment plans.

[0089] "Means of transmitting data from the terminal to the server" refers to the function that allows the terminal to electronically transmit the health data it has entered to the server.

[0090] "Data integration methods" refer to the processes used by a server to combine data received from different sources into a single dataset.

[0091] "Data conversion means" refers to the process by which a server converts the received and integrated data into a format suitable for subsequent processing.

[0092] A "generative AI model" is a machine learning-based model used by a server to generate multiple treatment plans based on integrated data.

[0093] "Means for transmitting and displaying treatment plans" refers to a function that transmits a treatment plan generated by the server to a terminal, and allows the terminal to display that plan to the user.

[0094] "Means of selecting a treatment plan" refers to a function that allows the user to select an appropriate treatment plan from the displayed options.

[0095] The "means for notifying and saving treatment plans" refer to a function that notifies the server of the selected treatment plan from the terminal, and the server saves that plan to a database.

[0096] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[0097] Data entry

[0098] The user first enters the patient's health data using a terminal. Specifically, they enter blood test results (e.g., red blood cell count, white blood cell count, platelet count, etc.), symptoms (e.g., fever, fatigue, etc.), and physical test results (e.g., heart rate, blood pressure, etc.). This data can be entered manually or imported electronically.

[0099] Sending and receiving data

[0100] When a user submits data through an input form on their device, the device sends it to the server. The server receives the HTTP request and retrieves the entered data. For example, TCP / IP or HTTP / HTTPS protocols are used for data transfer.

[0101] Data integration and transformation

[0102] The server integrates the received data and converts it into a format suitable for subsequent processing. Specifically, data integration tools such as Apache® Kafka and Apache NiFi are used to integrate data from different sources (blood test results and interview / physical examination results) and convert it into formats such as JSON. Data cleaning is also performed simultaneously to check for any omissions or inconsistencies.

[0103] Treatment plan generation

[0104] The server generates multiple treatment plans using a generative AI model based on integrated data. This generative AI model, for example, uses OpenAI's GPT-4® and is pre-tuned on medical datasets. This AI model generates the optimal treatment plan based on past performance and the latest medical research data.

[0105] Sending and displaying treatment plans

[0106] The treatment plan generated by the server is sent to the terminal and displayed to the user. The list of generated treatment plans is sent in JSON format, which the terminal parses and displays in list format on the user interface. Detailed information for each treatment plan (expected results, treatment duration, etc.) is also displayed.

[0107] Evaluation and selection of treatment plans

[0108] Users (healthcare professionals and patients) evaluate and select the most suitable treatment plan from the displayed options. Specifically, they evaluate plans based on criteria such as effectiveness, cost-effectiveness, and suitability for their lifestyle. The selected treatment plan is finalized by clicking a selection button on the device.

[0109] Final decision and implementation

[0110] The terminal notifies the server of the selected treatment plan, and the server saves this information to a database. Specific arrangements for treatment (such as medication procurement and follow-up schedules) are then made as needed.

[0111] Specific example

[0112] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into a terminal, and the server receives and integrates it. Next, a generative artificial intelligence model is used to generate multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary suggestions, etc.).

[0113] The generated treatment plan is displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. For example, if "iron supplementation therapy" is selected, the server records and saves this selection. The treatment is then implemented.

[0114] Example of a prompt

[0115] The following are some specific examples of prompt statements that can be used as input to a generative AI model.

[0116] Prompt: Based on the following patient data, propose the optimal treatment plan.

[0117] Patient data:

[0118] Fever: 2 weeks

[0119] Fatigue: 3 weeks

[0120] Blood test results: Red blood cell count: 3.8, White blood cell count: 5.5, Platelet count: 150

[0121] Symptoms: Mild anemia is suspected.

[0122] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[0123] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0124] Step 1: Data Entry

[0125] The user uses a terminal to input patient health data (blood test results, symptoms, physical test results, etc.). Specifically, the user either manually enters the necessary data into an input form on the terminal or imports the data using an electronic data import function. The input in this step is health data entered by the user, and the output is data temporarily stored on the terminal.

[0126] Step 2: Send

[0127] Once the user completes the input and clicks the submit button, the device sends the data to the server. Specifically, the device generates an HTTP request and sends it to the server along with the entered data. The input in this step is health data stored on the device, and the output is the data received by the server.

[0128] Step 3: Data Received

[0129] The server receives data sent from the terminal. Specifically, the server parses the HTTP request and verifies the received data. The input for this step is the data sent in the HTTP request, and the output is the received health data.

[0130] Step 4: Data Integration and Transformation

[0131] The server integrates the received data and converts it into a format. Specifically, the server combines data from different sources (such as blood test results and symptom data) into a single dataset and converts it to JSON format. It also performs data cleaning to check for any missing or inconsistent data. The input for this step is the received health data, and the output is the integrated JSON format data.

[0132] Step 5: Creating a Treatment Plan

[0133] The server inputs the integrated data into a generating AI model to generate multiple treatment plans. Specifically, the server generates prompts for the generating AI model (e.g., GPT-4), and inputs these prompts and the integrated data to generate treatment plans. The input for this step is integrated data in JSON format, and the output is the multiple treatment plans that have been generated.

[0134] Step 6: Submit and view treatment plan

[0135] The server generates a treatment plan and sends it to the terminal, which then displays it to the user. Specifically, the server sends a list of treatment plans to the terminal in JSON format, which the terminal parses and displays in list format on the user interface. The input to this step is the generated treatment plan data, and the output is the list of treatment plans displayed on the terminal.

[0136] Step 7: Evaluation and selection of treatment plan

[0137] Users (healthcare professionals and patients) evaluate the displayed treatment plans and select the optimal plan. Specifically, the user reviews the details of each treatment plan and selects the best one. They click a selection button to confirm their choice. The input for this step is the multiple displayed treatment plans, and the output is the selected treatment plan.

[0138] Step 8: Notify and save the treatment plan

[0139] The terminal notifies the server of the selected treatment plan, and the server saves it to the database. Specifically, after the terminal clicks the select button, it sends the selected plan to the server, and the server saves it to the database. The input to this step is the selected treatment plan, and the output is the data saved on the server.

[0140] (Application Example 1)

[0141] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0142] Current medical second opinion systems often require time and effort for patients and healthcare professionals to share detailed medical data face-to-face. Furthermore, there is a lack of visually clear means to present treatment plans, making it difficult to quickly obtain and present information, especially in clinical settings. This can delay the selection of the optimal treatment plan and potentially affect the patient's treatment outcomes. To solve this problem, a more efficient and intuitive system is needed.

[0143] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0144] In this invention, the server includes means for inputting blood test results via input means, means for inputting interview and physical examination results via input means, means for integrating the blood test results with the interview and physical examination results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to healthcare professionals and patients via display means, means for selecting the optimal treatment plan and saving it via final decision means, means for inputting health data via smart glasses and processing it via a cloud server, and means for visualizing the treatment plan on the smart glasses' display. This enables quick and intuitive data sharing and treatment plan selection in person, making it possible to provide the patient with the optimal treatment plan.

[0145] A "generative artificial intelligence device" is an artificial intelligence system that generates treatment plans based on a patient's health data.

[0146] A "medical second opinion" is the provision of an opinion from another medical professional to evaluate the appropriateness of the treatment a patient is currently receiving and to suggest alternative options.

[0147] "Health data" refers to information about a patient's health status, such as blood test results, interview results, and physical examination results.

[0148] "Blood test results" refer to data obtained by analyzing blood, such as the number of red blood cells, white blood cells, and platelets.

[0149] "Medical interview results" refer to information obtained from the patient themselves, such as their symptoms, medical history, and lifestyle.

[0150] "Palpation results" refer to information obtained through physical examinations, such as heart rate and blood pressure.

[0151] "Input means" refers to methods or devices for importing health data into a system.

[0152] "Means of integration" refers to methods or devices that combine multiple health data sets into a single dataset.

[0153] A "generative artificial intelligence model" is a model used by generative artificial intelligence devices that has been trained based on past medical data and the latest medical research.

[0154] A "treatment plan" is a plan that outlines the treatment policies and methods for a patient's specific health condition.

[0155] "Display means" refers to methods or devices for visually presenting the generated treatment plan to healthcare professionals and patients.

[0156] A "final decision-making tool" refers to a method or device for saving and implementing a selected treatment plan.

[0157] "Smart glasses" are glasses-type information processing devices that allow data input and display when worn.

[0158] A "cloud server" is a remote server that processes and stores data via the internet.

[0159] "Visualization" refers to presenting data and information in a way that is visible in the real world.

[0160] Modes for carrying out the invention

[0161] This invention relates to a system that provides medical second opinions using smart glasses and a cloud server. The system includes a series of processes for inputting patient health data, generating multiple treatment plans using a generative artificial intelligence model, and presenting them to healthcare professionals and patients in a visualized form.

[0162] Hardware and software configuration

[0163] Smart Glasses

[0164] Smart glasses are information processing devices worn by the user and have the following functions:

[0165] Input methods: It has a function to input health data using QR code (registered trademark) scanning or voice input.

[0166] Display method: It has the function of displaying and visualizing the generated treatment plan on a display screen.

[0167] Cloud Server

[0168] A cloud server is a remote service that integrates and processes data over the internet and has the following functions:

[0169] Data Integration and Processing: Blood test results, interview results, and palpation results received from smart glasses are integrated and compiled into a single dataset.

[0170] Execution of generative AI models: Based on integrated data, generative AI models are executed to generate multiple treatment plans.

[0171] Data saving: Has the function to ultimately save the selected treatment plan.

[0172] Program Processing Description

[0173] The server receives health data transmitted from the smart glasses and converts it into an appropriate format. It then uses a generative artificial intelligence model to generate a treatment plan, which is then sent back to the smart glasses.

[0174] The device (smart glasses) is used to scan and input health data and visually displays the generated treatment plan. Through the smart glasses' display, healthcare professionals and patients can review the treatment plan and select the most suitable option.

[0175] Specific hardware and software to be used

[0176] Smart glasses: A device for inputting and displaying information.

[0177] Cloud server: A remote server that performs data integration and runs generative artificial intelligence models.

[0178] Generative artificial intelligence models: AI models that generate treatment plans based on integrated data.

[0179] Specific example

[0180] For example, a patient wears smart glasses at the reception desk and scans QR codes containing their health data (e.g., blood test results, medical history, physical examination results). The scanned data is sent to a cloud server where a generative artificial intelligence model is executed. Multiple treatment plans generated by the AI ​​model are displayed on the smart glasses' screen. Healthcare professionals and patients review each treatment plan and select the optimal one, thereby proceeding with the treatment.

[0181] Example of a prompt

[0182] Input data

[0183] Blood test results: red blood cell count 4.5 million / μL, white blood cell count 7000 / μL, platelet count 150000 / μL

[0184] Symptoms: Fatigue, mild fever

[0185] Palpation results: Heart rate 80 bpm, blood pressure 120 / 80 mmHg

[0186] Content to generate

[0187] Based on the input data above, please propose several treatment plans. Each treatment plan should include details such as the treatment strategy, expected outcomes, duration of implementation, and frequency.

[0188] The above describes the embodiments for carrying out the present invention. This system enables quick and intuitive data sharing and treatment plan selection during face-to-face interactions, making it possible to provide patients with the optimal treatment plan.

[0189] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0190] Step 1:

[0191] The user wears smart glasses and scans QR codes containing health data (e.g., blood test results, medical questionnaire results, physical examination results). This input method uses the smart glasses' camera function. The QR codes encode various health data of the patient, which are then read and imported as data.

[0192] Step 2:

[0193] Smart glasses transmit scanned health data to a cloud server. This transmission utilizes an internet connection. The data output from the smart glasses is transferred to the cloud server, where it is integrated and processed.

[0194] Step 3:

[0195] The server receives and integrates data from the smart glasses. Before inputting the integrated dataset into a generative artificial intelligence model, it converts it to the required format. This data processing integrates blood test results, interview results, and physical examination results into a single dataset in the appropriate format.

[0196] Step 4:

[0197] The server inputs the integrated dataset into a generative artificial intelligence model to generate multiple treatment plans. This AI model is pre-trained on medical datasets and is designed to generate optimal treatment plans based on specific health conditions. In this processing step, the AI ​​model performs calculations based on the input data and outputs multiple treatment candidates.

[0198] Step 5:

[0199] The generated treatment plan is then sent back to the smart glasses via the cloud server. At this point, the data is visualized and converted into a format that can be viewed on the smart glasses.

[0200] Step 6:

[0201] The smart glasses display treatment plans received from the server. The displayed treatment plans include details of each treatment, expected outcomes, and estimated time required for implementation. Users (healthcare professionals and patients) can access this information through the smart glasses.

[0202] Step 7:

[0203] Healthcare professionals and patients evaluate multiple treatment plans displayed on smart glasses and select the optimal plan. The selected treatment plan is notified from the smart glasses to a cloud server, where the information is stored.

[0204] Step 8:

[0205] The server saves the selected treatment plan using a final decision-making mechanism and, if necessary, begins preparations to implement the treatment plan in detail. This saving process ensures that subsequent treatments and follow-ups proceed smoothly.

[0206] The above outlines the specific steps for implementing this invention. In each step, data collection, integration, generation, display, selection, and storage are carried out efficiently.

[0207] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0208] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for proposing an optimal treatment plan and adjusting the delivery method, taking the user's emotions into consideration.

[0209] Data entry

[0210] The user (medical staff) first enters blood test results using a terminal. Blood test results are numerical data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into an input form or imported electronically. The user also enters the results of the patient's interview and physical examination. This includes the symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.), which are also entered into a dedicated form.

[0211] Data integration and processing

[0212] The server receives blood test results, interview and physical examination results sent from the terminal. This data is integrated and compiled into a single dataset. This integrated data is then converted into a format suitable for subsequent analysis.

[0213] emotion recognition

[0214] The device incorporates an emotion engine that analyzes the user's voice, facial expressions, or biometric data (e.g., heart rate variability, skin electrical activity, etc.). This emotion engine recognizes the user's emotional state (e.g., stress, relief, anxiety, etc.) in real time and sends that data to a server.

[0215] Treatment plan generation

[0216] The server inputs integrated medical data and user emotional data into a generative artificial intelligence model to generate multiple treatment plans. This generative AI model is pre-tuned and trained on medical datasets. The model generates candidate treatment plans that are optimal for the patient's specific health and emotional state.

[0217] Display and selection of treatment plans

[0218] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). The terminal displays the treatment plan candidates in a list format, including detailed descriptions and expected outcomes for each treatment plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[0219] Users (patients and doctors) evaluate these treatment plans and select the optimal one. The evaluation is based on the effectiveness of the treatment, cost-effectiveness, and the user's emotional state and preferences.

[0220] Final decision and implementation

[0221] The user notifies the server of the selected treatment plan via their device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[0222] Specific example

[0223] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[0224] The generated treatment plans are displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. Each treatment plan is accompanied by appropriate comments from the emotion engine (e.g., "This treatment is gentle on the body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[0225] Thus, the present invention enables healthcare professionals to quickly provide and consider multiple highly accurate treatment plans within limited time and resources. Furthermore, because treatment plans are presented in an easily understandable format tailored to the patient's emotional state, it promotes treatment choices that are more based on informed consent.

[0226] The following describes the processing flow.

[0227] Step 1:

[0228] Users (medical staff) input blood test results using a terminal. Specifically, they manually enter numerical data such as red blood cell count, white blood cell count, and platelet count into an input form. Electronic data acquisition methods are also available.

[0229] Step 2:

[0230] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fatigue or fever) and physical examination results (e.g., heart rate or blood pressure). This information is manually entered into a dedicated input form.

[0231] Step 3:

[0232] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[0233] Step 4:

[0234] The server integrates the received blood test results with the results of the medical interview and physical examination. Specifically, it integrates the blood test results with the medical interview and physical examination results and combines them into a single dataset. The integrated data is then converted into an appropriate format.

[0235] Step 5:

[0236] The device recognizes the user's voice, facial expressions, or biometric data in real time and analyzes the user's emotional state using an emotion engine. For example, it recognizes emotions such as anxiety, relief, and stress based on voice recognition, facial expression analysis, and data from biosensors (such as heart rate fluctuations).

[0237] Step 6:

[0238] The server inputs integrated medical data and user sentiment data into a generative artificial intelligence model to generate numerous treatment plans. The generative AI model is pre-tuned and trained on medical datasets.

[0239] Step 7:

[0240] The server generates multiple treatment plans and sends them to the terminal. The terminal displays the candidate treatment plans in a list format, including detailed descriptions and expected outcomes for each plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[0241] Step 8:

[0242] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the user's emotional state and preferences.

[0243] Step 9:

[0244] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[0245] Step 10:

[0246] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[0247] (Example 2)

[0248] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0249] In modern medical practice, there is a demand for both speed and accuracy in developing treatment plans. However, within limited time and resources, it is difficult for healthcare professionals to process large amounts of data simultaneously and provide optimal treatment plans that take into account the patient's emotional state. As a result, it is difficult to reduce patient anxiety and stress during the selection of a treatment plan, and informed consent becomes challenging. Conventional technologies have not been able to generate treatment plans that take emotional states into account, which has placed a heavy burden on both patients and healthcare professionals.

[0250] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting blood test results by input means, means for inputting interview and palpation results by input means, means for integrating the blood test results with the interview and palpation results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to medical professionals and patients by display means, means for selecting the optimal treatment plan and saving it by final decision means, means for recognizing the user's emotional state by an emotion engine and processing the data, and means for providing appropriate comments and explanations according to the user's emotional state based on the generated treatment plan. As a result, medical professionals can not only provide treatment plans quickly and accurately, but also provide individualized responses according to the patient's emotional state, enabling the provision of better medical services.

[0251] A "generative artificial intelligence device" is a device that generates new information or results based on data provided by artificial intelligence technology.

[0252] A "medical second opinion" refers to a patient receiving a second opinion from another healthcare professional regarding the treatment or diagnosis they are currently receiving.

[0253] "Blood test results" is a general term for numerical values ​​and data obtained by analyzing a patient's blood, and includes measurement results such as red blood cell count, white blood cell count, and platelet count.

[0254] "Medical interview" refers to the act of a healthcare professional gathering information from a patient about their current symptoms, medical history, and other relevant details.

[0255] "Palpation" refers to the act of a medical professional examining a patient by touching their body with their hands.

[0256] "Integration" refers to the process of combining multiple different sets of data into one.

[0257] A "generative artificial intelligence model" refers to an artificial intelligence model that uses pre-trained algorithms to generate new information or results.

[0258] "Display means" refers to devices or methods that display information to a user through a computer screen, tablet, smartphone, etc.

[0259] "Healthcare professionals" refers to individuals engaged in medical-related professional occupations, such as doctors, nurses, and pharmacists.

[0260] The term "patient" refers to a person receiving medical services.

[0261] The term "final decision-making mechanism" refers to a system or process for storing and implementing selected results or information.

[0262] An "emotion engine" refers to a combination of software and hardware that analyzes a user's emotional state and outputs it as data.

[0263] "Means of providing comments and explanations" refers to devices or methods for adding appropriate explanations to the user based on the generated treatment plan.

[0264] A "database" refers to a system that centrally manages, stores, and allows data to be retrieved as needed.

[0265] "REST API" refers to an application programming interface based on Representational State Transfer, and is a standard for data communication.

[0266] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for generating an optimal treatment plan using medical data and user emotion data, and for providing that plan to healthcare professionals and patients.

[0267] Hardware configuration

[0268] Servers and terminals (e.g., PCs, tablets, smartphones) are the central hardware components of this system.

[0269] The server receives, integrates, processes, generates, and stores data, as well as treatment plans.

[0270] The device performs data input and display, as well as recognition and transmission of emotional data.

[0271] Software Configuration

[0272] The generative artificial intelligence model is embedded in a server and used to generate treatment plans from medical data. This model is fine-tuned in advance based on medical datasets.

[0273] The emotion engine is built into the device and analyzes the user's voice, facial expressions, and biometric data in real time. This data reflects the user's emotional state and is sent to the server.

[0274] System operation

[0275] The user (healthcare professional) first uses a terminal to input blood test results. This is done either manually or by importing from an electronic medical record system. Similarly, the results of the medical history interview and physical examination are also entered. This data is sent to the server and integrated.

[0276] The emotion engine built into the device analyzes the user's voice, facial expressions, heart rate fluctuations, etc., to generate emotion data. This data is also sent to a server and integrated with medical data.

[0277] The server inputs the integrated data into a generative artificial intelligence model to generate multiple treatment plans. These treatment plans are sent to the terminal and provided to the user. The displayed treatment plans include appropriate comments and explanations based on the user's emotional state as recognized by the emotion engine.

[0278] Users (healthcare professionals and patients) evaluate candidate treatment plans and select the optimal one. The selected treatment plan is notified to the server via the terminal and stored on the server. Subsequently, specific treatment is carried out based on this plan.

[0279] Specific example

[0280] For example, consider a case where a patient complains of fatigue and mild fever, and mild anemia is suspected from the blood test results. When the user inputs this data into the terminal, the server receives and integrates it. Next, based on the integrated data and the patient's emotion data recognized by the emotion engine (for example, a strong sense of uneasiness), the generative AI model generates multiple treatment plans (examples include iron supplementation therapy, vitamin B12 supplementation, and suggestions for dietary improvement). The generated treatment plans are displayed on the terminal, and the user checks the details of each plan and selects a specific approach. Each treatment plan is accompanied by appropriate comments from the emotion engine (e.g., This treatment is less burdensome on the body, so please feel at ease). The plan that is considered to be the most appropriate (for example, iron supplementation therapy) is selected, and the server records and saves this selection. After that, this treatment is put into practice.

[0281] Example of a prompt sentence

[0282] The following are examples of prompt sentences for input into the generative AI model:

[0283] "The current symptoms of the patient are fatigue and mild fever, and mild anemia is suspected from the blood test results. The patient is feeling strong uneasiness. Please propose an optimal treatment plan based on these."

[0284] "The patient's blood pressure and heart rate are within the normal range, but the white blood cell count is increasing. The patient is feeling stress. Please generate a treatment plan considering this situation."

[0285] The above are specific forms for implementing the present invention. With this system, medical staff can not only provide treatment plans quickly and with high accuracy, but also can provide individualized responses according to the patient's emotional state, enabling the realization of better medical services.

[0286] The flow of specific processing in Example 2 will be described using FIG. 13.

[0287] Step 1:

[0288] Users (medical staff) manually enter blood test results into a dedicated input form on the terminal. The data entered by the user includes numerical data such as red blood cell count, white blood cell count, and platelet count. If an electronic medical record (EMR) system exists, data can be automatically imported from the corresponding EMR system. This improves the accuracy and efficiency of data entry.

[0289] Step 2:

[0290] The user uses a terminal to input the results of the patient's medical history and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). The entered medical history and physical examination data is used for subsequent data integration and analysis.

[0291] Step 3:

[0292] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database on the server. The received data includes numerical and text data associated with each patient.

[0293] Step 4:

[0294] The server integrates the received blood test results, interview data, and physical examination results. Specifically, it converts data from different data sources into a standard format and combines them into a single dataset. This ensures that all medical information is presented in a consistently analyzable format.

[0295] Step 5:

[0296] An emotion engine built into the device recognizes the user's emotional state in real time. Emotion recognition uses data from voice, facial expressions, and biosensors (e.g., heart rate and skin electrical activity). The emotion engine analyzes this sensor data to estimate the user's emotional state (e.g., stress, reassurance, anxiety, etc.).

[0297] Step 6:

[0298] The terminal sends the recognized emotion data to the server. The emotion data to be sent includes specific emotional states and their intensities. These data become important elements in the subsequent generation of treatment plans.

[0299] Step 7:

[0300] The server inputs the integrated medical data and emotion data into the generative artificial intelligence model. This initiates the generation of a treatment plan based on the patient's specific health and emotional states. The input data includes the integrated medical dataset and the recognized emotion data.

[0301] Step 8:

[0302] The server uses the generative artificial intelligence model to generate multiple treatment plans. The generative artificial intelligence model has been pre-fine-tuned and learned based on a medical dataset. The generated treatment plans include specific methods and expected effects.

[0303] Step 9:

[0304] The server sends the generated treatment plans to the terminal. On the terminal, candidates for the treatment plans are displayed in list form, including detailed explanations and expected results for each treatment plan. Additionally, appropriate comments and explanations are added based on the recognized emotion data.

[0305] Step 10:

[0306] The user (patient and medical staff) checks the candidates for the treatment plans on the terminal. The user refers to the details of the treatment plans and evaluates the advantages and disadvantages of each plan.

[0307] Step 11:

[0308] The user selects the optimal treatment plan. Evaluation criteria include treatment effectiveness, cost-effectiveness, the user's emotional state, and the patient's wishes.

[0309] Step 12:

[0310] The user's selected treatment plan is notified to the server via the terminal. The server receives this information and saves the selected treatment plan to its database.

[0311] Step 13:

[0312] The server begins preparing for implementation based on the saved treatment plan. Specifically, it arranges for necessary medications and notifies relevant medical staff. This allows the selected treatment plan to be put into practice.

[0313] (Application Example 2)

[0314] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0315] Traditional medical second opinion systems propose treatment plans based on medical data, but they do not take into account the patient's emotional state. Therefore, it is difficult to determine whether the treatment plan is optimal for the patient, and there is a lack of consideration for reducing the patient's anxiety and stress. Furthermore, because relaxation content is not provided to patients undergoing treatment who require emotional support, there is a possibility that the overall effectiveness of the treatment will be reduced.

[0316] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0317] In this invention, the server includes means for inputting blood test results, means for inputting interview and physical examination results, means for analyzing the user's emotional state in real time using emotion recognition technology and integrating the data, means for utilizing a generative artificial intelligence model to generate multiple treatment plans, means for displaying the generated treatment plans to medical professionals and patients, and means for selecting and saving the optimal treatment plan. This makes it possible to provide personalized treatment plans and emotional support based on the user's medical and emotional data.

[0318] A "generative artificial intelligence device" is a device equipped with a model that takes medical data and emotional data as input and generates optimal treatment plans and relaxation content based on them.

[0319] A "medical second opinion" refers to a second expert medical opinion and treatment plan provided to a patient, intended to evaluate the patient's condition and treatment options from a different perspective.

[0320] "Emotion recognition technology" is a technology that identifies a user's emotional state by analyzing voice, facial expressions, or biometric data.

[0321] "Fine-tuning" is the process of fine-tuning a generative artificial intelligence model based on a specific dataset to improve its performance.

[0322] "Display means" refers to a device or interface for visually displaying the generated treatment plan or relaxation content.

[0323] "Relaxation content" refers to media content (such as meditation guides, calming music, and massage videos) designed to reduce patients' stress and anxiety and promote mental stability.

[0324] A "medical dataset" is a collection of data related to medical care, including diagnostic data, test results, and patient symptoms.

[0325] An "emotion dataset" is a collection of data that represents a user's emotional state, and includes data such as voice tone, facial expressions, and biometric data.

[0326] This invention is a system that combines a generative artificial intelligence device with emotion recognition technology to generate optimal treatment plans and relaxation content that take into account the patient's emotional state when providing a second medical opinion. A specific embodiment of this system is described in detail below.

[0327] System Configuration

[0328] This system consists of servers, terminals, and users.

[0329] 1. Enter blood test data

[0330] The terminal has a function for entering blood test results, which users (healthcare professionals) input using an input form or electronic means.

[0331] 2. Input of medical history and physical examination results

[0332] The terminal has a function to input the results of the medical questionnaire and physical examination, and this data is entered in the same way.

[0333] 3. Data Integration

[0334] The server receives blood test results, interview and physical examination results from the terminals, combines them into a single dataset, and converts them into an appropriate format.

[0335] 4. Emotion recognition

[0336] The device incorporates emotion recognition technology that analyzes voice, facial expressions, or biometric data to recognize the user's emotional state in real time. The results of this analysis are then sent to a server.

[0337] 5. Treatment planning and creation of relaxation content

[0338] The server inputs integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. This generative AI model has been pre-trained and fine-tuned using medical and emotional datasets.

[0339] 6. Display and Selection

[0340] The generated treatment plans and relaxation content are displayed on the device, allowing users (healthcare professionals and patients) to review them and select the most suitable plan and content.

[0341] 7. Preservation and Implementation

[0342] The selected treatment plan and relaxation content are notified to the server via the device, and the server stores this information and prepares for implementation as needed.

[0343] Program processing

[0344] The server analyzes blood test results and medical interview data, integrating the acquired data using emotion recognition technology. This process utilizes emotion recognition software such as EmotionEngine. The integrated data is input into a generative artificial intelligence model, which then generates treatment plans and relaxation content. For example, meditation guides, calming music, and massage videos are generated.

[0345] Specific example

[0346] For example, suppose we input health data for a user, such as "heart rate: 72, blood pressure: 120 / 80, symptoms: headache, fatigue," and emotional data such as "high anxiety." Based on this data, we input the following prompt into a generative artificial intelligence model:

[0347] User's health data: Heart rate 72, Blood pressure 120 / 80, Symptoms: Headache, Fatigue

[0348] User sentiment data: High levels of anxiety

[0349] What type of relaxation content is best?

[0350] Based on this prompt, the AI ​​suggests and displays optimal relaxation content (such as meditation guides or calming music) to the user.

[0351] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0352] Step 1:

[0353] The user (healthcare professional) uses a terminal to input blood test results. This can be done manually using an input form or electronically (e.g., by uploading a data file). The input includes numerical data such as red blood cell count, white blood cell count, and platelet count.

[0354] Input: Blood test result data

[0355] Output: Blood test result data stored on the device.

[0356] Step 2:

[0357] The user (healthcare professional) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). This data is also manually entered into a dedicated form.

[0358] Input: Medical history and physical examination results data

[0359] Output: Medical interview and palpation results data stored on the device.

[0360] Step 3:

[0361] The terminal integrates blood test results with interview and physical examination results and sends them to the server. This combines information from different data sources into a single dataset. The data is then converted to an appropriate format, such as JSON.

[0362] Input: Blood test results data, medical interview and physical examination results data

[0363] Output: Integrated dataset sent to the server

[0364] Step 4:

[0365] The device uses emotion recognition technology to analyze the user's voice, facial expressions, and biometric data (e.g., heart rate variability, skin electrical activity, etc.) in real time to recognize their emotional state. This data is also sent to the server.

[0366] Input: User's voice, facial expressions, biometric data

[0367] Output: Sentiment data sent to the server

[0368] Step 5:

[0369] The server inputs received integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. By generating prompt sentences and inputting them into the generative AI model, specific treatment plans and content suggestions are obtained.

[0370] Input: Integrative medical data, emotional data

[0371] Output: Generated treatment plan and relaxation content

[0372] Step 6:

[0373] The server sends the generated treatment plan and relaxation content to the device. This includes a detailed explanation of each treatment plan and expected results, as well as comments based on emotion recognition data.

[0374] Input: Generated treatment plan and relaxation content

[0375] Output: Treatment plan and relaxation content displayed on the device.

[0376] Step 7:

[0377] Users (healthcare professionals and patients) review the treatment plan and relaxation content displayed on their terminals and select the most suitable plan. This selection is then communicated to the server.

[0378] Input: Displayed treatment plan and relaxation content

[0379] Output: Selection results sent to the server

[0380] Step 8:

[0381] The server saves the selected treatment plan and relaxation content, and begins specific preparations for the treatment as needed. Details of the treatment plan will be communicated to the necessary stakeholders.

[0382] Input: User-selected treatment plan and relaxation content

[0383] Output: Saved treatment plan and relaxation content, start of treatment preparation

[0384] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0385] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0386] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0387] [Second Embodiment]

[0388] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0389] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0390] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0391] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0392] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0393] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0394] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0395] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0396] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0397] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0398] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0399] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0400] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[0401] Data entry

[0402] The user first enters blood test results using a terminal. These results include data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into the system's input form or imported electronically. The user also enters the results of the patient's medical history and physical examination. This includes the patient's symptoms (e.g., fever, fatigue) and physical examination results (e.g., heart rate, blood pressure).

[0403] Data integration and processing

[0404] The server receives blood test results and interview / palpation results sent from the terminal. The received data is integrated and compiled into a single dataset. Specifically, the blood test results and interview / palpation results are integrated and converted into a format suitable for subsequent processing.

[0405] Treatment plan generation

[0406] Based on the integrated data, the server uses a generative artificial intelligence model to generate multiple treatment plans. This model is pre-tuned and trained on past performance and the latest medical research data. This generates candidate treatment plans that are optimal for the patient's specific health condition.

[0407] Display and selection of treatment plans

[0408] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). On the terminal, multiple treatment plans are displayed in a list format, including a detailed description of each treatment plan and its expected outcome.

[0409] Users (patients and doctors) evaluate these treatment plans and select the optimal one. Options include highly effective treatments, cost-effective treatments, and treatments best suited to the patient's lifestyle and preferences.

[0410] Final decision and implementation

[0411] The selected treatment plan is communicated to the server via the user's device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[0412] Specific example

[0413] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, a generative artificial intelligence model generates multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[0414] The generated treatment plan is displayed on the terminal, allowing users (doctor and patient) to review the details of each plan and select a specific course of action. They choose the plan they deem most appropriate (for example, "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[0415] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[0416] The following describes the processing flow.

[0417] Step 1:

[0418] Users (medical staff) use a terminal to input blood test results. Blood test results consist of numerical data such as red blood cell count, white blood cell count, and platelet count. This data is either manually entered into an input form or imported into the system using an electronic import method.

[0419] Step 2:

[0420] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. The results of the medical interview and physical examination include the symptoms reported by the patient (e.g., fatigue, fever, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.). This information is also entered into a dedicated form.

[0421] Step 3:

[0422] The server receives blood test results and interview / physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[0423] Step 4:

[0424] The server integrates the received blood test results and the results of the medical interview and physical examination. The integrated data is converted into a format suitable for later analysis and compiled into a single dataset.

[0425] Step 5:

[0426] The server inputs the integrated data into a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. Using this model, multiple treatment plans are generated, tailored to the patient's specific health condition.

[0427] Step 6:

[0428] The server sends multiple treatment plans it has generated to the terminal. The terminal displays the candidate treatment plans in a list format, along with a detailed description of each plan and its expected outcome.

[0429] Step 7:

[0430] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the patient's lifestyle and preferences.

[0431] Step 8:

[0432] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[0433] Step 9:

[0434] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[0435] (Example 1)

[0436] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0437] In modern medicine, there is a need for systems that enable healthcare professionals to propose multiple treatment plans quickly and accurately within limited time and resources, allowing patients to choose the optimal treatment. However, conventional systems do not adequately provide mechanisms for integrating multiple data sources, automatically generating appropriate treatment plans, and ultimately enabling patients and doctors to efficiently select a treatment plan. This increases the burden on healthcare professionals and makes it difficult for patients to make appropriate treatment choices based on informed consent.

[0438] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0439] In this invention, the server includes means for a user to input patient health data using a terminal, means for transmitting data from the terminal to the server, means for the server to integrate the received data and convert it into a format, means for generating multiple treatment plans using a generation AI model based on the integrated data, means for transmitting the generated treatment plans to the terminal and displaying them to the user, means for the user to select the optimal plan from the displayed treatment plans, and means for notifying the server of the selected treatment plan and saving it. This enables healthcare professionals to efficiently generate and present multiple treatment plans within limited time and resources, and allows patients to make the optimal treatment choice based on informed consent.

[0440] A "user" is the person who inputs patient health data and operates the system.

[0441] A "terminal" is an electronic device used by the user to perform input tasks or select treatment plans.

[0442] A "server" is a central processing unit that receives data transmitted from terminals and performs data integration, transformation, analysis, and generation of treatment plans.

[0443] "Means of transmitting data from the terminal to the server" refers to the function that allows the terminal to electronically transmit the health data it has entered to the server.

[0444] "Data integration methods" refer to the processes used by a server to combine data received from different sources into a single dataset.

[0445] "Data conversion means" refers to the process by which a server converts the received and integrated data into a format suitable for subsequent processing.

[0446] A "generative AI model" is a machine learning-based model used by a server to generate multiple treatment plans based on integrated data.

[0447] "Means for transmitting and displaying treatment plans" refers to a function that transmits a treatment plan generated by the server to a terminal, and allows the terminal to display that plan to the user.

[0448] "Means of selecting a treatment plan" refers to a function that allows the user to select an appropriate treatment plan from the displayed options.

[0449] The "means for notifying and saving treatment plans" refer to a function that notifies the server of the selected treatment plan from the terminal, and the server saves that plan to a database.

[0450] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[0451] Data entry

[0452] The user first enters the patient's health data using a terminal. Specifically, they enter blood test results (e.g., red blood cell count, white blood cell count, platelet count, etc.), symptoms (e.g., fever, fatigue, etc.), and physical test results (e.g., heart rate, blood pressure, etc.). This data can be entered manually or imported electronically.

[0453] Sending and receiving data

[0454] When a user submits data through an input form on their device, the device sends it to the server. The server receives the HTTP request and retrieves the entered data. For example, TCP / IP or HTTP / HTTPS protocols are used for data transfer.

[0455] Data integration and transformation

[0456] The server integrates the received data and converts it into a format suitable for subsequent processing. Specifically, data integration tools such as Apache Kafka and Apache NiFi are used to integrate data from different sources (blood test results and interview / physical examination results) and convert it into formats such as JSON. Data cleaning is also performed simultaneously to check for any missing or inconsistent data.

[0457] Treatment plan generation

[0458] The server generates multiple treatment plans using a generative AI model based on integrated data. This generative AI model, for example, uses OpenAI's GPT-4 and is pre-tuned on medical datasets. This AI model generates the optimal treatment plan based on past performance and the latest medical research data.

[0459] Sending and displaying treatment plans

[0460] The treatment plan generated by the server is sent to the terminal and displayed to the user. The list of generated treatment plans is sent in JSON format, which the terminal parses and displays in list format on the user interface. Detailed information for each treatment plan (expected results, treatment duration, etc.) is also displayed.

[0461] Evaluation and selection of treatment plans

[0462] Users (healthcare professionals and patients) evaluate and select the most suitable treatment plan from the displayed options. Specifically, they evaluate plans based on criteria such as effectiveness, cost-effectiveness, and suitability for their lifestyle. The selected treatment plan is finalized by clicking a selection button on the device.

[0463] Final decision and implementation

[0464] The terminal notifies the server of the selected treatment plan, and the server saves this information to a database. Specific arrangements for treatment (such as medication procurement and follow-up schedules) are then made as needed.

[0465] Specific example

[0466] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into a terminal, and the server receives and integrates it. Next, a generative artificial intelligence model is used to generate multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary suggestions, etc.).

[0467] The generated treatment plan is displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. For example, if "iron supplementation therapy" is selected, the server records and saves this selection. The treatment is then implemented.

[0468] Example of a prompt

[0469] The following are some specific examples of prompt statements that can be used as input to a generative AI model.

[0470] Prompt: Based on the following patient data, propose the optimal treatment plan.

[0471] Patient data:

[0472] Fever: 2 weeks

[0473] Fatigue: 3 weeks

[0474] Blood test results: Red blood cell count: 3.8, White blood cell count: 5.5, Platelet count: 150

[0475] Symptoms: Mild anemia is suspected.

[0476] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[0477] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0478] Step 1: Data Entry

[0479] The user uses a terminal to input patient health data (blood test results, symptoms, physical test results, etc.). Specifically, the user either manually enters the necessary data into an input form on the terminal or imports the data using an electronic data import function. The input in this step is health data entered by the user, and the output is data temporarily stored on the terminal.

[0480] Step 2: Send

[0481] Once the user completes the input and clicks the submit button, the device sends the data to the server. Specifically, the device generates an HTTP request and sends it to the server along with the entered data. The input in this step is health data stored on the device, and the output is the data received by the server.

[0482] Step 3: Data Received

[0483] The server receives data sent from the terminal. Specifically, the server parses the HTTP request and verifies the received data. The input for this step is the data sent in the HTTP request, and the output is the received health data.

[0484] Step 4: Data Integration and Transformation

[0485] The server integrates the received data and converts it into a format. Specifically, the server combines data from different sources (such as blood test results and symptom data) into a single dataset and converts it to JSON format. It also performs data cleaning to check for any missing or inconsistent data. The input for this step is the received health data, and the output is the integrated JSON format data.

[0486] Step 5: Creating a Treatment Plan

[0487] The server inputs the integrated data into a generating AI model to generate multiple treatment plans. Specifically, the server generates prompts for the generating AI model (e.g., GPT-4), and inputs these prompts and the integrated data to generate treatment plans. The input for this step is integrated data in JSON format, and the output is the multiple treatment plans that have been generated.

[0488] Step 6: Submit and view treatment plan

[0489] The server generates a treatment plan and sends it to the terminal, which then displays it to the user. Specifically, the server sends a list of treatment plans to the terminal in JSON format, which the terminal parses and displays in list format on the user interface. The input to this step is the generated treatment plan data, and the output is the list of treatment plans displayed on the terminal.

[0490] Step 7: Evaluation and selection of treatment plan

[0491] Users (healthcare professionals and patients) evaluate the displayed treatment plans and select the optimal plan. Specifically, the user reviews the details of each treatment plan and selects the best one. They click a selection button to confirm their choice. The input for this step is the multiple displayed treatment plans, and the output is the selected treatment plan.

[0492] Step 8: Notify and save the treatment plan

[0493] The terminal notifies the server of the selected treatment plan, and the server saves it to the database. Specifically, after the terminal clicks the select button, it sends the selected plan to the server, and the server saves it to the database. The input to this step is the selected treatment plan, and the output is the data saved on the server.

[0494] (Application Example 1)

[0495] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0496] Current medical second opinion systems often require time and effort for patients and healthcare professionals to share detailed medical data face-to-face. Furthermore, there is a lack of visually clear means to present treatment plans, making it difficult to quickly obtain and present information, especially in clinical settings. This can delay the selection of the optimal treatment plan and potentially affect the patient's treatment outcomes. To solve this problem, a more efficient and intuitive system is needed.

[0497] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0498] In this invention, the server includes means for inputting blood test results via input means, means for inputting interview and physical examination results via input means, means for integrating the blood test results with the interview and physical examination results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to healthcare professionals and patients via display means, means for selecting the optimal treatment plan and saving it via final decision means, means for inputting health data via smart glasses and processing it via a cloud server, and means for visualizing the treatment plan on the smart glasses' display. This enables quick and intuitive data sharing and treatment plan selection in person, making it possible to provide the patient with the optimal treatment plan.

[0499] A "generative artificial intelligence device" is an artificial intelligence system that generates treatment plans based on a patient's health data.

[0500] A "medical second opinion" is the provision of an opinion from another medical professional to evaluate the appropriateness of the treatment a patient is currently receiving and to suggest alternative options.

[0501] "Health data" refers to information about a patient's health status, such as blood test results, interview results, and physical examination results.

[0502] "Blood test results" refer to data obtained by analyzing blood, such as the number of red blood cells, white blood cells, and platelets.

[0503] "Medical interview results" refer to information obtained from the patient themselves, such as their symptoms, medical history, and lifestyle.

[0504] "Palpation results" refer to information obtained through physical examinations, such as heart rate and blood pressure.

[0505] "Input means" refers to methods or devices for importing health data into a system.

[0506] "Means of integration" refers to methods or devices that combine multiple health data sets into a single dataset.

[0507] A "generative artificial intelligence model" is a model used by generative artificial intelligence devices that has been trained based on past medical data and the latest medical research.

[0508] A "treatment plan" is a plan that outlines the treatment policies and methods for a patient's specific health condition.

[0509] "Display means" refers to methods or devices for visually presenting the generated treatment plan to healthcare professionals and patients.

[0510] A "final decision-making tool" refers to a method or device for saving and implementing a selected treatment plan.

[0511] "Smart glasses" are glasses-type information processing devices that allow data input and display when worn.

[0512] A "cloud server" is a remote server that processes and stores data via the internet.

[0513] "Visualization" refers to presenting data and information in a way that is visible in the real world.

[0514] Modes for carrying out the invention

[0515] This invention relates to a system that provides medical second opinions using smart glasses and a cloud server. The system includes a series of processes for inputting patient health data, generating multiple treatment plans using a generative artificial intelligence model, and presenting them to healthcare professionals and patients in a visualized form.

[0516] Hardware and software configuration

[0517] Smart Glasses

[0518] Smart glasses are information processing devices worn by the user and have the following functions:

[0519] Input methods: It has a function to input health data using QR code scanning or voice input.

[0520] Display method: It has the function of displaying and visualizing the generated treatment plan on a display screen.

[0521] Cloud Server

[0522] A cloud server is a remote service that integrates and processes data over the internet and has the following functions:

[0523] Data Integration and Processing: Blood test results, interview results, and palpation results received from smart glasses are integrated and compiled into a single dataset.

[0524] Execution of generative AI models: Based on integrated data, generative AI models are executed to generate multiple treatment plans.

[0525] Data saving: Has the function to ultimately save the selected treatment plan.

[0526] Program Processing Description

[0527] The server receives health data transmitted from the smart glasses and converts it into an appropriate format. It then uses a generative artificial intelligence model to generate a treatment plan and returns it to the smart glasses.

[0528] The device (smart glasses) is used to scan and input health data and visually displays the generated treatment plan. Through the smart glasses' display, healthcare professionals and patients can review the treatment plan and select the most suitable option.

[0529] Specific hardware and software to be used

[0530] Smart glasses: A device for inputting and displaying information.

[0531] Cloud server: A remote server that performs data integration and runs generative artificial intelligence models.

[0532] Generative artificial intelligence models: AI models that generate treatment plans based on integrated data.

[0533] Specific example

[0534] For example, a patient wears smart glasses at the reception desk and scans QR codes containing their health data (e.g., blood test results, medical history, physical examination results). The scanned data is sent to a cloud server where a generative artificial intelligence model is executed. Multiple treatment plans generated by the AI ​​model are displayed on the smart glasses' screen. Healthcare professionals and patients review each treatment plan and select the optimal one, thereby proceeding with the treatment.

[0535] Example of a prompt

[0536] Input data

[0537] Blood test results: red blood cell count 4.5 million / μL, white blood cell count 7000 / μL, platelet count 150000 / μL

[0538] Symptoms: Fatigue, mild fever

[0539] Palpation results: Heart rate 80 bpm, blood pressure 120 / 80 mmHg

[0540] Content to generate

[0541] Based on the input data above, please propose several treatment plans. Each treatment plan should include details such as the treatment strategy, expected outcomes, duration of implementation, and frequency.

[0542] The above describes the embodiments for carrying out the present invention. This system enables quick and intuitive data sharing and treatment plan selection during face-to-face interactions, making it possible to provide patients with the optimal treatment plan.

[0543] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0544] Step 1:

[0545] The user wears smart glasses and scans QR codes containing health data (e.g., blood test results, medical questionnaire results, physical examination results). This input method uses the smart glasses' camera function. The QR codes encode various health data of the patient, which are then read and imported as data.

[0546] Step 2:

[0547] Smart glasses transmit scanned health data to a cloud server. This transmission utilizes an internet connection. The data output from the smart glasses is transferred to the cloud server, where it is integrated and processed.

[0548] Step 3:

[0549] The server receives and integrates data from the smart glasses. Before inputting the integrated dataset into a generative artificial intelligence model, it converts it to the required format. This data processing integrates blood test results, interview results, and physical examination results into a single dataset in the appropriate format.

[0550] Step 4:

[0551] The server inputs the integrated dataset into a generative artificial intelligence model to generate multiple treatment plans. This AI model is pre-trained on medical datasets and is designed to generate optimal treatment plans based on specific health conditions. In this processing step, the AI ​​model performs calculations based on the input data and outputs multiple treatment candidates.

[0552] Step 5:

[0553] The generated treatment plan is then sent back to the smart glasses via the cloud server. At this point, the data is visualized and converted into a format that can be viewed on the smart glasses.

[0554] Step 6:

[0555] The smart glasses display treatment plans received from the server. The displayed treatment plans include details of each treatment, expected outcomes, and estimated time required for implementation. Users (healthcare professionals and patients) can access this information through the smart glasses.

[0556] Step 7:

[0557] Healthcare professionals and patients evaluate multiple treatment plans displayed on smart glasses and select the optimal plan. The selected treatment plan is notified from the smart glasses to a cloud server, where the information is stored.

[0558] Step 8:

[0559] The server saves the selected treatment plan using a final decision-making mechanism and, if necessary, begins preparations to implement the treatment plan in detail. This saving process ensures that subsequent treatments and follow-ups proceed smoothly.

[0560] The above outlines the specific steps for implementing this invention. In each step, data collection, integration, generation, display, selection, and storage are carried out efficiently.

[0561] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0562] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for proposing an optimal treatment plan and adjusting the delivery method, taking the user's emotions into consideration.

[0563] Data entry

[0564] The user (medical staff) first enters blood test results using a terminal. Blood test results are numerical data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into an input form or imported electronically. The user also enters the results of the patient's interview and physical examination. This includes the symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.), which are also entered into a dedicated form.

[0565] Data integration and processing

[0566] The server receives blood test results, interview and physical examination results sent from the terminal. This data is integrated and compiled into a single dataset. This integrated data is then converted into a format suitable for subsequent analysis.

[0567] emotion recognition

[0568] The device incorporates an emotion engine that analyzes the user's voice, facial expressions, or biometric data (e.g., heart rate variability, skin electrical activity, etc.). This emotion engine recognizes the user's emotional state (e.g., stress, relief, anxiety, etc.) in real time and sends that data to a server.

[0569] Treatment plan generation

[0570] The server inputs integrated medical data and user emotional data into a generative artificial intelligence model to generate multiple treatment plans. This generative AI model is pre-tuned and trained on medical datasets. The model generates candidate treatment plans that are optimal for the patient's specific health and emotional state.

[0571] Display and selection of treatment plans

[0572] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). The terminal displays the treatment plan candidates in a list format, including detailed descriptions and expected outcomes for each treatment plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[0573] Users (patients and doctors) evaluate these treatment plans and select the optimal one. The evaluation is based on the effectiveness of the treatment, cost-effectiveness, and the user's emotional state and preferences.

[0574] Final decision and implementation

[0575] The user notifies the server of the selected treatment plan via their device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[0576] Specific example

[0577] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[0578] The generated treatment plans are displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. Each treatment plan is accompanied by appropriate comments from the emotion engine (e.g., "This treatment is gentle on the body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[0579] Thus, the present invention enables healthcare professionals to quickly provide and consider multiple highly accurate treatment plans within limited time and resources. Furthermore, because treatment plans are presented in an easily understandable format tailored to the patient's emotional state, it promotes treatment choices that are more based on informed consent.

[0580] The following describes the processing flow.

[0581] Step 1:

[0582] Users (medical staff) input blood test results using a terminal. Specifically, they manually enter numerical data such as red blood cell count, white blood cell count, and platelet count into an input form. Electronic data acquisition methods are also available.

[0583] Step 2:

[0584] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fatigue or fever) and physical examination results (e.g., heart rate or blood pressure). This information is manually entered into a dedicated input form.

[0585] Step 3:

[0586] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[0587] Step 4:

[0588] The server integrates the received blood test results with the results of the medical interview and physical examination. Specifically, it integrates the blood test results with the medical interview and physical examination results and combines them into a single dataset. The integrated data is then converted into an appropriate format.

[0589] Step 5:

[0590] The device recognizes the user's voice, facial expressions, or biometric data in real time and analyzes the user's emotional state using an emotion engine. For example, it recognizes emotions such as anxiety, relief, and stress based on voice recognition, facial expression analysis, and data from biosensors (such as heart rate fluctuations).

[0591] Step 6:

[0592] The server inputs integrated medical data and user sentiment data into a generative artificial intelligence model to generate numerous treatment plans. The generative AI model is pre-tuned and trained on medical datasets.

[0593] Step 7:

[0594] The server generates multiple treatment plans and sends them to the terminal. The terminal displays the candidate treatment plans in a list format, including detailed descriptions and expected outcomes for each plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[0595] Step 8:

[0596] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the user's emotional state and preferences.

[0597] Step 9:

[0598] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[0599] Step 10:

[0600] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[0601] (Example 2)

[0602] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0603] In modern medical practice, there is a demand for both speed and accuracy in developing treatment plans. However, within limited time and resources, it is difficult for healthcare professionals to process large amounts of data simultaneously and provide optimal treatment plans that take into account the patient's emotional state. As a result, it is difficult to reduce patient anxiety and stress during the selection of a treatment plan, and informed consent becomes challenging. Conventional technologies have not been able to generate treatment plans that take emotional states into account, which has placed a heavy burden on both patients and healthcare professionals.

[0604] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting blood test results by input means, means for inputting interview and palpation results by input means, means for integrating the blood test results with the interview and palpation results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to medical professionals and patients by display means, means for selecting the optimal treatment plan and saving it by final decision means, means for recognizing the user's emotional state by an emotion engine and processing the data, and means for providing appropriate comments and explanations according to the user's emotional state based on the generated treatment plan. As a result, medical professionals can not only provide treatment plans quickly and accurately, but also provide individualized responses according to the patient's emotional state, enabling the provision of better medical services.

[0605] A "generative artificial intelligence device" is a device that generates new information or results based on data provided by artificial intelligence technology.

[0606] A "medical second opinion" refers to a patient receiving a second opinion from another healthcare professional regarding the treatment or diagnosis they are currently receiving.

[0607] "Blood test results" is a general term for numerical values ​​and data obtained by analyzing a patient's blood, and includes measurement results such as red blood cell count, white blood cell count, and platelet count.

[0608] "Medical interview" refers to the act of a healthcare professional gathering information from a patient about their current symptoms, medical history, and other relevant details.

[0609] "Palpation" refers to the act of a medical professional examining a patient by touching their body with their hands.

[0610] "Integration" refers to the process of combining multiple different sets of data into one.

[0611] A "generative artificial intelligence model" refers to an artificial intelligence model that uses pre-trained algorithms to generate new information or results.

[0612] "Display means" refers to devices or methods that display information to a user through a computer screen, tablet, smartphone, etc.

[0613] "Healthcare professionals" refers to individuals engaged in medical-related professional occupations, such as doctors, nurses, and pharmacists.

[0614] The term "patient" refers to a person receiving medical services.

[0615] The term "final decision-making mechanism" refers to a system or process for storing and implementing selected results or information.

[0616] An "emotion engine" refers to a combination of software and hardware that analyzes a user's emotional state and outputs it as data.

[0617] "Means of providing comments and explanations" refers to devices or methods for adding appropriate explanations to the user based on the generated treatment plan.

[0618] A "database" refers to a system that centrally manages, stores, and allows data to be retrieved as needed.

[0619] "REST API" refers to an application programming interface based on Representational State Transfer, and is a standard for data communication.

[0620] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for generating an optimal treatment plan using medical data and user emotion data, and for providing that plan to healthcare professionals and patients.

[0621] Hardware configuration

[0622] Servers and terminals (e.g., PCs, tablets, smartphones) are the central hardware components of this system.

[0623] The server receives, integrates, processes, generates, and stores data, as well as treatment plans.

[0624] The device performs data input and display, as well as recognition and transmission of emotional data.

[0625] Software Configuration

[0626] The generative artificial intelligence model is embedded in a server and used to generate treatment plans from medical data. This model is fine-tuned in advance based on medical datasets.

[0627] The emotion engine is built into the device and analyzes the user's voice, facial expressions, and biometric data in real time. This data reflects the user's emotional state and is sent to the server.

[0628] System operation

[0629] The user (healthcare professional) first uses a terminal to input blood test results. This is done either manually or by importing from an electronic medical record system. Similarly, the results of the medical history interview and physical examination are also entered. This data is sent to the server and integrated.

[0630] The emotion engine built into the device analyzes the user's voice, facial expressions, heart rate fluctuations, etc., to generate emotion data. This data is also sent to a server and integrated with medical data.

[0631] The server inputs the integrated data into a generative artificial intelligence model to generate multiple treatment plans. These treatment plans are sent to the terminal and provided to the user. The displayed treatment plans include appropriate comments and explanations based on the user's emotional state as recognized by the emotion engine.

[0632] Users (healthcare professionals and patients) evaluate candidate treatment plans and select the optimal one. The selected treatment plan is notified to the server via the terminal and stored on the server. Subsequently, specific treatment is carried out based on this plan.

[0633] Specific example

[0634] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into the terminal, and the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvements, etc.). The generated treatment plans are displayed on the terminal, and the user reviews the details of each plan and selects a specific course of action. Each treatment plan is accompanied by an appropriate comment from the emotion engine (e.g., "This treatment is not burdensome on your body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., iron supplementation), and the server records and saves this selection. Subsequently, this treatment is implemented.

[0635] Example of a prompt

[0636] The following are examples of prompts to input into a generative AI model:

[0637] "The patient's current symptoms are fatigue and a mild fever, and blood test results suggest mild anemia. The patient is experiencing significant anxiety. Please propose an optimal treatment plan based on these factors."

[0638] "The patient's blood pressure and heart rate are within the normal range, but their white blood cell count is elevated. The patient is experiencing stress. Please generate a treatment plan considering this situation."

[0639] The above describes a specific embodiment for carrying out the present invention. This system not only enables healthcare professionals to provide rapid and highly accurate treatment plans, but also allows for individualized responses tailored to the patient's emotional state, thereby enabling the provision of better medical services.

[0640] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0641] Step 1:

[0642] Users (medical staff) manually enter blood test results into a dedicated input form on the terminal. The data entered by the user includes numerical data such as red blood cell count, white blood cell count, and platelet count. If an electronic medical record (EMR) system exists, data can be automatically imported from the corresponding EMR system. This improves the accuracy and efficiency of data entry.

[0643] Step 2:

[0644] The user uses a terminal to input the results of the patient's medical history and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). The entered medical history and physical examination data is used for subsequent data integration and analysis.

[0645] Step 3:

[0646] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database on the server. The received data includes numerical and text data associated with each patient.

[0647] Step 4:

[0648] The server integrates the received blood test results, interview data, and physical examination results. Specifically, it converts data from different data sources into a standard format and combines them into a single dataset. This ensures that all medical information is presented in a consistently analyzable format.

[0649] Step 5:

[0650] An emotion engine built into the device recognizes the user's emotional state in real time. Emotion recognition uses data from voice, facial expressions, and biosensors (e.g., heart rate and skin electrical activity). The emotion engine analyzes this sensor data to estimate the user's emotional state (e.g., stress, reassurance, anxiety, etc.).

[0651] Step 6:

[0652] The device transmits recognized emotional data to the server. This transmitted emotional data includes specific emotional states and their intensity. This data becomes an important element in the subsequent generation of treatment plans.

[0653] Step 7:

[0654] The server inputs integrated medical and emotional data into a generative artificial intelligence model. This initiates the generation of a treatment plan based on the patient's specific health and emotional state. The input data includes an integrated medical dataset and recognized emotional data.

[0655] Step 8:

[0656] The server generates multiple treatment plans using a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. The generated treatment plans include specific methods and expected effects.

[0657] Step 9:

[0658] The server generates a treatment plan and sends it to the terminal. The terminal displays a list of potential treatment plans, each containing a detailed explanation and expected outcomes. Furthermore, appropriate comments and explanations are added based on recognized emotional data.

[0659] Step 10:

[0660] Users (patients and healthcare professionals) review potential treatment plans on their devices. Users then view the details of each treatment plan and evaluate the advantages and disadvantages of each plan.

[0661] Step 11:

[0662] The user selects the optimal treatment plan. Evaluation criteria include treatment effectiveness, cost-effectiveness, the user's emotional state, and the patient's wishes.

[0663] Step 12:

[0664] The user's selected treatment plan is notified to the server via the terminal. The server receives this information and saves the selected treatment plan to its database.

[0665] Step 13:

[0666] The server begins preparing for implementation based on the saved treatment plan. Specifically, it arranges for necessary medications and notifies relevant medical staff. This allows the selected treatment plan to be put into practice.

[0667] (Application Example 2)

[0668] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0669] Traditional medical second opinion systems propose treatment plans based on medical data, but they do not take into account the patient's emotional state. Therefore, it is difficult to determine whether the treatment plan is optimal for the patient, and there is a lack of consideration for reducing the patient's anxiety and stress. Furthermore, because relaxation content is not provided to patients undergoing treatment who require emotional support, there is a possibility that the overall effectiveness of the treatment will be reduced.

[0670] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0671] In this invention, the server includes means for inputting blood test results, means for inputting interview and physical examination results, means for analyzing the user's emotional state in real time using emotion recognition technology and integrating the data, means for utilizing a generative artificial intelligence model to generate multiple treatment plans, means for displaying the generated treatment plans to medical professionals and patients, and means for selecting and saving the optimal treatment plan. This makes it possible to provide personalized treatment plans and emotional support based on the user's medical and emotional data.

[0672] A "generative artificial intelligence device" is a device equipped with a model that takes medical data and emotional data as input and generates optimal treatment plans and relaxation content based on them.

[0673] A "medical second opinion" refers to a second expert medical opinion and treatment plan provided to a patient, intended to evaluate the patient's condition and treatment options from a different perspective.

[0674] "Emotion recognition technology" is a technology that identifies a user's emotional state by analyzing voice, facial expressions, or biometric data.

[0675] "Fine-tuning" is the process of fine-tuning a generative artificial intelligence model based on a specific dataset to improve its performance.

[0676] "Display means" refers to a device or interface for visually displaying the generated treatment plan or relaxation content.

[0677] "Relaxation content" refers to media content (such as meditation guides, calming music, and massage videos) designed to reduce patients' stress and anxiety and promote mental stability.

[0678] A "medical dataset" is a collection of data related to medical care, including diagnostic data, test results, and patient symptoms.

[0679] An "emotion dataset" is a collection of data that represents a user's emotional state, and includes data such as voice tone, facial expressions, and biometric data.

[0680] This invention is a system that combines a generative artificial intelligence device with emotion recognition technology to generate optimal treatment plans and relaxation content that take into account the patient's emotional state when providing a second medical opinion. A specific embodiment of this system is described in detail below.

[0681] System Configuration

[0682] This system consists of servers, terminals, and users.

[0683] 1. Enter blood test data

[0684] The terminal has a function for entering blood test results, which users (healthcare professionals) input using an input form or electronic means.

[0685] 2. Input of medical history and physical examination results

[0686] The terminal has a function to input the results of the medical questionnaire and physical examination, and this data is entered in the same way.

[0687] 3. Data Integration

[0688] The server receives blood test results, interview and physical examination results from the terminals, combines them into a single dataset, and converts them into an appropriate format.

[0689] 4. Emotion recognition

[0690] The device incorporates emotion recognition technology that analyzes voice, facial expressions, or biometric data to recognize the user's emotional state in real time. The results of this analysis are then sent to a server.

[0691] 5. Treatment planning and creation of relaxation content

[0692] The server inputs integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. This generative AI model has been pre-trained and fine-tuned using medical and emotional datasets.

[0693] 6. Display and Selection

[0694] The generated treatment plans and relaxation content are displayed on the device, allowing users (healthcare professionals and patients) to review them and select the most suitable plan and content.

[0695] 7. Preservation and Implementation

[0696] The selected treatment plan and relaxation content are notified to the server via the device, and the server stores this information and prepares for implementation as needed.

[0697] Program processing

[0698] The server analyzes blood test results and medical interview data, integrating the acquired data using emotion recognition technology. This process utilizes emotion recognition software such as EmotionEngine. The integrated data is input into a generative artificial intelligence model, which then generates treatment plans and relaxation content. For example, meditation guides, calming music, and massage videos are generated.

[0699] Specific example

[0700] For example, suppose we input health data for a user, such as "heart rate: 72, blood pressure: 120 / 80, symptoms: headache, fatigue," and emotional data such as "high anxiety." Based on this data, we input the following prompt into a generative artificial intelligence model:

[0701] User's health data: Heart rate 72, Blood pressure 120 / 80, Symptoms: Headache, Fatigue

[0702] User sentiment data: High levels of anxiety

[0703] What type of relaxation content is best?

[0704] Based on this prompt, the AI ​​suggests and displays optimal relaxation content (such as meditation guides or calming music) to the user.

[0705] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0706] Step 1:

[0707] The user (healthcare professional) uses a terminal to input blood test results. This can be done manually using an input form or electronically (e.g., by uploading a data file). The input includes numerical data such as red blood cell count, white blood cell count, and platelet count.

[0708] Input: Blood test result data

[0709] Output: Blood test result data stored on the device.

[0710] Step 2:

[0711] The user (healthcare professional) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). This data is also manually entered into a dedicated form.

[0712] Input: Medical history and physical examination results data

[0713] Output: Medical interview and palpation results data stored on the device.

[0714] Step 3:

[0715] The terminal integrates blood test results with interview and physical examination results and sends them to the server. This combines information from different data sources into a single dataset. The data is then converted to an appropriate format, such as JSON.

[0716] Input: Blood test results data, medical interview and physical examination results data

[0717] Output: Integrated dataset sent to the server

[0718] Step 4:

[0719] The device uses emotion recognition technology to analyze the user's voice, facial expressions, and biometric data (e.g., heart rate variability, skin electrical activity, etc.) in real time to recognize their emotional state. This data is also sent to the server.

[0720] Input: User's voice, facial expressions, biometric data

[0721] Output: Sentiment data sent to the server

[0722] Step 5:

[0723] The server inputs received integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. By generating prompt sentences and inputting them into the generative AI model, specific treatment plans and content suggestions are obtained.

[0724] Input: Integrative medical data, emotional data

[0725] Output: Generated treatment plan and relaxation content

[0726] Step 6:

[0727] The server sends the generated treatment plan and relaxation content to the device. This includes a detailed explanation of each treatment plan and expected results, as well as comments based on emotion recognition data.

[0728] Input: Generated treatment plan and relaxation content

[0729] Output: Treatment plan and relaxation content displayed on the device.

[0730] Step 7:

[0731] Users (healthcare professionals and patients) review the treatment plan and relaxation content displayed on their terminals and select the most suitable plan. This selection is then communicated to the server.

[0732] Input: Displayed treatment plan and relaxation content

[0733] Output: Selection results sent to the server

[0734] Step 8:

[0735] The server saves the selected treatment plan and relaxation content, and begins specific preparations for the treatment as needed. Details of the treatment plan will be communicated to the necessary stakeholders.

[0736] Input: User-selected treatment plan and relaxation content

[0737] Output: Saved treatment plan and relaxation content, start of treatment preparation

[0738] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0739] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0740] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0741] [Third Embodiment]

[0742] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0743] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0744] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0745] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0746] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0747] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0748] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0749] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0750] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0751] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0752] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0753] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0754] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[0755] Data entry

[0756] The user first enters blood test results using a terminal. These results include data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into the system's input form or imported electronically. The user also enters the results of the patient's medical history and physical examination. This includes the patient's symptoms (e.g., fever, fatigue) and physical examination results (e.g., heart rate, blood pressure).

[0757] Data integration and processing

[0758] The server receives blood test results and interview / palpation results sent from the terminal. The received data is integrated and compiled into a single dataset. Specifically, the blood test results and interview / palpation results are integrated and converted into a format suitable for subsequent processing.

[0759] Treatment plan generation

[0760] Based on the integrated data, the server uses a generative artificial intelligence model to generate multiple treatment plans. This model is pre-tuned and trained on past performance and the latest medical research data. This generates candidate treatment plans that are optimal for the patient's specific health condition.

[0761] Display and selection of treatment plans

[0762] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). On the terminal, multiple treatment plans are displayed in a list format, including a detailed description of each treatment plan and its expected outcome.

[0763] Users (patients and doctors) evaluate these treatment plans and select the optimal one. Options include highly effective treatments, cost-effective treatments, and treatments best suited to the patient's lifestyle and preferences.

[0764] Final decision and implementation

[0765] The selected treatment plan is communicated to the server via the user's device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[0766] Specific example

[0767] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, a generative artificial intelligence model generates multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[0768] The generated treatment plan is displayed on the terminal, allowing users (doctor and patient) to review the details of each plan and select a specific course of action. They choose the plan they deem most appropriate (for example, "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[0769] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[0770] The following describes the processing flow.

[0771] Step 1:

[0772] Users (medical staff) use a terminal to input blood test results. Blood test results consist of numerical data such as red blood cell count, white blood cell count, and platelet count. This data is either manually entered into an input form or imported into the system using an electronic import method.

[0773] Step 2:

[0774] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. The results of the medical interview and physical examination include the symptoms reported by the patient (e.g., fatigue, fever, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.). This information is also entered into a dedicated form.

[0775] Step 3:

[0776] The server receives blood test results and interview / physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[0777] Step 4:

[0778] The server integrates the received blood test results and the results of the medical interview and physical examination. The integrated data is converted into a format suitable for later analysis and compiled into a single dataset.

[0779] Step 5:

[0780] The server inputs the integrated data into a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. Using this model, multiple treatment plans are generated, tailored to the patient's specific health condition.

[0781] Step 6:

[0782] The server sends multiple treatment plans it has generated to the terminal. The terminal displays the candidate treatment plans in a list format, along with a detailed description of each plan and its expected outcome.

[0783] Step 7:

[0784] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the patient's lifestyle and preferences.

[0785] Step 8:

[0786] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[0787] Step 9:

[0788] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[0789] (Example 1)

[0790] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0791] In modern medicine, there is a need for systems that enable healthcare professionals to propose multiple treatment plans quickly and accurately within limited time and resources, allowing patients to choose the optimal treatment. However, conventional systems do not adequately provide mechanisms for integrating multiple data sources, automatically generating appropriate treatment plans, and ultimately enabling patients and doctors to efficiently select a treatment plan. This increases the burden on healthcare professionals and makes it difficult for patients to make appropriate treatment choices based on informed consent.

[0792] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0793] In this invention, the server includes means for a user to input patient health data using a terminal, means for transmitting data from the terminal to the server, means for the server to integrate the received data and convert it into a format, means for generating multiple treatment plans using a generation AI model based on the integrated data, means for transmitting the generated treatment plans to the terminal and displaying them to the user, means for the user to select the optimal plan from the displayed treatment plans, and means for notifying the server of the selected treatment plan and saving it. This enables healthcare professionals to efficiently generate and present multiple treatment plans within limited time and resources, and allows patients to make the optimal treatment choice based on informed consent.

[0794] A "user" is the person who inputs patient health data and operates the system.

[0795] A "terminal" is an electronic device used by the user to perform input tasks or select treatment plans.

[0796] A "server" is a central processing unit that receives data transmitted from terminals and performs data integration, transformation, analysis, and generation of treatment plans.

[0797] "Means of transmitting data from the terminal to the server" refers to the function that allows the terminal to electronically transmit the health data it has entered to the server.

[0798] "Data integration methods" refer to the processes used by a server to combine data received from different sources into a single dataset.

[0799] "Data conversion means" refers to the process by which a server converts the received and integrated data into a format suitable for subsequent processing.

[0800] A "generative AI model" is a machine learning-based model used by a server to generate multiple treatment plans based on integrated data.

[0801] "Means for transmitting and displaying treatment plans" refers to a function that transmits a treatment plan generated by the server to a terminal, and allows the terminal to display that plan to the user.

[0802] "Means of selecting a treatment plan" refers to a function that allows the user to select an appropriate treatment plan from the displayed options.

[0803] The "means for notifying and saving treatment plans" refer to a function that notifies the server of the selected treatment plan from the terminal, and the server saves that plan to a database.

[0804] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[0805] Data entry

[0806] The user first enters the patient's health data using a terminal. Specifically, they enter blood test results (e.g., red blood cell count, white blood cell count, platelet count, etc.), symptoms (e.g., fever, fatigue, etc.), and physical test results (e.g., heart rate, blood pressure, etc.). This data can be entered manually or imported electronically.

[0807] Sending and receiving data

[0808] When a user submits data through an input form on their device, the device sends it to the server. The server receives the HTTP request and retrieves the entered data. For example, TCP / IP or HTTP / HTTPS protocols are used for data transfer.

[0809] Data integration and transformation

[0810] The server integrates the received data and converts it into a format suitable for subsequent processing. Specifically, data integration tools such as Apache Kafka and Apache NiFi are used to integrate data from different sources (blood test results and interview / physical examination results) and convert it into formats such as JSON. Data cleaning is also performed simultaneously to check for any missing or inconsistent data.

[0811] Treatment plan generation

[0812] The server generates multiple treatment plans using a generative AI model based on integrated data. This generative AI model, for example, uses OpenAI's GPT-4 and is pre-tuned on medical datasets. This AI model generates the optimal treatment plan based on past performance and the latest medical research data.

[0813] Sending and displaying treatment plans

[0814] The treatment plan generated by the server is sent to the terminal and displayed to the user. The list of generated treatment plans is sent in JSON format, which the terminal parses and displays in list format on the user interface. Detailed information for each treatment plan (expected results, treatment duration, etc.) is also displayed.

[0815] Evaluation and selection of treatment plans

[0816] Users (healthcare professionals and patients) evaluate and select the most suitable treatment plan from the displayed options. Specifically, they evaluate plans based on criteria such as effectiveness, cost-effectiveness, and suitability for their lifestyle. The selected treatment plan is finalized by clicking a selection button on the device.

[0817] Final decision and implementation

[0818] The terminal notifies the server of the selected treatment plan, and the server saves this information to a database. Specific arrangements for treatment (such as medication procurement and follow-up schedules) are then made as needed.

[0819] Specific example

[0820] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into a terminal, and the server receives and integrates it. Next, a generative artificial intelligence model is used to generate multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary suggestions, etc.).

[0821] The generated treatment plan is displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. For example, if "iron supplementation therapy" is selected, the server records and saves this selection. The treatment is then implemented.

[0822] Example of a prompt

[0823] The following are some specific examples of prompt statements that can be used as input to a generative AI model.

[0824] Prompt: Based on the following patient data, propose the optimal treatment plan.

[0825] Patient data:

[0826] Fever: 2 weeks

[0827] Fatigue: 3 weeks

[0828] Blood test results: Red blood cell count: 3.8, White blood cell count: 5.5, Platelet count: 150

[0829] Symptoms: Mild anemia is suspected.

[0830] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[0831] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0832] Step 1: Data Entry

[0833] The user uses a terminal to input patient health data (blood test results, symptoms, physical test results, etc.). Specifically, the user either manually enters the necessary data into an input form on the terminal or imports the data using an electronic data import function. The input in this step is health data entered by the user, and the output is data temporarily stored on the terminal.

[0834] Step 2: Send

[0835] Once the user completes the input and clicks the submit button, the device sends the data to the server. Specifically, the device generates an HTTP request and sends it to the server along with the entered data. The input in this step is health data stored on the device, and the output is the data received by the server.

[0836] Step 3: Data Received

[0837] The server receives data sent from the terminal. Specifically, the server parses the HTTP request and verifies the received data. The input for this step is the data sent in the HTTP request, and the output is the received health data.

[0838] Step 4: Data Integration and Transformation

[0839] The server integrates the received data and converts it into a format. Specifically, the server combines data from different sources (such as blood test results and symptom data) into a single dataset and converts it to JSON format. It also performs data cleaning to check for any missing or inconsistent data. The input for this step is the received health data, and the output is the integrated JSON format data.

[0840] Step 5: Creating a Treatment Plan

[0841] The server inputs the integrated data into a generating AI model to generate multiple treatment plans. Specifically, the server generates prompts for the generating AI model (e.g., GPT-4), and inputs these prompts and the integrated data to generate treatment plans. The input for this step is integrated data in JSON format, and the output is the multiple treatment plans that have been generated.

[0842] Step 6: Submit and view treatment plan

[0843] The server generates a treatment plan and sends it to the terminal, which then displays it to the user. Specifically, the server sends a list of treatment plans to the terminal in JSON format, which the terminal parses and displays in list format on the user interface. The input to this step is the generated treatment plan data, and the output is the list of treatment plans displayed on the terminal.

[0844] Step 7: Evaluation and selection of treatment plan

[0845] Users (healthcare professionals and patients) evaluate the displayed treatment plans and select the optimal plan. Specifically, the user reviews the details of each treatment plan and selects the best one. They click a selection button to confirm their choice. The input for this step is the multiple displayed treatment plans, and the output is the selected treatment plan.

[0846] Step 8: Notify and save the treatment plan

[0847] The terminal notifies the server of the selected treatment plan, and the server saves it to the database. Specifically, after the terminal clicks the select button, it sends the selected plan to the server, and the server saves it to the database. The input to this step is the selected treatment plan, and the output is the data saved on the server.

[0848] (Application Example 1)

[0849] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0850] Current medical second opinion systems often require time and effort for patients and healthcare professionals to share detailed medical data face-to-face. Furthermore, there is a lack of visually clear means to present treatment plans, making it difficult to quickly obtain and present information, especially in clinical settings. This can delay the selection of the optimal treatment plan and potentially affect the patient's treatment outcomes. To solve this problem, a more efficient and intuitive system is needed.

[0851] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0852] In this invention, the server includes means for inputting blood test results via input means, means for inputting interview and physical examination results via input means, means for integrating the blood test results with the interview and physical examination results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to healthcare professionals and patients via display means, means for selecting the optimal treatment plan and saving it via final decision means, means for inputting health data via smart glasses and processing it via a cloud server, and means for visualizing the treatment plan on the smart glasses' display. This enables quick and intuitive data sharing and treatment plan selection in person, making it possible to provide the patient with the optimal treatment plan.

[0853] A "generative artificial intelligence device" is an artificial intelligence system that generates treatment plans based on a patient's health data.

[0854] A "medical second opinion" is the provision of an opinion from another medical professional to evaluate the appropriateness of the treatment a patient is currently receiving and to suggest alternative options.

[0855] "Health data" refers to information about a patient's health status, such as blood test results, interview results, and physical examination results.

[0856] "Blood test results" refer to data obtained by analyzing blood, such as the number of red blood cells, white blood cells, and platelets.

[0857] "Medical interview results" refer to information obtained from the patient themselves, such as their symptoms, medical history, and lifestyle.

[0858] "Palpation results" refer to information obtained through physical examinations, such as heart rate and blood pressure.

[0859] "Input means" refers to methods or devices for importing health data into a system.

[0860] "Means of integration" refers to methods or devices that combine multiple health data sets into a single dataset.

[0861] A "generative artificial intelligence model" is a model used by generative artificial intelligence devices that has been trained based on past medical data and the latest medical research.

[0862] A "treatment plan" is a plan that outlines the treatment policies and methods for a patient's specific health condition.

[0863] "Display means" refers to methods or devices for visually presenting the generated treatment plan to healthcare professionals and patients.

[0864] A "final decision-making tool" refers to a method or device for saving and implementing a selected treatment plan.

[0865] "Smart glasses" are glasses-type information processing devices that allow data input and display when worn.

[0866] A "cloud server" is a remote server that processes and stores data via the internet.

[0867] "Visualization" refers to presenting data and information in a way that is visible in the real world.

[0868] Modes for carrying out the invention

[0869] This invention relates to a system that provides medical second opinions using smart glasses and a cloud server. The system includes a series of processes for inputting patient health data, generating multiple treatment plans using a generative artificial intelligence model, and presenting them to healthcare professionals and patients in a visualized form.

[0870] Hardware and software configuration

[0871] Smart Glasses

[0872] Smart glasses are information processing devices worn by the user and have the following functions:

[0873] Input methods: It has a function to input health data using QR code scanning or voice input.

[0874] Display method: It has the function of displaying and visualizing the generated treatment plan on a display screen.

[0875] Cloud Server

[0876] A cloud server is a remote service that integrates and processes data over the internet and has the following functions:

[0877] Data Integration and Processing: Blood test results, interview results, and palpation results received from smart glasses are integrated and compiled into a single dataset.

[0878] Execution of generative AI models: Based on integrated data, generative AI models are executed to generate multiple treatment plans.

[0879] Data saving: Has the function to ultimately save the selected treatment plan.

[0880] Program Processing Description

[0881] The server receives health data transmitted from the smart glasses and converts it into an appropriate format. It then uses a generative artificial intelligence model to generate a treatment plan, which is then sent back to the smart glasses.

[0882] The device (smart glasses) is used to scan and input health data and visually displays the generated treatment plan. Through the smart glasses' display, healthcare professionals and patients can review the treatment plan and select the most suitable option.

[0883] Specific hardware and software to be used

[0884] Smart glasses: A device for inputting and displaying information.

[0885] Cloud server: A remote server that performs data integration and runs generative artificial intelligence models.

[0886] Generative artificial intelligence models: AI models that generate treatment plans based on integrated data.

[0887] Specific example

[0888] For example, a patient wears smart glasses at the reception desk and scans QR codes containing their health data (e.g., blood test results, medical history, physical examination results). The scanned data is sent to a cloud server where a generative artificial intelligence model is executed. Multiple treatment plans generated by the AI ​​model are displayed on the smart glasses' screen. Healthcare professionals and patients review each treatment plan and select the optimal one, thereby proceeding with the treatment.

[0889] Example of a prompt

[0890] Input data

[0891] Blood test results: red blood cell count 4.5 million / μL, white blood cell count 7000 / μL, platelet count 150000 / μL

[0892] Symptoms: Fatigue, mild fever

[0893] Palpation results: Heart rate 80 bpm, blood pressure 120 / 80 mmHg

[0894] Content to generate

[0895] Based on the input data above, please propose several treatment plans. Each treatment plan should include details such as the treatment strategy, expected outcomes, duration of implementation, and frequency.

[0896] The above describes the embodiments for carrying out the present invention. This system enables quick and intuitive data sharing and treatment plan selection during face-to-face interactions, making it possible to provide patients with the optimal treatment plan.

[0897] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0898] Step 1:

[0899] The user wears smart glasses and scans QR codes containing health data (e.g., blood test results, medical questionnaire results, physical examination results). This input method uses the smart glasses' camera function. The QR codes encode various health data of the patient, which are then read and imported as data.

[0900] Step 2:

[0901] Smart glasses transmit scanned health data to a cloud server. This transmission utilizes an internet connection. The data output from the smart glasses is transferred to the cloud server, where it is integrated and processed.

[0902] Step 3:

[0903] The server receives and integrates data from the smart glasses. Before inputting the integrated dataset into a generative artificial intelligence model, it converts it to the required format. This data processing integrates blood test results, interview results, and physical examination results into a single dataset in the appropriate format.

[0904] Step 4:

[0905] The server inputs the integrated dataset into a generative artificial intelligence model to generate multiple treatment plans. This AI model is pre-trained on medical datasets and is designed to generate optimal treatment plans based on specific health conditions. In this processing step, the AI ​​model performs calculations based on the input data and outputs multiple treatment candidates.

[0906] Step 5:

[0907] The generated treatment plan is then sent back to the smart glasses via the cloud server. At this point, the data is visualized and converted into a format that can be viewed on the smart glasses.

[0908] Step 6:

[0909] The smart glasses display treatment plans received from the server. The displayed treatment plans include details of each treatment, expected outcomes, and estimated time required for implementation. Users (healthcare professionals and patients) can access this information through the smart glasses.

[0910] Step 7:

[0911] Healthcare professionals and patients evaluate multiple treatment plans displayed on smart glasses and select the optimal plan. The selected treatment plan is notified from the smart glasses to a cloud server, where the information is stored.

[0912] Step 8:

[0913] The server saves the selected treatment plan using a final decision-making mechanism and, if necessary, begins preparations to implement the treatment plan in detail. This saving process ensures that subsequent treatments and follow-ups proceed smoothly.

[0914] The above outlines the specific steps for implementing this invention. In each step, data collection, integration, generation, display, selection, and storage are carried out efficiently.

[0915] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0916] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for proposing an optimal treatment plan and adjusting the delivery method, taking the user's emotions into consideration.

[0917] Data entry

[0918] The user (medical staff) first enters blood test results using a terminal. Blood test results are numerical data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into an input form or imported electronically. The user also enters the results of the patient's interview and physical examination. This includes the symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.), which are also entered into a dedicated form.

[0919] Data integration and processing

[0920] The server receives blood test results, interview and physical examination results sent from the terminal. This data is integrated and compiled into a single dataset. This integrated data is then converted into a format suitable for subsequent analysis.

[0921] emotion recognition

[0922] The device incorporates an emotion engine that analyzes the user's voice, facial expressions, or biometric data (e.g., heart rate variability, skin electrical activity, etc.). This emotion engine recognizes the user's emotional state (e.g., stress, relief, anxiety, etc.) in real time and sends that data to a server.

[0923] Treatment plan generation

[0924] The server inputs integrated medical data and user emotional data into a generative artificial intelligence model to generate multiple treatment plans. This generative AI model is pre-tuned and trained on medical datasets. The model generates candidate treatment plans that are optimal for the patient's specific health and emotional state.

[0925] Display and selection of treatment plans

[0926] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). The terminal displays the treatment plan candidates in a list format, including detailed descriptions and expected outcomes for each treatment plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[0927] Users (patients and doctors) evaluate these treatment plans and select the optimal one. The evaluation is based on the effectiveness of the treatment, cost-effectiveness, and the user's emotional state and preferences.

[0928] Final decision and implementation

[0929] The user notifies the server of the selected treatment plan via their device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[0930] Specific example

[0931] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[0932] The generated treatment plans are displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. Each treatment plan is accompanied by appropriate comments from the emotion engine (e.g., "This treatment is gentle on the body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[0933] Thus, the present invention enables healthcare professionals to quickly provide and consider multiple highly accurate treatment plans within limited time and resources. Furthermore, because treatment plans are presented in an easily understandable format tailored to the patient's emotional state, it promotes treatment choices that are more based on informed consent.

[0934] The following describes the processing flow.

[0935] Step 1:

[0936] Users (medical staff) input blood test results using a terminal. Specifically, they manually enter numerical data such as red blood cell count, white blood cell count, and platelet count into an input form. Electronic data acquisition methods are also available.

[0937] Step 2:

[0938] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fatigue or fever) and physical examination results (e.g., heart rate or blood pressure). This information is manually entered into a dedicated input form.

[0939] Step 3:

[0940] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[0941] Step 4:

[0942] The server integrates the received blood test results with the results of the medical interview and physical examination. Specifically, it integrates the blood test results with the medical interview and physical examination results and combines them into a single dataset. The integrated data is then converted into an appropriate format.

[0943] Step 5:

[0944] The device recognizes the user's voice, facial expressions, or biometric data in real time and analyzes the user's emotional state using an emotion engine. For example, it recognizes emotions such as anxiety, relief, and stress based on voice recognition, facial expression analysis, and data from biosensors (such as heart rate fluctuations).

[0945] Step 6:

[0946] The server inputs integrated medical data and user sentiment data into a generative artificial intelligence model to generate numerous treatment plans. The generative AI model is pre-tuned and trained on medical datasets.

[0947] Step 7:

[0948] The server generates multiple treatment plans and sends them to the terminal. The terminal displays the candidate treatment plans in a list format, including detailed descriptions and expected outcomes for each plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[0949] Step 8:

[0950] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the user's emotional state and preferences.

[0951] Step 9:

[0952] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[0953] Step 10:

[0954] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[0955] (Example 2)

[0956] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0957] In modern medical practice, there is a demand for both speed and accuracy in developing treatment plans. However, within limited time and resources, it is difficult for healthcare professionals to process large amounts of data simultaneously and provide optimal treatment plans that take into account the patient's emotional state. As a result, it is difficult to reduce patient anxiety and stress during the selection of a treatment plan, and informed consent becomes challenging. Conventional technologies have not been able to generate treatment plans that take emotional states into account, which has placed a heavy burden on both patients and healthcare professionals.

[0958] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting blood test results by input means, means for inputting interview and palpation results by input means, means for integrating the blood test results with the interview and palpation results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to medical professionals and patients by display means, means for selecting the optimal treatment plan and saving it by final decision means, means for recognizing the user's emotional state by an emotion engine and processing the data, and means for providing appropriate comments and explanations according to the user's emotional state based on the generated treatment plan. As a result, medical professionals can not only provide treatment plans quickly and accurately, but also provide individualized responses according to the patient's emotional state, enabling the provision of better medical services.

[0959] A "generative artificial intelligence device" is a device that generates new information or results based on data provided by artificial intelligence technology.

[0960] A "medical second opinion" refers to a patient receiving a second opinion from another healthcare professional regarding the treatment or diagnosis they are currently receiving.

[0961] "Blood test results" is a general term for numerical values ​​and data obtained by analyzing a patient's blood, and includes measurement results such as red blood cell count, white blood cell count, and platelet count.

[0962] "Medical interview" refers to the act of a healthcare professional gathering information from a patient about their current symptoms, medical history, and other relevant details.

[0963] "Palpation" refers to the act of a medical professional examining a patient by touching their body with their hands.

[0964] "Integration" refers to the process of combining multiple different sets of data into one.

[0965] A "generative artificial intelligence model" refers to an artificial intelligence model that uses pre-trained algorithms to generate new information or results.

[0966] "Display means" refers to devices or methods that display information to a user through a computer screen, tablet, smartphone, etc.

[0967] "Healthcare professionals" refers to individuals engaged in medical-related professional occupations, such as doctors, nurses, and pharmacists.

[0968] The term "patient" refers to a person receiving medical services.

[0969] The term "final decision-making mechanism" refers to a system or process for storing and implementing selected results or information.

[0970] An "emotion engine" refers to a combination of software and hardware that analyzes a user's emotional state and outputs it as data.

[0971] "Means of providing comments and explanations" refers to devices or methods for adding appropriate explanations to the user based on the generated treatment plan.

[0972] A "database" refers to a system that centrally manages, stores, and allows data to be retrieved as needed.

[0973] "REST API" refers to an application programming interface based on Representational State Transfer, and is a standard for data communication.

[0974] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for generating an optimal treatment plan using medical data and user emotion data, and for providing that plan to healthcare professionals and patients.

[0975] Hardware configuration

[0976] Servers and terminals (e.g., PCs, tablets, smartphones) are the central hardware components of this system.

[0977] The server receives, integrates, processes, generates, and stores data, as well as treatment plans.

[0978] The device performs data input and display, as well as recognition and transmission of emotional data.

[0979] Software Configuration

[0980] The generative artificial intelligence model is embedded in a server and used to generate treatment plans from medical data. This model is fine-tuned in advance based on medical datasets.

[0981] The emotion engine is built into the device and analyzes the user's voice, facial expressions, and biometric data in real time. This data reflects the user's emotional state and is sent to the server.

[0982] System operation

[0983] The user (healthcare professional) first uses a terminal to input blood test results. This is done either manually or by importing from an electronic medical record system. Similarly, the results of the medical history interview and physical examination are also entered. This data is sent to the server and integrated.

[0984] The emotion engine built into the device analyzes the user's voice, facial expressions, heart rate fluctuations, etc., to generate emotion data. This data is also sent to a server and integrated with medical data.

[0985] The server inputs the integrated data into a generative artificial intelligence model to generate multiple treatment plans. These treatment plans are sent to the terminal and provided to the user. The displayed treatment plans include appropriate comments and explanations based on the user's emotional state as recognized by the emotion engine.

[0986] Users (healthcare professionals and patients) evaluate candidate treatment plans and select the optimal one. The selected treatment plan is notified to the server via the terminal and stored on the server. Subsequently, specific treatment is carried out based on this plan.

[0987] Specific example

[0988] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into the terminal, and the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvements, etc.). The generated treatment plans are displayed on the terminal, and the user reviews the details of each plan and selects a specific course of action. Each treatment plan is accompanied by an appropriate comment from the emotion engine (e.g., "This treatment is not burdensome on your body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., iron supplementation), and the server records and saves this selection. Subsequently, this treatment is implemented.

[0989] Example of a prompt

[0990] The following are examples of prompts to input into a generative AI model:

[0991] "The patient's current symptoms are fatigue and a mild fever, and blood test results suggest mild anemia. The patient is experiencing significant anxiety. Please propose an optimal treatment plan based on these factors."

[0992] "The patient's blood pressure and heart rate are within the normal range, but their white blood cell count is elevated. The patient is experiencing stress. Please generate a treatment plan considering this situation."

[0993] The above describes a specific embodiment for carrying out the present invention. This system not only enables healthcare professionals to provide rapid and highly accurate treatment plans, but also allows for individualized responses tailored to the patient's emotional state, thereby enabling the provision of better medical services.

[0994] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0995] Step 1:

[0996] Users (medical staff) manually enter blood test results into a dedicated input form on the terminal. The data entered by the user includes numerical data such as red blood cell count, white blood cell count, and platelet count. If an electronic medical record (EMR) system exists, data can be automatically imported from the corresponding EMR system. This improves the accuracy and efficiency of data entry.

[0997] Step 2:

[0998] The user uses a terminal to input the results of the patient's medical history and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). The entered medical history and physical examination data is used for subsequent data integration and analysis.

[0999] Step 3:

[1000] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database on the server. The received data includes numerical and text data associated with each patient.

[1001] Step 4:

[1002] The server integrates the received blood test results, interview data, and physical examination results. Specifically, it converts data from different data sources into a standard format and combines them into a single dataset. This ensures that all medical information is presented in a consistently analyzable format.

[1003] Step 5:

[1004] An emotion engine built into the device recognizes the user's emotional state in real time. Emotion recognition uses data from voice, facial expressions, and biosensors (e.g., heart rate and skin electrical activity). The emotion engine analyzes this sensor data to estimate the user's emotional state (e.g., stress, reassurance, anxiety, etc.).

[1005] Step 6:

[1006] The device transmits recognized emotional data to the server. This transmitted emotional data includes specific emotional states and their intensity. This data becomes an important element in the subsequent generation of treatment plans.

[1007] Step 7:

[1008] The server inputs integrated medical and emotional data into a generative artificial intelligence model. This initiates the generation of a treatment plan based on the patient's specific health and emotional state. The input data includes an integrated medical dataset and recognized emotional data.

[1009] Step 8:

[1010] The server generates multiple treatment plans using a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. The generated treatment plans include specific methods and expected effects.

[1011] Step 9:

[1012] The server generates a treatment plan and sends it to the terminal. The terminal displays a list of potential treatment plans, each containing a detailed explanation and expected outcomes. Furthermore, appropriate comments and explanations are added based on recognized emotional data.

[1013] Step 10:

[1014] Users (patients and healthcare professionals) review potential treatment plans on their devices. Users then view the details of each treatment plan and evaluate the advantages and disadvantages of each plan.

[1015] Step 11:

[1016] The user selects the optimal treatment plan. Evaluation criteria include treatment effectiveness, cost-effectiveness, the user's emotional state, and the patient's wishes.

[1017] Step 12:

[1018] The user's selected treatment plan is notified to the server via the terminal. The server receives this information and saves the selected treatment plan to its database.

[1019] Step 13:

[1020] The server begins preparing for implementation based on the saved treatment plan. Specifically, it arranges for necessary medications and notifies relevant medical staff. This allows the selected treatment plan to be put into practice.

[1021] (Application Example 2)

[1022] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1023] Traditional medical second opinion systems propose treatment plans based on medical data, but they do not take into account the patient's emotional state. Therefore, it is difficult to determine whether the treatment plan is optimal for the patient, and there is a lack of consideration for reducing the patient's anxiety and stress. Furthermore, because relaxation content is not provided to patients undergoing treatment who require emotional support, there is a possibility that the overall effectiveness of the treatment will be reduced.

[1024] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1025] In this invention, the server includes means for inputting blood test results, means for inputting interview and physical examination results, means for analyzing the user's emotional state in real time using emotion recognition technology and integrating the data, means for utilizing a generative artificial intelligence model to generate multiple treatment plans, means for displaying the generated treatment plans to medical professionals and patients, and means for selecting and saving the optimal treatment plan. This makes it possible to provide personalized treatment plans and emotional support based on the user's medical and emotional data.

[1026] A "generative artificial intelligence device" is a device equipped with a model that takes medical data and emotional data as input and generates optimal treatment plans and relaxation content based on them.

[1027] A "medical second opinion" refers to a second expert medical opinion and treatment plan provided to a patient, intended to evaluate the patient's condition and treatment options from a different perspective.

[1028] "Emotion recognition technology" is a technology that identifies a user's emotional state by analyzing voice, facial expressions, or biometric data.

[1029] "Fine-tuning" is the process of fine-tuning a generative artificial intelligence model based on a specific dataset to improve its performance.

[1030] "Display means" refers to a device or interface for visually displaying the generated treatment plan or relaxation content.

[1031] "Relaxation content" refers to media content (such as meditation guides, calming music, and massage videos) designed to reduce patients' stress and anxiety and promote mental stability.

[1032] A "medical dataset" is a collection of data related to medical care, including diagnostic data, test results, and patient symptoms.

[1033] An "emotion dataset" is a collection of data that represents a user's emotional state, and includes data such as voice tone, facial expressions, and biometric data.

[1034] This invention is a system that combines a generative artificial intelligence device with emotion recognition technology to generate optimal treatment plans and relaxation content that take into account the patient's emotional state when providing a second medical opinion. A specific embodiment of this system is described in detail below.

[1035] System Configuration

[1036] This system consists of servers, terminals, and users.

[1037] 1. Enter blood test data

[1038] The terminal has a function for entering blood test results, which users (healthcare professionals) input using an input form or electronic means.

[1039] 2. Input of medical history and physical examination results

[1040] The terminal has a function to input the results of the medical questionnaire and physical examination, and this data is entered in the same way.

[1041] 3. Data Integration

[1042] The server receives blood test results, interview and physical examination results from the terminals, combines them into a single dataset, and converts them into an appropriate format.

[1043] 4. Emotion recognition

[1044] The device incorporates emotion recognition technology that analyzes voice, facial expressions, or biometric data to recognize the user's emotional state in real time. The results of this analysis are then sent to a server.

[1045] 5. Treatment planning and creation of relaxation content

[1046] The server inputs integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. This generative AI model has been pre-trained and fine-tuned using medical and emotional datasets.

[1047] 6. Display and Selection

[1048] The generated treatment plans and relaxation content are displayed on the device, allowing users (healthcare professionals and patients) to review them and select the most suitable plan and content.

[1049] 7. Preservation and Implementation

[1050] The selected treatment plan and relaxation content are notified to the server via the device, and the server stores this information and prepares for implementation as needed.

[1051] Program processing

[1052] The server analyzes blood test results and medical interview data, integrating the acquired data using emotion recognition technology. This process utilizes emotion recognition software such as EmotionEngine. The integrated data is input into a generative artificial intelligence model, which then generates treatment plans and relaxation content. For example, meditation guides, calming music, and massage videos are generated.

[1053] Specific example

[1054] For example, suppose we input health data for a user, such as "heart rate: 72, blood pressure: 120 / 80, symptoms: headache, fatigue," and emotional data such as "high anxiety." Based on this data, we input the following prompt into a generative artificial intelligence model:

[1055] User's health data: Heart rate 72, Blood pressure 120 / 80, Symptoms: Headache, Fatigue

[1056] User sentiment data: High levels of anxiety

[1057] What type of relaxation content is best?

[1058] Based on this prompt, the AI ​​suggests and displays optimal relaxation content (such as meditation guides or calming music) to the user.

[1059] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1060] Step 1:

[1061] The user (healthcare professional) uses a terminal to input blood test results. This can be done manually using an input form or electronically (e.g., by uploading a data file). The input includes numerical data such as red blood cell count, white blood cell count, and platelet count.

[1062] Input: Blood test result data

[1063] Output: Blood test result data stored on the device.

[1064] Step 2:

[1065] The user (healthcare professional) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). This data is also manually entered into a dedicated form.

[1066] Input: Medical history and physical examination results data

[1067] Output: Medical interview and palpation results data stored on the device.

[1068] Step 3:

[1069] The terminal integrates blood test results with interview and physical examination results and sends them to the server. This combines information from different data sources into a single dataset. The data is then converted to an appropriate format, such as JSON.

[1070] Input: Blood test results data, medical interview and physical examination results data

[1071] Output: Integrated dataset sent to the server

[1072] Step 4:

[1073] The device uses emotion recognition technology to analyze the user's voice, facial expressions, and biometric data (e.g., heart rate variability, skin electrical activity, etc.) in real time to recognize their emotional state. This data is also sent to the server.

[1074] Input: User's voice, facial expressions, biometric data

[1075] Output: Sentiment data sent to the server

[1076] Step 5:

[1077] The server inputs received integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. By generating prompt sentences and inputting them into the generative AI model, specific treatment plans and content suggestions are obtained.

[1078] Input: Integrative medical data, emotional data

[1079] Output: Generated treatment plan and relaxation content

[1080] Step 6:

[1081] The server sends the generated treatment plan and relaxation content to the device. This includes a detailed explanation of each treatment plan and expected results, as well as comments based on emotion recognition data.

[1082] Input: Generated treatment plan and relaxation content

[1083] Output: Treatment plan and relaxation content displayed on the device.

[1084] Step 7:

[1085] Users (healthcare professionals and patients) review the treatment plan and relaxation content displayed on their terminals and select the most suitable plan. This selection is then communicated to the server.

[1086] Input: Displayed treatment plan and relaxation content

[1087] Output: Selection results sent to the server

[1088] Step 8:

[1089] The server saves the selected treatment plan and relaxation content, and begins specific preparations for the treatment as needed. Details of the treatment plan will be communicated to the necessary stakeholders.

[1090] Input: User-selected treatment plan and relaxation content

[1091] Output: Saved treatment plan and relaxation content, start of treatment preparation

[1092] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1093] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1094] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1095] [Fourth Embodiment]

[1096] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1097] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1098] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1099] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1100] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1101] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1102] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1103] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1104] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1105] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1106] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1107] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1108] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1109] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[1110] Data entry

[1111] The user first enters blood test results using a terminal. These results include data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into the system's input form or imported electronically. The user also enters the results of the patient's medical history and physical examination. This includes the patient's symptoms (e.g., fever, fatigue) and physical examination results (e.g., heart rate, blood pressure).

[1112] Data integration and processing

[1113] The server receives blood test results and interview / palpation results sent from the terminal. The received data is integrated and compiled into a single dataset. Specifically, the blood test results and interview / palpation results are integrated and converted into a format suitable for subsequent processing.

[1114] Treatment plan generation

[1115] Based on the integrated data, the server uses a generative artificial intelligence model to generate multiple treatment plans. This model is pre-tuned and trained on past performance and the latest medical research data. This generates candidate treatment plans that are optimal for the patient's specific health condition.

[1116] Display and selection of treatment plans

[1117] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). On the terminal, multiple treatment plans are displayed in a list format, including a detailed description of each treatment plan and its expected outcome.

[1118] Users (patients and doctors) evaluate these treatment plans and select the optimal one. Options include highly effective treatments, cost-effective treatments, and treatments best suited to the patient's lifestyle and preferences.

[1119] Final decision and implementation

[1120] The selected treatment plan is communicated to the server via the user's device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[1121] Specific example

[1122] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, a generative artificial intelligence model generates multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[1123] The generated treatment plan is displayed on the terminal, allowing users (doctor and patient) to review the details of each plan and select a specific course of action. They choose the plan they deem most appropriate (for example, "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[1124] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[1125] The following describes the processing flow.

[1126] Step 1:

[1127] Users (medical staff) use a terminal to input blood test results. Blood test results consist of numerical data such as red blood cell count, white blood cell count, and platelet count. This data is either manually entered into an input form or imported into the system using an electronic import method.

[1128] Step 2:

[1129] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. The results of the medical interview and physical examination include the symptoms reported by the patient (e.g., fatigue, fever, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.). This information is also entered into a dedicated form.

[1130] Step 3:

[1131] The server receives blood test results and interview / physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[1132] Step 4:

[1133] The server integrates the received blood test results and the results of the medical interview and physical examination. The integrated data is converted into a format suitable for later analysis and compiled into a single dataset.

[1134] Step 5:

[1135] The server inputs the integrated data into a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. Using this model, multiple treatment plans are generated, tailored to the patient's specific health condition.

[1136] Step 6:

[1137] The server sends multiple treatment plans it has generated to the terminal. The terminal displays the candidate treatment plans in a list format, along with a detailed description of each plan and its expected outcome.

[1138] Step 7:

[1139] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the patient's lifestyle and preferences.

[1140] Step 8:

[1141] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[1142] Step 9:

[1143] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[1144] (Example 1)

[1145] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1146] In modern medicine, there is a need for systems that enable healthcare professionals to propose multiple treatment plans quickly and accurately within limited time and resources, allowing patients to choose the optimal treatment. However, conventional systems do not adequately provide mechanisms for integrating multiple data sources, automatically generating appropriate treatment plans, and ultimately enabling patients and doctors to efficiently select a treatment plan. This increases the burden on healthcare professionals and makes it difficult for patients to make appropriate treatment choices based on informed consent.

[1147] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1148] In this invention, the server includes means for a user to input patient health data using a terminal, means for transmitting data from the terminal to the server, means for the server to integrate the received data and convert it into a format, means for generating multiple treatment plans using a generation AI model based on the integrated data, means for transmitting the generated treatment plans to the terminal and displaying them to the user, means for the user to select the optimal plan from the displayed treatment plans, and means for notifying the server of the selected treatment plan and saving it. This enables healthcare professionals to efficiently generate and present multiple treatment plans within limited time and resources, and allows patients to make the optimal treatment choice based on informed consent.

[1149] A "user" is the person who inputs patient health data and operates the system.

[1150] A "terminal" is an electronic device used by the user to perform input tasks or select treatment plans.

[1151] A "server" is a central processing unit that receives data transmitted from terminals and performs data integration, transformation, analysis, and generation of treatment plans.

[1152] "Means of transmitting data from the terminal to the server" refers to the function that allows the terminal to electronically transmit the health data it has entered to the server.

[1153] "Data integration methods" refer to the processes used by a server to combine data received from different sources into a single dataset.

[1154] "Data conversion means" refers to the process by which a server converts the received and integrated data into a format suitable for subsequent processing.

[1155] A "generative AI model" is a machine learning-based model used by a server to generate multiple treatment plans based on integrated data.

[1156] "Means for transmitting and displaying treatment plans" refers to a function that transmits a treatment plan generated by the server to a terminal, and allows the terminal to display that plan to the user.

[1157] "Means of selecting a treatment plan" refers to a function that allows the user to select an appropriate treatment plan from the displayed options.

[1158] The "means for notifying and saving treatment plans" refer to a function that notifies the server of the selected treatment plan from the terminal, and the server saves that plan to a database.

[1159] This invention relates to a system that provides medical second opinions using a generative artificial intelligence device. The system includes a series of processes for inputting patient health data and generating multiple treatment plans.

[1160] Data entry

[1161] The user first enters the patient's health data using a terminal. Specifically, they enter blood test results (e.g., red blood cell count, white blood cell count, platelet count, etc.), symptoms (e.g., fever, fatigue, etc.), and physical test results (e.g., heart rate, blood pressure, etc.). This data can be entered manually or imported electronically.

[1162] Sending and receiving data

[1163] When a user submits data through an input form on their device, the device sends it to the server. The server receives the HTTP request and retrieves the entered data. For example, TCP / IP or HTTP / HTTPS protocols are used for data transfer.

[1164] Data integration and transformation

[1165] The server integrates the received data and converts it into a format suitable for subsequent processing. Specifically, data integration tools such as Apache Kafka and Apache NiFi are used to integrate data from different sources (blood test results and interview / physical examination results) and convert it into formats such as JSON. Data cleaning is also performed simultaneously to check for any missing or inconsistent data.

[1166] Treatment plan generation

[1167] The server generates multiple treatment plans using a generative AI model based on integrated data. This generative AI model, for example, uses OpenAI's GPT-4 and is pre-tuned on medical datasets. This AI model generates the optimal treatment plan based on past performance and the latest medical research data.

[1168] Sending and displaying treatment plans

[1169] The treatment plan generated by the server is sent to the terminal and displayed to the user. The list of generated treatment plans is sent in JSON format, which the terminal parses and displays in list format on the user interface. Detailed information for each treatment plan (expected results, treatment duration, etc.) is also displayed.

[1170] Evaluation and selection of treatment plans

[1171] Users (healthcare professionals and patients) evaluate and select the most suitable treatment plan from the displayed options. Specifically, they evaluate plans based on criteria such as effectiveness, cost-effectiveness, and suitability for their lifestyle. The selected treatment plan is finalized by clicking a selection button on the device.

[1172] Final decision and implementation

[1173] The terminal notifies the server of the selected treatment plan, and the server saves this information to a database. Specific arrangements for treatment (such as medication procurement and follow-up schedules) are then made as needed.

[1174] Specific example

[1175] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into a terminal, and the server receives and integrates it. Next, a generative artificial intelligence model is used to generate multiple treatment plans based on this case (e.g., iron supplementation, vitamin B12 supplementation, dietary suggestions, etc.).

[1176] The generated treatment plan is displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. For example, if "iron supplementation therapy" is selected, the server records and saves this selection. The treatment is then implemented.

[1177] Example of a prompt

[1178] The following are some specific examples of prompt statements that can be used as input to a generative AI model.

[1179] Prompt: Based on the following patient data, propose the optimal treatment plan.

[1180] Patient data:

[1181] Fever: 2 weeks

[1182] Fatigue: 3 weeks

[1183] Blood test results: Red blood cell count: 3.8, White blood cell count: 5.5, Platelet count: 150

[1184] Symptoms: Mild anemia is suspected.

[1185] Based on the above, the present invention enables healthcare professionals to propose and consider multiple treatment plans quickly and accurately within limited time and resources. Furthermore, because patients are provided with multiple treatment plans in an easy-to-understand format, treatment choices based on more informed consent are promoted.

[1186] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1187] Step 1: Data Entry

[1188] The user uses a terminal to input patient health data (blood test results, symptoms, physical test results, etc.). Specifically, the user either manually enters the necessary data into an input form on the terminal or imports the data using an electronic data import function. The input in this step is health data entered by the user, and the output is data temporarily stored on the terminal.

[1189] Step 2: Send

[1190] Once the user completes the input and clicks the submit button, the device sends the data to the server. Specifically, the device generates an HTTP request and sends it to the server along with the entered data. The input in this step is health data stored on the device, and the output is the data received by the server.

[1191] Step 3: Data Received

[1192] The server receives data sent from the terminal. Specifically, the server parses the HTTP request and verifies the received data. The input for this step is the data sent in the HTTP request, and the output is the received health data.

[1193] Step 4: Data Integration and Transformation

[1194] The server integrates the received data and converts it into a format. Specifically, the server combines data from different sources (such as blood test results and symptom data) into a single dataset and converts it to JSON format. It also performs data cleaning to check for any missing or inconsistent data. The input for this step is the received health data, and the output is the integrated JSON format data.

[1195] Step 5: Creating a Treatment Plan

[1196] The server inputs the integrated data into a generating AI model to generate multiple treatment plans. Specifically, the server generates prompts for the generating AI model (e.g., GPT-4), and inputs these prompts and the integrated data to generate treatment plans. The input for this step is integrated data in JSON format, and the output is the multiple treatment plans that have been generated.

[1197] Step 6: Submit and view treatment plan

[1198] The server generates a treatment plan and sends it to the terminal, which then displays it to the user. Specifically, the server sends a list of treatment plans to the terminal in JSON format, which the terminal parses and displays in list format on the user interface. The input to this step is the generated treatment plan data, and the output is the list of treatment plans displayed on the terminal.

[1199] Step 7: Evaluation and selection of treatment plan

[1200] Users (healthcare professionals and patients) evaluate the displayed treatment plans and select the optimal plan. Specifically, the user reviews the details of each treatment plan and selects the best one. They click a selection button to confirm their choice. The input for this step is the multiple displayed treatment plans, and the output is the selected treatment plan.

[1201] Step 8: Notify and save the treatment plan

[1202] The terminal notifies the server of the selected treatment plan, and the server saves it to the database. Specifically, after the terminal clicks the select button, it sends the selected plan to the server, and the server saves it to the database. The input to this step is the selected treatment plan, and the output is the data saved on the server.

[1203] (Application Example 1)

[1204] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1205] Current medical second opinion systems often require time and effort for patients and healthcare professionals to share detailed medical data face-to-face. Furthermore, there is a lack of visually clear means to present treatment plans, making it difficult to quickly obtain and present information, especially in clinical settings. This can delay the selection of the optimal treatment plan and potentially affect the patient's treatment outcomes. To solve this problem, a more efficient and intuitive system is needed.

[1206] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1207] In this invention, the server includes means for inputting blood test results via input means, means for inputting interview and physical examination results via input means, means for integrating the blood test results with the interview and physical examination results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to healthcare professionals and patients via display means, means for selecting the optimal treatment plan and saving it via final decision means, means for inputting health data via smart glasses and processing it via a cloud server, and means for visualizing the treatment plan on the smart glasses' display. This enables quick and intuitive data sharing and treatment plan selection in person, making it possible to provide the patient with the optimal treatment plan.

[1208] A "generative artificial intelligence device" is an artificial intelligence system that generates treatment plans based on a patient's health data.

[1209] A "medical second opinion" is the provision of an opinion from another medical professional to evaluate the appropriateness of the treatment a patient is currently receiving and to suggest alternative options.

[1210] "Health data" refers to information about a patient's health status, such as blood test results, interview results, and physical examination results.

[1211] "Blood test results" refer to data obtained by analyzing blood, such as the number of red blood cells, white blood cells, and platelets.

[1212] "Medical interview results" refer to information obtained from the patient themselves, such as their symptoms, medical history, and lifestyle.

[1213] "Palpation results" refer to information obtained through physical examinations, such as heart rate and blood pressure.

[1214] "Input means" refers to methods or devices for importing health data into a system.

[1215] "Means of integration" refers to methods or devices that combine multiple health data sets into a single dataset.

[1216] A "generative artificial intelligence model" is a model used by generative artificial intelligence devices that has been trained based on past medical data and the latest medical research.

[1217] A "treatment plan" is a plan that outlines the treatment policies and methods for a patient's specific health condition.

[1218] "Display means" refers to methods or devices for visually presenting the generated treatment plan to healthcare professionals and patients.

[1219] A "final decision-making tool" refers to a method or device for saving and implementing a selected treatment plan.

[1220] "Smart glasses" are glasses-type information processing devices that allow data input and display when worn.

[1221] A "cloud server" is a remote server that processes and stores data via the internet.

[1222] "Visualization" refers to presenting data and information in a way that is visible in the real world.

[1223] Modes for carrying out the invention

[1224] This invention relates to a system that provides medical second opinions using smart glasses and a cloud server. The system includes a series of processes for inputting patient health data, generating multiple treatment plans using a generative artificial intelligence model, and presenting them to healthcare professionals and patients in a visualized form.

[1225] Hardware and software configuration

[1226] Smart Glasses

[1227] Smart glasses are information processing devices worn by the user and have the following functions:

[1228] Input methods: It has a function to input health data using QR code scanning or voice input.

[1229] Display method: It has the function of displaying and visualizing the generated treatment plan on a display screen.

[1230] Cloud Server

[1231] A cloud server is a remote service that integrates and processes data over the internet and has the following functions:

[1232] Data Integration and Processing: Blood test results, interview results, and palpation results received from smart glasses are integrated and compiled into a single dataset.

[1233] Execution of generative AI models: Based on integrated data, generative AI models are executed to generate multiple treatment plans.

[1234] Data saving: Has the function to ultimately save the selected treatment plan.

[1235] Program Processing Description

[1236] The server receives health data transmitted from the smart glasses and converts it into an appropriate format. It then uses a generative artificial intelligence model to generate a treatment plan, which is then sent back to the smart glasses.

[1237] The device (smart glasses) is used to scan and input health data and visually displays the generated treatment plan. Through the smart glasses' display, healthcare professionals and patients can review the treatment plan and select the most suitable option.

[1238] Specific hardware and software to be used

[1239] Smart glasses: A device for inputting and displaying information.

[1240] Cloud server: A remote server that performs data integration and runs generative artificial intelligence models.

[1241] Generative artificial intelligence models: AI models that generate treatment plans based on integrated data.

[1242] Specific example

[1243] For example, a patient wears smart glasses at the reception desk and scans QR codes containing their health data (e.g., blood test results, medical history, physical examination results). The scanned data is sent to a cloud server where a generative artificial intelligence model is executed. Multiple treatment plans generated by the AI ​​model are displayed on the smart glasses' screen. Healthcare professionals and patients review each treatment plan and select the optimal one, thereby proceeding with the treatment.

[1244] Example of a prompt

[1245] Input data

[1246] Blood test results: red blood cell count 4.5 million / μL, white blood cell count 7000 / μL, platelet count 150000 / μL

[1247] Symptoms: Fatigue, mild fever

[1248] Palpation results: Heart rate 80 bpm, blood pressure 120 / 80 mmHg

[1249] Content to generate

[1250] Based on the input data above, please propose several treatment plans. Each treatment plan should include details such as the treatment strategy, expected outcomes, duration of implementation, and frequency.

[1251] The above describes the embodiments for carrying out the present invention. This system enables quick and intuitive data sharing and treatment plan selection during face-to-face interactions, making it possible to provide patients with the optimal treatment plan.

[1252] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1253] Step 1:

[1254] The user wears smart glasses and scans QR codes containing health data (e.g., blood test results, medical questionnaire results, physical examination results). This input method uses the smart glasses' camera function. The QR codes encode various health data of the patient, which are then read and imported as data.

[1255] Step 2:

[1256] Smart glasses transmit scanned health data to a cloud server. This transmission utilizes an internet connection. The data output from the smart glasses is transferred to the cloud server, where it is integrated and processed.

[1257] Step 3:

[1258] The server receives and integrates data from the smart glasses. Before inputting the integrated dataset into a generative artificial intelligence model, it converts it to the required format. This data processing integrates blood test results, interview results, and physical examination results into a single dataset in the appropriate format.

[1259] Step 4:

[1260] The server inputs the integrated dataset into a generative artificial intelligence model to generate multiple treatment plans. This AI model is pre-trained on medical datasets and is designed to generate optimal treatment plans based on specific health conditions. In this processing step, the AI ​​model performs calculations based on the input data and outputs multiple treatment candidates.

[1261] Step 5:

[1262] The generated treatment plan is then sent back to the smart glasses via the cloud server. At this point, the data is visualized and converted into a format that can be viewed on the smart glasses.

[1263] Step 6:

[1264] The smart glasses display treatment plans received from the server. The displayed treatment plans include details of each treatment, expected outcomes, and estimated time required for implementation. Users (healthcare professionals and patients) can access this information through the smart glasses.

[1265] Step 7:

[1266] Healthcare professionals and patients evaluate multiple treatment plans displayed on smart glasses and select the optimal plan. The selected treatment plan is notified from the smart glasses to a cloud server, where the information is stored.

[1267] Step 8:

[1268] The server saves the selected treatment plan using a final decision-making mechanism and, if necessary, begins preparations to implement the treatment plan in detail. This saving process ensures that subsequent treatments and follow-ups proceed smoothly.

[1269] The above outlines the specific steps for implementing this invention. In each step, data collection, integration, generation, display, selection, and storage are carried out efficiently.

[1270] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1271] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for proposing an optimal treatment plan and adjusting the delivery method, taking the user's emotions into consideration.

[1272] Data entry

[1273] The user (medical staff) first enters blood test results using a terminal. Blood test results are numerical data such as red blood cell count, white blood cell count, and platelet count, which are entered manually into an input form or imported electronically. The user also enters the results of the patient's interview and physical examination. This includes the symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and the results of the physical examination (e.g., heart rate, blood pressure, etc.), which are also entered into a dedicated form.

[1274] Data integration and processing

[1275] The server receives blood test results, interview and physical examination results sent from the terminal. This data is integrated and compiled into a single dataset. This integrated data is then converted into a format suitable for subsequent analysis.

[1276] emotion recognition

[1277] The device incorporates an emotion engine that analyzes the user's voice, facial expressions, or biometric data (e.g., heart rate variability, skin electrical activity, etc.). This emotion engine recognizes the user's emotional state (e.g., stress, relief, anxiety, etc.) in real time and sends that data to a server.

[1278] Treatment plan generation

[1279] The server inputs integrated medical data and user emotional data into a generative artificial intelligence model to generate multiple treatment plans. This generative AI model is pre-tuned and trained on medical datasets. The model generates candidate treatment plans that are optimal for the patient's specific health and emotional state.

[1280] Display and selection of treatment plans

[1281] The server sends the generated treatment plan candidates to the terminal and provides them to the user (healthcare professional and patient). The terminal displays the treatment plan candidates in a list format, including detailed descriptions and expected outcomes for each treatment plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[1282] Users (patients and doctors) evaluate these treatment plans and select the optimal one. The evaluation is based on the effectiveness of the treatment, cost-effectiveness, and the user's emotional state and preferences.

[1283] Final decision and implementation

[1284] The user notifies the server of the selected treatment plan via their device. The server stores this information and, if necessary, begins preparations to implement the treatment plan.

[1285] Specific example

[1286] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. When a user (medical staff) enters this data into a terminal, the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvement suggestions, etc.).

[1287] The generated treatment plans are displayed on the terminal, allowing users (doctors and patients) to review the details of each plan and select a specific course of action. Each treatment plan is accompanied by appropriate comments from the emotion engine (e.g., "This treatment is gentle on the body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., "iron supplementation therapy"), and the server records and saves this selection. The treatment is then implemented.

[1288] Thus, the present invention enables healthcare professionals to quickly provide and consider multiple highly accurate treatment plans within limited time and resources. Furthermore, because treatment plans are presented in an easily understandable format tailored to the patient's emotional state, it promotes treatment choices that are more based on informed consent.

[1289] The following describes the processing flow.

[1290] Step 1:

[1291] Users (medical staff) input blood test results using a terminal. Specifically, they manually enter numerical data such as red blood cell count, white blood cell count, and platelet count into an input form. Electronic data acquisition methods are also available.

[1292] Step 2:

[1293] The user (medical staff) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fatigue or fever) and physical examination results (e.g., heart rate or blood pressure). This information is manually entered into a dedicated input form.

[1294] Step 3:

[1295] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database and integrated for subsequent processing.

[1296] Step 4:

[1297] The server integrates the received blood test results with the results of the medical interview and physical examination. Specifically, it integrates the blood test results with the medical interview and physical examination results and combines them into a single dataset. The integrated data is then converted into an appropriate format.

[1298] Step 5:

[1299] The device recognizes the user's voice, facial expressions, or biometric data in real time and analyzes the user's emotional state using an emotion engine. For example, it recognizes emotions such as anxiety, relief, and stress based on voice recognition, facial expression analysis, and data from biosensors (such as heart rate fluctuations).

[1300] Step 6:

[1301] The server inputs integrated medical data and user sentiment data into a generative artificial intelligence model to generate numerous treatment plans. The generative AI model is pre-tuned and trained on medical datasets.

[1302] Step 7:

[1303] The server generates multiple treatment plans and sends them to the terminal. The terminal displays the candidate treatment plans in a list format, including detailed descriptions and expected outcomes for each plan. Furthermore, appropriate comments and explanations are added based on the user's emotional state as recognized by the emotion engine.

[1304] Step 8:

[1305] Users (patients and doctors) evaluate multiple treatment plans provided on the device. The evaluation considers factors such as treatment effectiveness, cost-effectiveness, and suitability to the user's emotional state and preferences.

[1306] Step 9:

[1307] Users (patients and doctors) select the optimal treatment plan and send that information to the server via their terminals. The selected treatment plan is saved along with the patient's consent.

[1308] Step 10:

[1309] The server saves the selected treatment plan to the database. Based on the saved treatment plan, necessary preparations are made, and the steps toward initiating the specific treatment are taken.

[1310] (Example 2)

[1311] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1312] In modern medical practice, there is a demand for both speed and accuracy in developing treatment plans. However, within limited time and resources, it is difficult for healthcare professionals to process large amounts of data simultaneously and provide optimal treatment plans that take into account the patient's emotional state. As a result, it is difficult to reduce patient anxiety and stress during the selection of a treatment plan, and informed consent becomes challenging. Conventional technologies have not been able to generate treatment plans that take emotional states into account, which has placed a heavy burden on both patients and healthcare professionals.

[1313] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting blood test results by input means, means for inputting interview and palpation results by input means, means for integrating the blood test results with the interview and palpation results, means for inputting the integrated data into a generative artificial intelligence model and generating multiple treatment plans, means for providing the generated treatment plans to medical professionals and patients by display means, means for selecting the optimal treatment plan and saving it by final decision means, means for recognizing the user's emotional state by an emotion engine and processing the data, and means for providing appropriate comments and explanations according to the user's emotional state based on the generated treatment plan. As a result, medical professionals can not only provide treatment plans quickly and accurately, but also provide individualized responses according to the patient's emotional state, enabling the provision of better medical services.

[1314] A "generative artificial intelligence device" is a device that generates new information or results based on data provided by artificial intelligence technology.

[1315] A "medical second opinion" refers to a patient receiving a second opinion from another healthcare professional regarding the treatment or diagnosis they are currently receiving.

[1316] "Blood test results" is a general term for numerical values ​​and data obtained by analyzing a patient's blood, and includes measurement results such as red blood cell count, white blood cell count, and platelet count.

[1317] "Medical interview" refers to the act of a healthcare professional gathering information from a patient about their current symptoms, medical history, and other relevant details.

[1318] "Palpation" refers to the act of a medical professional examining a patient by touching their body with their hands.

[1319] "Integration" refers to the process of combining multiple different sets of data into one.

[1320] A "generative artificial intelligence model" refers to an artificial intelligence model that uses pre-trained algorithms to generate new information or results.

[1321] "Display means" refers to devices or methods that display information to a user through a computer screen, tablet, smartphone, etc.

[1322] "Healthcare professionals" refers to individuals engaged in medical-related professional occupations, such as doctors, nurses, and pharmacists.

[1323] The term "patient" refers to a person receiving medical services.

[1324] The term "final decision-making mechanism" refers to a system or process for storing and implementing selected results or information.

[1325] An "emotion engine" refers to a combination of software and hardware that analyzes a user's emotional state and outputs it as data.

[1326] "Means of providing comments and explanations" refers to devices or methods for adding appropriate explanations to the user based on the generated treatment plan.

[1327] A "database" refers to a system that centrally manages, stores, and allows data to be retrieved as needed.

[1328] "REST API" refers to an application programming interface based on Representational State Transfer, and is a standard for data communication.

[1329] This invention combines a system that provides medical second opinions using a generative artificial intelligence device with an emotion engine that recognizes the user's emotions. This system includes a series of processes for generating an optimal treatment plan using medical data and user emotion data, and for providing that plan to healthcare professionals and patients.

[1330] Hardware configuration

[1331] Servers and terminals (e.g., PCs, tablets, smartphones) are the central hardware components of this system.

[1332] The server receives, integrates, processes, generates, and stores data, as well as treatment plans.

[1333] The device performs data input and display, as well as recognition and transmission of emotional data.

[1334] Software Configuration

[1335] The generative artificial intelligence model is embedded in a server and used to generate treatment plans from medical data. This model is fine-tuned in advance based on medical datasets.

[1336] The emotion engine is built into the device and analyzes the user's voice, facial expressions, and biometric data in real time. This data reflects the user's emotional state and is sent to the server.

[1337] System operation

[1338] The user (healthcare professional) first uses a terminal to input blood test results. This is done either manually or by importing from an electronic medical record system. Similarly, the results of the medical history interview and physical examination are also entered. This data is sent to the server and integrated.

[1339] The emotion engine built into the device analyzes the user's voice, facial expressions, heart rate fluctuations, etc., to generate emotion data. This data is also sent to a server and integrated with medical data.

[1340] The server inputs the integrated data into a generative artificial intelligence model to generate multiple treatment plans. These treatment plans are sent to the terminal and provided to the user. The displayed treatment plans include appropriate comments and explanations based on the user's emotional state as recognized by the emotion engine.

[1341] Users (healthcare professionals and patients) evaluate candidate treatment plans and select the optimal one. The selected treatment plan is notified to the server via the terminal and stored on the server. Subsequently, specific treatment is carried out based on this plan.

[1342] Specific example

[1343] For example, consider a case where a patient complains of fatigue and a mild fever, and blood test results suggest mild anemia. The user enters this data into the terminal, and the server receives and integrates it. Next, based on the patient's emotional data recognized by the emotion engine (e.g., high levels of anxiety) and the integrated data, a generative artificial intelligence model generates multiple treatment plans (e.g., iron supplementation, vitamin B12 supplementation, dietary improvements, etc.). The generated treatment plans are displayed on the terminal, and the user reviews the details of each plan and selects a specific course of action. Each treatment plan is accompanied by an appropriate comment from the emotion engine (e.g., "This treatment is not burdensome on your body, so please don't worry"). The user selects the plan they deem most appropriate (e.g., iron supplementation), and the server records and saves this selection. Subsequently, this treatment is implemented.

[1344] Example of a prompt

[1345] The following are examples of prompts to input into a generative AI model:

[1346] "The patient's current symptoms are fatigue and a mild fever, and blood test results suggest mild anemia. The patient is experiencing significant anxiety. Please propose an optimal treatment plan based on these factors."

[1347] "The patient's blood pressure and heart rate are within the normal range, but their white blood cell count is elevated. The patient is experiencing stress. Please generate a treatment plan considering this situation."

[1348] The above describes a specific embodiment for carrying out the present invention. This system not only enables healthcare professionals to provide rapid and highly accurate treatment plans, but also allows for individualized responses tailored to the patient's emotional state, thereby enabling the provision of better medical services.

[1349] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1350] Step 1:

[1351] Users (medical staff) manually enter blood test results into a dedicated input form on the terminal. The data entered by the user includes numerical data such as red blood cell count, white blood cell count, and platelet count. If an electronic medical record (EMR) system exists, data can be automatically imported from the corresponding EMR system. This improves the accuracy and efficiency of data entry.

[1352] Step 2:

[1353] The user uses a terminal to input the results of the patient's medical history and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). The entered medical history and physical examination data is used for subsequent data integration and analysis.

[1354] Step 3:

[1355] The server receives blood test results, interview and physical examination results sent from the terminal. The received data is stored in a database on the server. The received data includes numerical and text data associated with each patient.

[1356] Step 4:

[1357] The server integrates the received blood test results, interview data, and physical examination results. Specifically, it converts data from different data sources into a standard format and combines them into a single dataset. This ensures that all medical information is presented in a consistently analyzable format.

[1358] Step 5:

[1359] An emotion engine built into the device recognizes the user's emotional state in real time. Emotion recognition uses data from voice, facial expressions, and biosensors (e.g., heart rate and skin electrical activity). The emotion engine analyzes this sensor data to estimate the user's emotional state (e.g., stress, reassurance, anxiety, etc.).

[1360] Step 6:

[1361] The device transmits recognized emotional data to the server. This transmitted emotional data includes specific emotional states and their intensity. This data becomes an important element in the subsequent generation of treatment plans.

[1362] Step 7:

[1363] The server inputs integrated medical and emotional data into a generative artificial intelligence model. This initiates the generation of a treatment plan based on the patient's specific health and emotional state. The input data includes an integrated medical dataset and recognized emotional data.

[1364] Step 8:

[1365] The server generates multiple treatment plans using a generative artificial intelligence model. This model is pre-tuned and trained on medical datasets. The generated treatment plans include specific methods and expected effects.

[1366] Step 9:

[1367] The server generates a treatment plan and sends it to the terminal. The terminal displays a list of potential treatment plans, each containing a detailed explanation and expected outcomes. Furthermore, appropriate comments and explanations are added based on recognized emotional data.

[1368] Step 10:

[1369] Users (patients and healthcare professionals) review potential treatment plans on their devices. Users then view the details of each treatment plan and evaluate the advantages and disadvantages of each plan.

[1370] Step 11:

[1371] The user selects the optimal treatment plan. Evaluation criteria include treatment effectiveness, cost-effectiveness, the user's emotional state, and the patient's wishes.

[1372] Step 12:

[1373] The user's selected treatment plan is notified to the server via the terminal. The server receives this information and saves the selected treatment plan to its database.

[1374] Step 13:

[1375] The server begins preparing for implementation based on the saved treatment plan. Specifically, it arranges for necessary medications and notifies relevant medical staff. This allows the selected treatment plan to be put into practice.

[1376] (Application Example 2)

[1377] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1378] Traditional medical second opinion systems propose treatment plans based on medical data, but they do not take into account the patient's emotional state. Therefore, it is difficult to determine whether the treatment plan is optimal for the patient, and there is a lack of consideration for reducing the patient's anxiety and stress. Furthermore, because relaxation content is not provided to patients undergoing treatment who require emotional support, there is a possibility that the overall effectiveness of the treatment will be reduced.

[1379] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1380] In this invention, the server includes means for inputting blood test results, means for inputting interview and physical examination results, means for analyzing the user's emotional state in real time using emotion recognition technology and integrating the data, means for utilizing a generative artificial intelligence model to generate multiple treatment plans, means for displaying the generated treatment plans to medical professionals and patients, and means for selecting and saving the optimal treatment plan. This makes it possible to provide personalized treatment plans and emotional support based on the user's medical and emotional data.

[1381] A "generative artificial intelligence device" is a device equipped with a model that takes medical data and emotional data as input and generates optimal treatment plans and relaxation content based on them.

[1382] A "medical second opinion" refers to a second expert medical opinion and treatment plan provided to a patient, intended to evaluate the patient's condition and treatment options from a different perspective.

[1383] "Emotion recognition technology" is a technology that identifies a user's emotional state by analyzing voice, facial expressions, or biometric data.

[1384] "Fine-tuning" is the process of fine-tuning a generative artificial intelligence model based on a specific dataset to improve its performance.

[1385] "Display means" refers to a device or interface for visually displaying the generated treatment plan or relaxation content.

[1386] "Relaxation content" refers to media content (such as meditation guides, calming music, and massage videos) designed to reduce patients' stress and anxiety and promote mental stability.

[1387] A "medical dataset" is a collection of data related to medical care, including diagnostic data, test results, and patient symptoms.

[1388] An "emotion dataset" is a collection of data that represents a user's emotional state, and includes data such as voice tone, facial expressions, and biometric data.

[1389] This invention is a system that combines a generative artificial intelligence device with emotion recognition technology to generate optimal treatment plans and relaxation content that take into account the patient's emotional state when providing a second medical opinion. A specific embodiment of this system is described in detail below.

[1390] System Configuration

[1391] This system consists of servers, terminals, and users.

[1392] 1. Enter blood test data

[1393] The terminal has a function for entering blood test results, which users (healthcare professionals) input using an input form or electronic means.

[1394] 2. Input of medical history and physical examination results

[1395] The terminal has a function to input the results of the medical questionnaire and physical examination, and this data is entered in the same way.

[1396] 3. Data Integration

[1397] The server receives blood test results, interview and physical examination results from the terminals, combines them into a single dataset, and converts them into an appropriate format.

[1398] 4. Emotion recognition

[1399] The device incorporates emotion recognition technology that analyzes voice, facial expressions, or biometric data to recognize the user's emotional state in real time. The results of this analysis are then sent to a server.

[1400] 5. Treatment planning and creation of relaxation content

[1401] The server inputs integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. This generative AI model has been pre-trained and fine-tuned using medical and emotional datasets.

[1402] 6. Display and Selection

[1403] The generated treatment plans and relaxation content are displayed on the device, allowing users (healthcare professionals and patients) to review them and select the most suitable plan and content.

[1404] 7. Preservation and Implementation

[1405] The selected treatment plan and relaxation content are notified to the server via the device, and the server stores this information and prepares for implementation as needed.

[1406] Program processing

[1407] The server analyzes blood test results and medical interview data, integrating the acquired data using emotion recognition technology. This process utilizes emotion recognition software such as EmotionEngine. The integrated data is input into a generative artificial intelligence model, which then generates treatment plans and relaxation content. For example, meditation guides, calming music, and massage videos are generated.

[1408] Specific example

[1409] For example, suppose we input health data for a user, such as "heart rate: 72, blood pressure: 120 / 80, symptoms: headache, fatigue," and emotional data such as "high anxiety." Based on this data, we input the following prompt into a generative artificial intelligence model:

[1410] User's health data: Heart rate 72, Blood pressure 120 / 80, Symptoms: Headache, Fatigue

[1411] User sentiment data: High levels of anxiety

[1412] What type of relaxation content is best?

[1413] Based on this prompt, the AI ​​suggests and displays optimal relaxation content (such as meditation guides or calming music) to the user.

[1414] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1415] Step 1:

[1416] The user (healthcare professional) uses a terminal to input blood test results. This can be done manually using an input form or electronically (e.g., by uploading a data file). The input includes numerical data such as red blood cell count, white blood cell count, and platelet count.

[1417] Input: Blood test result data

[1418] Output: Blood test result data stored on the device.

[1419] Step 2:

[1420] The user (healthcare professional) uses a terminal to input the results of the medical interview and physical examination. This includes symptoms the patient is experiencing (e.g., fever, fatigue, etc.) and physical examination results (e.g., heart rate, blood pressure, etc.). This data is also manually entered into a dedicated form.

[1421] Input: Medical history and physical examination results data

[1422] Output: Medical interview and palpation results data stored on the device.

[1423] Step 3:

[1424] The terminal integrates blood test results with interview and physical examination results and sends them to the server. This combines information from different data sources into a single dataset. The data is then converted to an appropriate format, such as JSON.

[1425] Input: Blood test results data, medical interview and physical examination results data

[1426] Output: Integrated dataset sent to the server

[1427] Step 4:

[1428] The device uses emotion recognition technology to analyze the user's voice, facial expressions, and biometric data (e.g., heart rate variability, skin electrical activity, etc.) in real time to recognize their emotional state. This data is also sent to the server.

[1429] Input: User's voice, facial expressions, biometric data

[1430] Output: Sentiment data sent to the server

[1431] Step 5:

[1432] The server inputs received integrated medical data and emotional data into a generative artificial intelligence model to generate multiple treatment plans and relaxation content. By generating prompt sentences and inputting them into the generative AI model, specific treatment plans and content suggestions are obtained.

[1433] Input: Integrative medical data, emotional data

[1434] Output: Generated treatment plan and relaxation content

[1435] Step 6:

[1436] The server sends the generated treatment plan and relaxation content to the device. This includes a detailed explanation of each treatment plan and expected results, as well as comments based on emotion recognition data.

[1437] Input: Generated treatment plan and relaxation content

[1438] Output: Treatment plan and relaxation content displayed on the device.

[1439] Step 7:

[1440] Users (healthcare professionals and patients) review the treatment plan and relaxation content displayed on their terminals and select the most suitable plan. This selection is then communicated to the server.

[1441] Input: Displayed treatment plan and relaxation content

[1442] Output: Selection results sent to the server

[1443] Step 8:

[1444] The server saves the selected treatment plan and relaxation content, and begins specific preparations for the treatment as needed. Details of the treatment plan will be communicated to the necessary stakeholders.

[1445] Input: User-selected treatment plan and relaxation content

[1446] Output: Saved treatment plan and relaxation content, start of treatment preparation

[1447] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1448] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1449] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1450] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1451] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1452] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1453] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1454] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1455] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1456] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1457] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1458] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1459] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1460] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1461] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1462] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1463] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1464] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1465] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1466] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1467] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1468] The following is further disclosed regarding the embodiments described above.

[1469] (Claim 1)

[1470] A system that provides medical second opinions using a generative artificial intelligence device,

[1471] A means for inputting blood test results using an input device,

[1472] A means for inputting the results of the medical interview and physical examination,

[1473] A means of integrating the above blood test results with the results of the medical interview and physical examination,

[1474] A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans,

[1475] A means for providing the generated treatment plan to healthcare professionals and patients via a display means,

[1476] A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism,

[1477] A system that includes this.

[1478] (Claim 2)

[1479] The system according to claim 1, wherein the generative artificial intelligence device is the result of fine-tuning the aforementioned generative artificial intelligence model.

[1480] (Claim 3)

[1481] The system according to claim 1, wherein the aforementioned generative artificial intelligence model is a model trained on a medical dataset.

[1482] "Example 1"

[1483] (Claim 1)

[1484] A means by which the user enters patient health data using a terminal,

[1485] A means of sending data from a terminal to a server,

[1486] A means by which the server integrates the received data and converts it into a format,

[1487] A means of generating multiple treatment plans using an AI model based on integrated data,

[1488] A means of sending the generated treatment plan to a terminal and displaying it to the user,

[1489] A means for the user to select the optimal treatment plan from the displayed treatment plans,

[1490] A means of notifying and saving the selected treatment plan to the server,

[1491] A system that includes this.

[1492] (Claim 2)

[1493] The system according to claim 1, characterized in that the aforementioned generative AI model is fine-tuned in advance.

[1494] (Claim 3)

[1495] The system according to claim 1, characterized in that the aforementioned generative AI model is trained on a medical dataset.

[1496] "Application Example 1"

[1497] (Claim 1)

[1498] A system that provides medical second opinions using a generative artificial intelligence device,

[1499] A means for inputting blood test results using an input device,

[1500] A means for inputting the results of the medical interview and physical examination,

[1501] A means of integrating the above blood test results with the results of the medical interview and physical examination,

[1502] A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans,

[1503] A means for providing the generated treatment plan to healthcare professionals and patients via a display means,

[1504] A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism,

[1505] A method for inputting health data using smart glasses and processing it via a cloud server,

[1506] A means of visualizing treatment plans using the display of smart glasses,

[1507] A system that includes this.

[1508] (Claim 2)

[1509] The system according to claim 1, wherein the generative artificial intelligence device is the result of fine-tuning the aforementioned generative artificial intelligence model.

[1510] (Claim 3)

[1511] The system according to claim 1, wherein the aforementioned generative artificial intelligence model is a model trained on a medical dataset.

[1512] "Example 2 of combining an emotion engine"

[1513] (Claim 1)

[1514] A system that provides medical second opinions using a generative artificial intelligence device,

[1515] A means for inputting blood test results using an input device,

[1516] A means for inputting the results of the medical interview and physical examination,

[1517] A means of integrating the above blood test results with the results of the medical interview and physical examination,

[1518] A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans,

[1519] A means for providing the generated treatment plan to healthcare professionals and patients via a display means,

[1520] A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism,

[1521] A means of recognizing the user's emotional state using an emotion engine and processing that data,

[1522] A means of providing appropriate comments and explanations based on the generated treatment plan, according to the user's emotional state,

[1523] A system that includes this.

[1524] (Claim 2)

[1525] The system according to claim 1, wherein the generative artificial intelligence device is the result of fine-tuning the aforementioned generative artificial intelligence model.

[1526] (Claim 3)

[1527] The system according to claim 1, wherein the aforementioned generative artificial intelligence model is a model trained on a medical dataset.

[1528] "Application example 2 when combining with an emotional engine"

[1529] (Claim 1)

[1530] A system that provides medical second opinions using a generative artificial intelligence device,

[1531] A means for inputting blood test results using an input device,

[1532] A means for inputting the results of the medical interview and physical examination,

[1533] A means of integrating the above blood test results with the results of the medical interview and physical examination,

[1534] A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans,

[1535] A means of analyzing a user's emotional state in real time using emotion recognition technology and integrating that data,

[1536] A means for providing the generated treatment plan to healthcare professionals and patients via a display means,

[1537] A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism,

[1538] A system that includes this.

[1539] (Claim 2)

[1540] The system according to claim 1, wherein the generative artificial intelligence device is the result of fine-tuning the aforementioned generative artificial intelligence model, and includes means for generating relaxation content by integrating emotion recognition technology.

[1541] (Claim 3)

[1542] The system according to claim 1, wherein the aforementioned generative artificial intelligence model is a model trained on a medical dataset and an emotional dataset. [Explanation of Symbols]

[1543] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A system that provides medical second opinions using a generative artificial intelligence device, A means for inputting blood test results using an input device, A means for inputting the results of the medical interview and physical examination, A means of integrating the above blood test results with the results of the medical interview and physical examination, A means for inputting integrated data into a generative artificial intelligence model to generate multiple treatment plans, A means for providing the generated treatment plan to healthcare professionals and patients via a display means, A means for selecting the optimal treatment plan and saving that treatment plan using a final decision-making mechanism, A system that includes this.

2. The system according to claim 1, wherein the generative artificial intelligence device is the result of fine-tuning the aforementioned generative artificial intelligence model.

3. The system according to claim 1, wherein the aforementioned generative artificial intelligence model is a model trained on a medical dataset.

Images