System

Abstract

A system is provided.SOLUTION: This system is provided with a means for receiving inputted medical information, a means for preparing the draft of a clinical document by using a generation AI based on the received medical information, a means for displaying, correcting and confirming the prepared clinical document and a means for storing the confirmed clinical document in a database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention relates to a system that aims to reduce the workload of medical professionals, reduce patient waiting times, and improve the accuracy of initial diagnoses and diagnostic imaging in medical settings. Specifically, the system aims to reduce the burden on medical professionals and improve the efficiency of medical settings by automating the creation of clinical documents, improving the efficiency of medical treatment through automated interviews, automatic analysis of diagnostic imaging, advanced diagnostic analysis, and drug design support. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system including the following means.

[0006] 1. A means for receiving input medical information, creating a draft clinical document using generative AI, displaying the created clinical document, making corrections and confirmations, and storing the confirmed clinical document in a database.

[0007] 2. A means for presenting medical history questions to the patient, receiving the patient's answers, comparing the received answers with a medical database to generate relevant disease name candidates, presenting the generated disease name candidates, and handing them over to a medical professional.

[0008] 3. A means of inputting medical images, analyzing abnormalities using image recognition AI, and generating and displaying the analysis results as a report.

[0009] 4. A means of integrating data collected from multiple patients, performing diagnostic analysis using generative AI, and generating and displaying the results of the diagnostic analysis as a report.

[0010] 5. A means of using generative AI to generate and predict molecular structures for drug design, and storing the generated molecular structures in a database and making them searchable.

[0011] "Medical information" refers to all data related to medical care, such as a patient's medical records, symptoms, test results, and treatment details.

[0012] "Generative AI" is an artificial intelligence technology that generates sentences in natural language based on large amounts of data.

[0013] A "clinical document" is a document created by a medical professional to record the results of medical treatment or diagnosis.

[0014] A "medical interview" is the questions a doctor asks to inquire about a patient's symptoms and medical history.

[0015] A "medical database" is a computer system that stores medical-related information and data and allows them to be searched and collated.

[0016] "Candidate disease names" are disease names that may be inferred from medical interviews and test results.

[0017] "Medical images" refers to image data used in medical care, such as X-rays, MRIs, and CT scans.

[0018] "Image recognition AI" refers to artificial intelligence that has the ability to recognize and analyze image data.

[0019] An "abnormal portion" is a portion where an abnormality is observed compared to a normal state.

[0020] A "report" is a document summarizing the analysis results and diagnosis.

[0021] "Diagnostic analysis" refers to detailed diagnostic analysis based on collected medical data.

[0022] "Molecular structure" refers to the structure that shows the atomic arrangement and bond shape of a molecule.

[0023] "Drug design" refers to the design and improvement of molecular structures that are carried out in the process of developing new drugs.

[0024] A "database" is a system for efficiently storing, managing, and searching large amounts of data. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] This invention is a system that uses generative AI to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[0047] Clinical Documentation Support

[0048] explanation

[0049] This function allows medical professionals to input medical treatment details and examination results, and the generation AI automatically creates a draft of the clinical document, assisting medical professionals in the process of reviewing, correcting, and confirming the content.

[0050] Specific examples

[0051] The user (healthcare worker) inputs the details of the patient's medical treatment. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[0052] The server receives this information and uses generative AI to generate a draft clinical document stating, "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[0053] The terminal displays the generated document, and the user checks it, makes corrections as necessary, and finalizes it as the final document.

[0054] The server stores the established clinical documents in a database.

[0055] Automated medical interview system

[0056] explanation

[0057] This function allows patients to answer questions via their device, and the generative AI then suggests disease names as candidates for initial diagnosis, enabling efficient initial diagnosis.

[0058] Specific examples

[0059] The user (patient) answers the medical interview questions displayed on the terminal. For example, they might enter, "I have had a headache, fever, and cough for three days."

[0060] The server analyzes this information and generates potential disease names such as "cold, flu, COVID-19."

[0061] The device presents these potential disease names to the patient and then passes on the necessary information to medical professionals.

[0062] Diagnostic Imaging System

[0063] explanation

[0064] This feature allows medical professionals to upload medical images (e.g., X-rays, MRIs, etc.), and image recognition AI analyzes abnormalities and generates a report.

[0065] Specific examples

[0066] The user (healthcare worker) inputs the patient's chest X-ray image into the terminal.

[0067] The server analyzes the received images and uses image recognition AI to identify abnormalities (e.g., shadows in the lungs).

[0068] The server generates and displays the analysis results as a report.

[0069] The terminal displays the generated report, which the user (healthcare professional) can check.

[0070] Diagnosis system based on multiple patient information

[0071] explanation

[0072] This feature integrates data collected from multiple patients and uses generative AI to perform highly accurate diagnostic analysis.

[0073] Specific examples

[0074] The user (healthcare professional) inputs the medical records, vital data, and diagnostic imaging data of Patients A, B, and C into the terminal.

[0075] The server integrates this data and uses generative AI to perform a diagnostic analysis for each patient.

[0076] The server compiles diagnostic results into a report, such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having a decline in kidney function."

[0077] The terminal displays the generated report, and the user (medical professional) checks it and makes a final diagnosis.

[0078] Pharmaceutical research and design support

[0079] explanation

[0080] This feature uses generative AI to generate and predict molecular structures of new drugs, supporting the drug design process.

[0081] Specific examples

[0082] The user (researcher) inputs the conditions necessary for designing a new drug into the terminal.

[0083] The server uses generative AI to generate and predict candidate molecular structures.

[0084] The server stores the generated molecular structures in a database, allowing users (researchers) to search and use them as needed.

[0085] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI to various medical tasks.

[0086] The processing flow will be explained below.

[0087] Clinical Documentation Support

[0088] Processing Steps

[0089] Step 1:

[0090] The user (healthcare worker) uses a terminal to input details of the patient's medical treatment, including symptoms, examination results, and treatment details.

[0091] Step 2:

[0092] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[0093] Step 3:

[0094] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, including the diagnosis and treatment plan.

[0095] Step 4:

[0096] The server transmits the generated draft of the clinical document to the terminal.

[0097] Step 5:

[0098] The terminal displays the received draft of the clinical document on the screen, and the user (healthcare professional) can check and correct it.

[0099] Step 6:

[0100] The user inputs the corrections and presses the confirm button.

[0101] Step 7:

[0102] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[0103] Automated medical interview system

[0104] Processing Steps

[0105] Step 1:

[0106] The terminal displays medical interview questions to the patient on the screen.

[0107] Step 2:

[0108] The user (patient) answers questions and inputs their symptoms and physical condition.

[0109] Step 3:

[0110] The terminal transmits the input answer to the server.

[0111] Step 4:

[0112] The server analyzes the received responses and compares them with a medical database to generate relevant disease name candidates.

[0113] Step 5:

[0114] The server sends the generated disease name candidates to the terminal.

[0115] Step 6:

[0116] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[0117] Diagnostic Imaging System

[0118] Processing Steps

[0119] Step 1:

[0120] The user (medical professional) inputs medical images (e.g., X-rays, MRIs, etc.) into the terminal.

[0121] Step 2:

[0122] The terminal transmits the input image data to the server.

[0123] Step 3:

[0124] The server analyzes the received image data and identifies any abnormalities using image recognition AI.

[0125] Step 4:

[0126] The server generates a report based on the analysis results.

[0127] Step 5:

[0128] The server sends the generated report to the terminal.

[0129] Step 6:

[0130] The terminal displays the report, which the user (healthcare professional) can review.

[0131] Diagnosis system based on multiple patient information

[0132] Processing Steps

[0133] Step 1:

[0134] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[0135] Step 2:

[0136] The terminal transmits the input data to the server.

[0137] Step 3:

[0138] The server integrates the received data and performs diagnostic analysis using generative AI.

[0139] Step 4:

[0140] The server generates a diagnostic report based on the analysis results.

[0141] Step 5:

[0142] The server transmits the generated diagnostic report to the terminal.

[0143] Step 6:

[0144] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[0145] Pharmaceutical research and design support

[0146] Processing Steps

[0147] Step 1:

[0148] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[0149] Step 2:

[0150] The terminal transmits the input conditions to the server.

[0151] Step 3:

[0152] The server uses generative AI to generate and predict candidate molecular structures.

[0153] Step 4:

[0154] The server stores the generated molecular structures in a database.

[0155] Step 5:

[0156] The user (researcher) searches for the information they need and displays it on their terminal.

[0157] Example 1

[0158] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] The purpose of this invention is to reduce the wide-ranging workload of medical professionals and improve efficiency in the medical field. Conventional systems require a lot of manual work, such as document creation, medical interviews, image diagnosis, diagnostic analysis of multiple patients, and pharmaceutical research, which is time-consuming and labor-intensive, and also has a high risk of human error. This has resulted in an excessive workload for medical professionals, making it difficult to provide prompt and accurate medical care.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0161] In this invention, the server includes: [means for receiving input medical data;] [means for creating a draft of a clinical document using a generative AI model based on the received medical data;] [means for displaying the created clinical document and correcting and confirming it; and] [means for saving the confirmed clinical document in a database. This automates document creation work in medical settings, reducing time and effort and human error.

[0162] "Medical data" refers to a wide range of data related to medical care, such as patient medical information, vital signs, diagnostic imaging data, and medical records.

[0163] A "generative AI model" refers to an artificial intelligence algorithm that uses technologies such as natural language processing and image recognition to generate output tailored to a specific purpose from input data.

[0164] "Clinical documents" refer to important documents in the medical field that contain information such as medical treatment details and examination results.

[0165] "Medical interview items" refer to questions presented to understand the patient's symptoms, medical history, and current health condition.

[0166] "Candidate disease names" refers to a list of possible disease names generated based on medical interview responses and other medical data.

[0167] "Medical images" refers to medical image data such as X-rays, MRIs, and CT scans.

[0168] "Image analysis AI" refers to artificial intelligence algorithms that analyze medical images and identify or diagnose abnormalities.

[0169] "Patient data" refers to all medical information related to individual patients, including medical information, vital signs, and diagnostic imaging data.

[0170] "Diagnosis result" refers to a diagnostic conclusion generated through analysis based on medical data and patient data.

[0171] "Drug design criteria" refers to the scientific and clinical requirements and specifications for designing new drugs.

[0172] "Molecular structure" refers to the internal structure of a drug or other compound, represented as a chemical formula or 3D model.

[0173] A "report" is a document that documents the results of analysis or diagnosis and is compiled in a format that can be reviewed and used by medical professionals.

[0174] "Database" refers to a computer system for systematically storing and managing information such as medical data, clinical documents, and molecular structures.

[0175] This invention is a system that uses generative AI models to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[0176] Hardware and software used

[0177] The system's main hardware consists of a server and a terminal. The server receives and analyzes data, runs the generative AI model, and stores the data. The terminal provides an interface with the user, allowing data input and display of results.

[0178] Specific software includes:

[0179] Generative AI models: Natural language processing models such as OpenAI's GPT-4

[0180] Image analysis AI: Image recognition services such as Google Cloud Vision and AWS Rekognition

[0181] Database: A data management system such as Firebase or PostgreSQL

[0182] Clinical Documentation Support

[0183] The user (healthcare professional) inputs the details of the medical treatment from the terminal. For example, they input data such as "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal then sends the input details of the medical treatment to the server. The server inputs the received data as prompts into the generative AI model, generating a draft clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal displays the generated document, and the user can review it, make corrections as necessary, and finalize it as the final document. The server then saves the finalized clinical document in a database.

[0184] Example prompt: "Please use this clinical content to draft a clinical document."

[0185] Automated medical interview system

[0186] The user (patient) answers the medical interview questions displayed on the device. For example, they might enter, "I have had a headache, fever, and cough for three days." The device then sends the entered medical interview data to the server. The server then inputs the received data as prompts into the generative AI model, generating disease name candidates such as "cold, flu, COVID-19." The device then presents these disease name candidates to the patient and then conveys the necessary information to medical professionals.

[0187] Example prompt: "Please list possible diseases based on this interview data."

[0188] Diagnostic Imaging System

[0189] The user (healthcare professional) uploads an image file (e.g., a chest X-ray image) to the device. The device sends the image file to the server. The server inputs the received image into the image analysis AI and identifies abnormal areas (e.g., shadows in the lungs). The image analysis AI generates a report of the analysis results. The server sends the generated report to the device, which displays the report and the user (healthcare professional) confirms it.

[0190] Example prompt: "Analyze this image to identify anomalies and generate a report."

[0191] Diagnosis system based on multiple patient information

[0192] The user (healthcare professional) inputs multiple patient information (e.g., medical record information, vital signs, diagnostic imaging data) into the terminal. The terminal sends all input data to the server. The server integrates the received data and inputs it as prompts into the generative AI model, performing diagnostic analysis for each patient. For example, it generates a diagnosis result such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having decreased renal function." The server then sends the generated diagnosis result as a report to the terminal, which displays the report and the user confirms it.

[0193] Example prompt: "Generate a diagnosis based on these patient data."

[0194] Pharmaceutical research and design support

[0195] The user (researcher) inputs the conditions necessary for designing a new drug into a terminal. The terminal then sends the input design conditions to the server. The server inputs the received conditions as prompts into the generative AI model, generating and predicting candidate molecular structures. The server stores the generated molecular structures in a database, allowing users to search and use them as needed.

[0196] Example prompt: "Generate molecular structure candidates based on these conditions."

[0197] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI models to various medical tasks.

[0198] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0199] Clinical Documentation Support

[0200] Step 1: The user inputs the details of the medical treatment from the terminal.

[0201] Specifically, the user uses a keyboard or voice input to input, "The patient complains of pain in the right shoulder, and examination reveals no fractures."

[0202] Step 2: The terminal sends the entered medical details to the server.

[0203] The input is the medical data, and the output is the transmitted data. HTTPS is used as the communication protocol.

[0204] Step 3: The server inputs the received data into the generative AI model as prompts.

[0205] The input is medical content data, and the output is a prompt: "Please use this medical content to draft a clinical document." The generative AI model analyzes the received data and generates an appropriate draft clinical document.

[0206] Step 4: The generative AI model returns the generated draft clinical document to the server.

[0207] The input is the analysis result data, and the output is the generated draft clinical document.

[0208] Step 5: The server sends the generated draft to the device.

[0209] The input is the draft data, and the output is the submitted draft data.

[0210] Step 6: The terminal displays the generated document, which the user can review and modify if necessary.

[0211] As a specific operation, the user uses a text editor to edit a document on the screen of the terminal.

[0212] Step 7: The user finalizes the document.

[0213] The input is the corrected document and the output is the finalized document.

[0214] Step 8: The terminal sends the confirmed document to the server.

[0215] The input is the finalized document data, and the output is the transmitted data.

[0216] Step 9: The server saves the finalized document to the database.

[0217] The input is the finalized document data, and the output is the data stored in the database.

[0218] Automated medical interview system

[0219] Step 1: The user (patient) answers the medical interview questions displayed on the terminal.

[0220] Specifically, in response to the question, enter "I have had a headache, fever, and cough for three days" in the text box.

[0221] Step 2: The terminal transmits the entered medical interview data to the server.

[0222] The input is the medical interview response data, and the output is the transmitted data.

[0223] Step 3: The server inputs the received data into the generative AI model as prompts.

[0224] The input is the patient's interview response data, and the output is a prompt statement: "Please list possible disease names based on this interview data."

[0225] Step 4: The generative AI model generates disease name candidates based on the received data.

[0226] The input is medical interview response data, and the output is disease name candidate data.

[0227] Step 5: The server sends the generated disease name candidates to the terminal.

[0228] The input is the disease name candidate data, and the output is the transmitted data.

[0229] Step 6: The terminal presents the patient with possible disease names and then conveys the necessary information to the medical professional.

[0230] Specifically, the system displays possible disease names on the screen, such as "cold, influenza, COVID-19," and the user confirms them.

[0231] Diagnostic Imaging System

[0232] Step 1: The user (healthcare professional) uploads an image file to the device.

[0233] As a specific operation, a file selection dialog is used to select a chest X-ray image.

[0234] Step 2: The device sends the image file to the server.

[0235] The input is the image file and the output is the transmitted data.

[0236] Step 3: The server inputs the received image into the image analysis AI.

[0237] The input is an image file, and the output is a prompt: "Analyze this image to identify any abnormalities and generate a report."

[0238] Step 4: Image analysis AI analyzes the pixel data of the image and identifies abnormalities.

[0239] The input is an image file, and the output is the analysis result data.

[0240] Step 5: The image analysis AI generates a report of the analysis results.

[0241] The input is analysis result data, and the output is report data.

[0242] Step 6: The server sends the generated report to the terminal.

[0243] The input is the report data and the output is the transmitted data.

[0244] Step 7: The terminal displays the report and the user (healthcare professional) confirms it.

[0245] Specifically, the generated report is displayed on the screen and checked by a medical professional.

[0246] Diagnosis system based on multiple patient information

[0247] Step 1: A user (healthcare worker) inputs multiple patient information into a terminal.

[0248] Specifically, the patient enters medical record information and vital data into a form.

[0249] Step 2: The device sends all entered data to the server.

[0250] The input is a plurality of patient information data, and the output is the transmitted data.

[0251] Step 3: The server consolidates the received data.

[0252] The input is multiple patient information data, and the output is the integrated data.

[0253] Step 4: The server inputs the integrated data as prompts into the generative AI model.

[0254] The input is the integrated data, and the output is the prompt statement "Please generate a diagnosis based on these patient data."

[0255] Step 5: The generative AI model generates a diagnosis based on the integrated data.

[0256] The input is the integrated data, and the output is the diagnostic result data.

[0257] Step 6: The server sends the generated diagnostic results to the terminal as a report.

[0258] The input is the diagnostic result data, and the output is the transmitted data.

[0259] Step 7: The terminal displays the report, which the user confirms and makes a final diagnosis.

[0260] Specifically, the report is displayed and a medical professional makes a final diagnosis.

[0261] Pharmaceutical research and design support

[0262] Step 1: The user (researcher) inputs the design conditions for a new drug into the terminal.

[0263] As a specific operation, the design conditions are input into the text box.

[0264] Step 2: The terminal transmits the input design conditions to the server.

[0265] The input is the design condition data, and the output is the transmitted data.

[0266] Step 3: The server inputs the received conditions into the generative AI model as prompts.

[0267] The input is design condition data, and the output is a prompt statement: "Generate molecular structure candidates based on these conditions."

[0268] Step 4: The generative AI model generates candidate molecular structures based on the conditions.

[0269] The input is design condition data, and the output is molecular structure candidate data.

[0270] Step 5: The server stores the generated molecular structure in a database.

[0271] The input is molecular structure candidate data, and the output is the saved data.

[0272] Step 6: The user searches for and uses molecular structures from the terminal as needed.

[0273] Specifically, the molecular structure search interface is used to search and evaluate stored molecular structures.

[0274] The specific processing steps for each function have been described above.

[0275] (Application example 1)

[0276] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0277] Conventional medical support systems require medical personnel to be present at all times, making it difficult to respond quickly in emergencies. In particular, emergency medical support systems using mobile means such as autonomous vehicles have difficulty providing first aid and diagnosis when medical personnel are not present on board, making it impossible to provide appropriate medical support.

[0278] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0279] In this invention, the server includes a means for receiving input medical information, a means for creating a draft clinical document using a generative artificial intelligence based on the received medical information, a means for displaying, correcting, and confirming the created clinical document, a means for inputting patient symptoms in an emergency and starting processing, and a means for using the generative artificial intelligence to create instructions for first aid in real time based on the input symptoms. This enables the autonomous vehicle to provide prompt and appropriate first aid and safely transport the patient even when medical personnel are not on-site.

[0280] "Medical information" refers to all data related to patients, such as medical treatment details, symptoms, examination results, and medical images.

[0281] "Generative AI" is an AI technology that automatically generates natural language sentences and reports from given data.

[0282] "Clinical documents" are official documents for medical professionals that contain information such as examination results and treatment plans.

[0283] "Emergency" refers to situations where medical personnel are not available or where immediate first aid is required.

[0284] "Real-time first aid instructions" are specific instructions for performing emergency medical procedures that are instantly generated using signal processing and generative AI.

[0285] A "medical history" is a series of questions asked to understand a patient's symptoms and medical history.

[0286] A "medical database" is a system that accumulates and manages medical information and case data.

[0287] "Medical images" are image data obtained using medical equipment such as X-rays and MRIs.

[0288] "Image recognition artificial intelligence" is an artificial intelligence technology used to analyze medical images and identify abnormalities.

[0289] "Instructions for first aid based on diagnostic results" are specific treatment procedures and treatment details created by the generative artificial intelligence based on the diagnostic results.

[0290]

[0291] This invention provides a concrete implementation of a generative AI-based emergency medical support system to be installed in an autonomous vehicle. This system enables rapid first aid in emergencies and is primarily composed of the following main components:

[0292] 1. Server:

[0293] The server is the central system that performs various processes using generative AI models. The server is responsible for the following processes:

[0294] Receiving and generating medical information. Drafting clinical documentation using AI.

[0295] Generate emergency first aid instructions based on patient symptom information.

[0296] An initial diagnosis is made based on the results of the interview, after comparing them with a medical database.

[0297] The server software includes generative AI models using TensorFlow and Keras, and a database management system.

[0298] 2. Terminal:

[0299] The terminal is a device such as a smartphone or smart glasses that provides an interface for patients and medical professionals to input and view information. The terminal has the following functions:

[0300] The user (patient or healthcare professional) enters symptoms and medical questionnaire answers.

[0301] Displays generated documents and diagnostic results from the server.

[0302] Displays emergency procedure instructions as needed.

[0303] For example, a patient may type into their smartphone, "I have severe chest pain and difficulty breathing." The device can also automatically input information using voice commands.

[0304] 3. User:

[0305] The user is a patient or a medical professional who uses the system. The user performs the following operations:

[0306] Enter your symptoms in case of an emergency.

[0307] Respond based on emergency instructions from the server.

[0308] Review and correct clinical documentation as needed.

[0309] The user can utilize the system through an intuitive interface, maximizing the effectiveness of the invention.

[0310] The following are examples of prompt sentences:

[0311] Example prompt sentence:

[0312] "The patient is experiencing chest pain and difficulty breathing. Urgent treatment is needed. What are the possible diagnoses? What first aid would be needed?"

[0313] Specific examples of hardware include self-driving vehicles, smartphones, smart glasses, and IoT medical devices (e.g., robotic arms, first aid kits).

[0314] Specific examples of software include generative AI models (TensorFlow / Keras models), real-time image analysis software, and control software for IoT devices.

[0315] In this way, the present invention realizes a system that can provide prompt and appropriate first aid in an autonomous vehicle even when medical personnel are not present.

[0316] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0317] Step 1:

[0318] The user inputs symptoms into a smartphone or smart glasses. For example, if the user inputs "severe chest pain and difficulty breathing," the device receives this input. The input data includes the patient's symptoms and condition.

[0319] Step 2:

[0320] The device sends the symptom information it receives to the server. At this time, the symptom information is transferred to the server as text data. The server receives this text data and inputs it into the generative AI model.

[0321] Step 3:

[0322] The server uses generative AI to create a draft clinical document from the received symptom information. The generative AI model generates an initial diagnosis based on the input data. Here, the prompt used is "The patient complains of severe chest pain and difficulty breathing. What are some possible diagnosis candidates?" As a result, a candidate diagnosis and first aid instructions are generated.

[0323] Step 4:

[0324] The server sends the generated clinical document and candidate diagnoses to the terminal, which displays the generated document for the user to review. The user can then review the displayed information and make corrections as necessary.

[0325] Step 5:

[0326] The server stores the clinical document in a database after the user has confirmed and corrected it. The confirmed document is stored in the medical database for future reference and use.

[0327] Step 6:

[0328] The server generates emergency first aid instructions based on the initial diagnosis. Using generative artificial intelligence, it generates instructions in real time, including specific first aid procedures. For example, it creates "operation instructions for the robot arm to begin cardiac massage."

[0329] Step 7:

[0330] The server sends the generated first aid instructions to the IoT device. The terminal receives the instructions and sends a signal to control a nearby medical device (e.g., a robotic arm). Emergency first aid is then performed based on the specific control procedure.

[0331] Step 8:

[0332] The terminal displays the final instructions sent from the server and the manufactured treatment details to the user, who can then continue responding to the emergency based on this information.

[0333] In this way, the system can provide fast and accurate real-time first aid in emergencies.

[0334] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0335] This invention is an integrated system that uses generative AI and an emotion engine to support medical professionals in their work and improve communication with patients. This system combines clinical documentation support, an automated medical interview system, an image diagnosis system, a diagnosis system using multiple patient information, and pharmaceutical research and design support functions with an emotion engine.

[0336] Clinical Documentation Support

[0337] explanation

[0338] This function allows medical professionals to input medical treatment details and examination results, and the generative AI automatically creates a draft of the clinical document, uses an emotion engine to recognize the medical professional's emotions, and assists in the process of creating an appropriate document based on the input content.

[0339] Specific examples

[0340] The user (healthcare worker) inputs the details of the patient's medical treatment into the terminal, for example, "The patient complains of pain in the right shoulder, and the examination reveals no fractures."

[0341] The device sends this information to a server, which then uses a generative AI to generate a draft clinical document stating, "The patient's main complaint was pain in the right shoulder, and examination revealed no fractures."

[0342] Additionally, the device uses an emotion engine to analyze the user's emotions as they type and detect signs of stress or fatigue.

[0343] The server adjusts the relevance of document content depending on the emotional state, for example, producing quicker and more concise documents.

[0344] The terminal displays the generated document, and the user checks and corrects it, and then presses the confirm button to confirm it.

[0345] The server stores the established clinical documents in a database.

[0346] Automated medical interview system

[0347] explanation

[0348] This function allows patients to answer questions via their device, and the generative AI then suggests possible disease names for initial diagnosis, using an emotion engine to recognize the patient's emotions and adjust the interview process accordingly.

[0349] Specific examples

[0350] The patient enters into the terminal, "I have had a headache, fever, and cough for three days."

[0351] The device sends this information to a server, which then uses generative AI to generate potential disease names such as "cold, flu, COVID-19."

[0352] At the same time, the device uses an emotion engine to check the patient's emotions and detect, for example, anxiety or tension.

[0353] The server adjusts the display of the medical questionnaire based on the results of the emotion analysis, changing the explanation of the possible disease name to easier-to-understand language and asking additional questions.

[0354] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[0355] Diagnostic Imaging System

[0356] explanation

[0357] This function allows medical professionals to upload medical images and uses image recognition AI and an emotion engine to analyze abnormalities, supporting diagnosis while reducing stress.

[0358] Specific examples

[0359] A medical professional inputs the patient's chest X-ray image into a terminal.

[0360] The device sends image data to a server, which then uses image recognition AI to identify abnormalities.

[0361] At the same time, the device uses an emotion engine to check the current emotional state of medical workers and detect high levels of stress or fatigue.

[0362] The server generates a report based on the analysis results and tailors the report content depending on the emotional state, for example providing additional analytical details to help reduce stress.

[0363] The terminal displays the report, which is then reviewed by medical personnel.

[0364] Diagnosis system based on multiple patient information

[0365] explanation

[0366] This function integrates data collected from multiple patients, uses generative AI and an emotion engine to perform highly accurate diagnostic analysis, and presents results based on the emotional state of medical professionals.

[0367] Specific examples

[0368] Medical staff enter the medical records, vital signs, and diagnostic imaging data of patients A, B, and C into the terminal.

[0369] The device sends this data to a server, which uses generative AI to perform a diagnostic analysis for each patient.

[0370] The emotion engine monitors the emotional state of healthcare professionals during input and detects high levels of fatigue and stress.

[0371] The server compiles the diagnostic results into a report, adapts the content depending on the emotional state, and provides additional information and analysis results to, for example, reduce the burden on medical professionals.

[0372] The terminal displays the report, which is then reviewed and a final diagnosis is made by a medical professional.

[0373] Pharmaceutical research and design support

[0374] explanation

[0375] This feature uses generative AI to generate and predict molecular structures of new drugs, and an emotion engine to recognize researchers' emotions while supporting the drug design process.

[0376] Specific examples

[0377] Researchers input the conditions necessary for new drug design into the terminal.

[0378] The device sends this information to a server, which then uses generative AI to generate and predict candidate molecular structures.

[0379] At the same time, the device uses an emotion engine to analyze the researcher's emotional state and detect stress and fatigue.

[0380] The server stores the generated list of molecular structures in a database and presents the researcher with the most appropriate information based on their emotional state, for example by adding detailed annotations for complex molecular structures.

[0381] Researchers use the terminal to search for the necessary information and check the displayed molecular structure.

[0382] As described above, by applying generative AI and an emotion engine to various medical tasks, the present invention reduces the burden on medical professionals, improves efficiency in the medical field, and enables smooth communication with patients.

[0383] The processing flow will be explained below.

[0384] Clinical Documentation Support

[0385] Processing Steps

[0386] Step 1:

[0387] The user (healthcare worker) inputs the details of the patient's medical treatment into the terminal. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[0388] Step 2:

[0389] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[0390] Step 3:

[0391] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, such as "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[0392] Step 4:

[0393] The device uses an emotion engine to analyze the user's emotions, for example, detecting signs of stress or fatigue through facial recognition and voice tone analysis.

[0394] Step 5:

[0395] The server adjusts document content to take into account the user's emotional state, for example by generating a more concise document if the user is feeling stressed.

[0396] Step 6:

[0397] The server transmits the generated document to the terminal.

[0398] Step 7:

[0399] The terminal displays the generated clinical document on the screen, allowing the user to check and correct it.

[0400] Step 8:

[0401] After completing the corrections, the user presses the confirm button.

[0402] Step 9:

[0403] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[0404] Automated medical interview system

[0405] Processing Steps

[0406] Step 1:

[0407] The terminal displays medical interview questions to the patient on the screen.

[0408] Step 2:

[0409] The user (patient) answers questions and inputs their symptoms and physical condition. For example, they might input "I have had a headache, fever, and cough for three days."

[0410] Step 3:

[0411] The terminal receives the response and transmits the data to the server.

[0412] Step 4:

[0413] The server analyzes the received response and compares it with a medical database to generate relevant disease name candidates, such as "cold, flu, COVID-19."

[0414] Step 5:

[0415] The device uses an emotion engine to analyze the patient's emotions and detect feelings such as anxiety and tension.

[0416] Step 6:

[0417] The server takes into account the patient's emotional state and adjusts the content of the interview, for example by making the explanation of potential illnesses more concise and easy to understand if anxiety is detected.

[0418] Step 7:

[0419] The server sends the generated disease name candidates to the terminal and displays them to the patient.

[0420] Step 8:

[0421] The terminal displays possible disease names and then sends the necessary information to medical professionals.

[0422] Diagnostic Imaging System

[0423] Processing Steps

[0424] Step 1:

[0425] A user (medical worker) inputs a medical image into the terminal, for example, uploading a chest X-ray image.

[0426] Step 2:

[0427] The terminal receives the image data and transmits the data to the server.

[0428] Step 3:

[0429] The server analyzes the received image data and uses image recognition AI to identify abnormalities, such as shadows in the lungs.

[0430] Step 4:

[0431] The device uses an emotion engine to analyze the emotions of medical staff, for example, to detect fatigue and stress.

[0432] Step 5:

[0433] The server generates a report of the analysis results, taking into account the emotional state, for example, if fatigue is detected, it will briefly summarize the analysis results.

[0434] Step 6:

[0435] The server sends the generated report to the terminal.

[0436] Step 7:

[0437] The terminal displays the report, which the user (healthcare professional) can review.

[0438] Diagnosis system based on multiple patient information

[0439] Processing Steps

[0440] Step 1:

[0441] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[0442] Step 2:

[0443] The terminal transmits the input data to the server.

[0444] Step 3:

[0445] The server integrates the received data and performs diagnostic analysis using generative AI, generating diagnostic results such as, for example, that Patient A is at high risk of myocardial infarction, Patient B is suspected of having pneumonia, and Patient C is suspected of having declining kidney function.

[0446] Step 4:

[0447] The device uses an emotion engine to analyze the emotions of medical workers, for example, to detect stress or fatigue.

[0448] Step 5:

[0449] The server generates a report of the analysis results, taking into account the emotional state, for example, if stress is detected, it will summarize the diagnosis in a concise and easy-to-understand manner.

[0450] Step 6:

[0451] The server transmits the generated diagnostic report to the terminal.

[0452] Step 7:

[0453] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[0454] Pharmaceutical research and design support

[0455] Processing Steps

[0456] Step 1:

[0457] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[0458] Step 2:

[0459] The terminal transmits the input conditions to the server.

[0460] Step 3:

[0461] The server uses generative AI to generate and predict candidate molecular structures.

[0462] Step 4:

[0463] The device uses an emotion engine to analyze the researcher's emotions, for example, to detect stress or fatigue.

[0464] Step 5:

[0465] The server takes emotional state into account to tailor the molecular structure generation results, for example by adding detailed annotations.

[0466] Step 6:

[0467] The server stores the generated molecular structures in a database and makes them available for user retrieval.

[0468] Step 7:

[0469] The terminal displays the search results, which the user (researcher) can confirm.

[0470] Example 2

[0471] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0472] In the current medical system, medical professionals are burdened with a heavy workload and are unable to properly manage their emotions and stress, resulting in problems with smooth communication with patients. Furthermore, emotional adjustments are not made during interviews or diagnostic imaging, which can lead to reduced diagnostic accuracy and work efficiency. It is necessary to solve these problems and provide a better environment for both medical professionals and patients.

[0473] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: [means for receiving input medical information]; [means for creating a draft of a clinical document using a generation AI based on the received medical information]; [means for analyzing the user's emotions using an emotion engine]; [means for adjusting the clinical document content based on the emotion analysis results]; [means for displaying the created clinical document, correcting and confirming it]; and [means for saving the confirmed clinical document in a database]. This enables document creation that takes into account the emotional state of medical professionals, thereby realizing effective communication.

[0474] The server also includes means for presenting medical interview questions to the patient and receiving the patient's responses, means for comparing the received responses with a medical database and generating relevant disease name candidates, means for analyzing the patient's emotions using an emotion engine, means for adjusting the method for presenting the disease name candidates based on the emotion analysis results, and means for presenting the generated disease name candidates to a medical professional. This enables medical interviews that take the patient's emotions into consideration, resulting in more accurate initial diagnoses.

[0475] Furthermore, the server includes: [means for inputting medical images and analyzing abnormalities using image recognition AI; [means for analyzing the emotions of medical professionals using an emotion engine;] [means for adjusting the report content based on the emotion analysis results; and [means for generating and displaying the analysis results as a report.] This enables image diagnosis that takes into account the emotional state of medical professionals, thereby improving diagnostic accuracy.

[0476] "Inputted medical information" refers to medical-related information such as medical treatment details, symptoms, and vital data that medical professionals and patients provide to the system.

[0477] "Generative AI" is a system that uses artificial intelligence technology to automatically generate text, images, etc., and specifically refers to a model that uses natural language processing technology.

[0478] "Clinical documents" are documents in which medical professionals record the details of medical treatment and examinations, including the patient's medical history, symptoms, and examination results.

[0479] An "emotion engine" is a technology that analyzes the emotional state of a user or patient and adjusts the system's operation and output based on that emotion.

[0480] The "emotion analysis result" is the result of the emotion analyzed by the emotion engine, and is data indicating the state of stress, fatigue, anxiety, etc. of the user or patient.

[0481] "Medical history questions" are a set of questions presented by the system to understand the patient's symptoms and medical history.

[0482] "Responses" are responses given by patients to medical interview questions, and include information about symptoms and physical conditions.

[0483] A "medical database" is a database that stores medical information, diagnosis results, disease name information, etc., and is used as reference information for medical interviews and diagnoses.

[0484] "Candidate disease names" is a list of possible disease names generated by the system based on the patient's answers and data received.

[0485] "Medical images" are medical image data taken for diagnosis or treatment, such as X-ray images, CT scan images, and MRI images.

[0486] "Image recognition AI" is an artificial intelligence technology that automatically analyzes medical images and identifies abnormalities and features.

[0487] A "report" is a document of diagnostic and analytical results generated by the system, intended to provide information to medical professionals.

[0488] A "database" is a system for efficiently storing, searching, and managing large amounts of data, and in this case is used to store clinical documents and medical information.

[0489] "Display" refers to the act of showing the data or documents generated by the system to the user or patient, and means outputting them on a screen.

[0490] "Modification" refers to the act of a user making changes or corrections to a generated document or data.

[0491] "Confirming" refers to the act of the user finally confirming the corrections and saving them as official data.

[0492] This invention is an integrated system that uses generative AI and an emotion engine to support medical professionals in their work and improve communication with patients. This system combines the emotion engine with clinical documentation support, an automated medical interview system, an image diagnosis system, a diagnosis system using multiple patient information, and pharmaceutical research and design support functions.

[0493] Clinical Documentation Support

[0494] explanation

[0495] With this function, the user (medical professional) inputs the details of medical treatment and examination results into the device, and the generative AI automatically creates a draft of the clinical document based on that information.The emotion engine analyzes the user's emotional state and adjusts the appropriateness of the document according to the input content.

[0496] Specific examples

[0497] The user enters "The patient complains of pain in the right shoulder, and examination revealed no fractures" into the terminal. The terminal sends this information to the server, which uses a generative AI to generate a draft clinical document stating, "The patient complains of pain in the right shoulder, and examination revealed no fractures." At the same time, the terminal uses an emotion engine to analyze the user's emotions as they type and detect signs of stress or fatigue. The server adjusts the content of the document according to the user's emotional state, creating a concise document. The terminal displays the generated document, and the user can review and edit it, then press the confirm button to confirm. The server then saves the confirmed clinical document in its database.

[0498] Prompt Sentence Examples

[0499] The patient complained of pain in the right shoulder, and examination revealed no fractures.

[0500] Automated medical interview system

[0501] explanation

[0502] In the automated medical interview system, patients answer questions via a device, and the generative AI then suggests possible initial diagnosis names. It also uses an emotion engine to recognize the patient's emotions and adjust the interview process accordingly.

[0503] Specific examples

[0504] The patient enters into the device, "I have had a headache, fever, and cough for three days." The device sends this information to the server, which uses generative AI to generate disease name candidates such as "cold, flu, and COVID-19." At the same time, the device uses an emotion engine to check the patient's emotional state and detect anxiety or tension. Based on the emotion analysis results, the server adjusts the display of the medical interview content, changing the explanation of the disease name candidates to easier-to-understand language or asking additional questions. The device displays the disease name candidates to the patient and then conveys the necessary information to medical professionals.

[0505] Prompt Sentence Examples

[0506] Headache, fever, and cough for 3 days

[0507] Diagnostic Imaging System

[0508] explanation

[0509] In the image diagnosis system, when medical professionals upload medical images, the image recognition AI analyzes abnormalities. An emotion engine is also used to analyze the emotional state of the medical professionals and support the diagnosis.

[0510] Specific examples

[0511] A medical professional inputs a patient's chest X-ray image into a terminal. The terminal then sends the image data to a server, which uses image recognition AI to identify abnormalities. At the same time, the terminal uses an emotion engine to analyze the medical professional's emotional state and detect high levels of stress or fatigue. The server generates a report based on the analysis results and adjusts the report content according to the medical professional's emotional state. The terminal then displays the generated report for the medical professional to review.

[0512] Prompt Sentence Examples

[0513] Analysis of chest X-ray images

[0514] Diagnosis system based on multiple patient information

[0515] explanation

[0516] The diagnostic system for multiple patient information integrates data collected from multiple patients, and generative AI performs highly accurate diagnostic analysis. An emotion engine is used to present results according to the emotional state of medical professionals.

[0517] Specific examples

[0518] A medical professional inputs the medical records, vital signs, and diagnostic imaging data of Patients A, B, and C into a terminal. The terminal then transmits this data to a server, which then uses generative AI to perform diagnostic analysis for each patient. At the same time, an emotion engine monitors the emotional state of the medical professional and detects high levels of fatigue or stress. The server then compiles the diagnostic results into a report, adjusts the content according to the emotional state, and provides additional information and analysis results. The terminal then displays the report, allowing the medical professional to review and make a final diagnosis.

[0519] Prompt Sentence Examples

[0520] Analyze the medical records of Patient A, Patient B, and Patient C

[0521] Pharmaceutical research and design support

[0522] explanation

[0523] The pharmaceutical research and design support function uses generative AI to generate and predict the molecular structure of new drugs, and an emotion engine to analyze researchers' emotions while supporting the pharmaceutical design process.

[0524] Specific examples

[0525] Researchers input the conditions required for new drug design into a device. The device sends this information to a server, which uses generative AI to generate and predict candidate molecular structures. At the same time, the device uses an emotion engine to analyze the researcher's emotional state and detect stress or fatigue. The server stores the list of generated molecular structures in a database and presents the researcher with the most appropriate information based on their emotional state. For example, detailed annotations are added for complex molecular structures. The researcher then uses the device to search for the required information and review the displayed molecular structures.

[0526] Prompt Sentence Examples

[0527] Enter the conditions for new drug design

[0528] As described above, by applying generative AI and an emotion engine to various medical tasks, the present invention reduces the burden on medical professionals, improves efficiency in the medical field, and enables smooth communication with patients.

[0529] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0530] Clinical Documentation Support

[0531] Step 1:

[0532] The user inputs the details of the medical treatment into the terminal. Specifically, the user inputs the text "The patient complained of pain in the right shoulder, and the examination revealed no fractures."

[0533] Input: Medical treatment text

[0534] Output: Input data

[0535] Step 2:

[0536] The terminal sends the input data to the server using a secure communication protocol such as HTTPS.

[0537] Input: Text data entered

[0538] Output: The data sent.

[0539] Step 3:

[0540] The server uses a generation AI to generate a draft of a clinical document from the input data. Specifically, the generation AI (for example, GPT-3) receives a prompt sentence, "The patient complained of pain in the right shoulder, and examination revealed no fractures," and generates a clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures."

[0541] Input: Text data entered

[0542] Output: Draft clinical document

[0543] Step 4:

[0544] The device uses an emotion engine to analyze the user's emotional state. Specifically, the emotion engine analyzes facial expressions and typing speed to detect signs of stress or fatigue.

[0545] Input: User facial expression, input speed

[0546] Output: Emotion analysis results

[0547] Step 5:

[0548] The server adjusts the content of the generated clinical document based on the results of the sentiment analysis, for example, shortening the document if the user is fatigued.

[0549] Input: Sentiment analysis results, clinical document draft

[0550] Output: Reconciled clinical documentation

[0551] Step 6:

[0552] The terminal displays the adjusted clinical document to the user, who reviews it and makes corrections as necessary. Specific actions include correcting and reviewing the document.

[0553] Input: Reconciled clinical documentation

[0554] Output: Verified clinical documentation

[0555] Step 7:

[0556] When the user has finished correcting the document, he or she presses the confirm button to confirm the document.

[0557] Input: Verified clinical documentation

[0558] Output: Confirmed clinical documentation

[0559] Step 8:

[0560] The server stores the confirmed clinical documents in a database, taking appropriate measures to ensure data integrity and security.

[0561] Input: Confirmed clinical documentation

[0562] Output: Documents stored in a database

[0563] ---

[0564] Automated medical interview system

[0565] Step 1:

[0566] The patient enters their symptoms into the terminal, specifically, "I have had a headache, fever, and cough for three days."

[0567] Input: Patient symptom text

[0568] Output: Input data

[0569] Step 2:

[0570] The terminal transmits the input data to the server.

[0571] Input: Symptom text data

[0572] Output: The data sent.

[0573] Step 3:

[0574] The server uses generation AI to generate disease name candidates from the input data. For example, based on the information "headache, fever, and cough that has continued for three days," it generates candidate disease names such as "cold, flu, and COVID-19."

[0575] Input: Text data of patient symptoms

[0576] Output: List of disease candidates

[0577] Step 4:

[0578] The device uses an emotion engine to analyze the patient's emotional state, specifically detecting anxiety or tension by analyzing facial expressions and voice during input.

[0579] Input: Patient's facial expression, voice

[0580] Output: Emotion analysis results

[0581] Step 5:

[0582] The server adjusts the way it presents disease candidates based on the results of emotion analysis. For example, if anxiety is high, it changes the language to simpler, easier-to-understand terms.

[0583] Input: Sentiment analysis results, disease name candidate list

[0584] Output: Adjusted disease name candidates

[0585] Step 6:

[0586] The device displays the adjusted disease candidate list to the patient, who can review it and provide additional information if necessary.

[0587] Input: Adjusted disease name candidate list

[0588] Output: List of confirmed disease candidates

[0589] Step 7:

[0590] The device compiles the patient's responses and sends the results to medical professionals.

[0591] Input: List of confirmed disease candidates, additional information

[0592] Output: Interview results data for medical professionals

[0593] ---

[0594] Diagnostic Imaging System

[0595] Step 1:

[0596] A medical professional uploads a patient's medical image to a terminal, for example, a chest X-ray image.

[0597] Input: Medical image data

[0598] Output: Uploaded data

[0599] Step 2:

[0600] The terminal transmits the image data to the server.

[0601] Input: Medical image data

[0602] Output: The data sent.

[0603] Step 3:

[0604] The server uses image recognition AI to analyze medical images and identify abnormalities. For example, AI can analyze X-ray images and detect abnormalities.

[0605] Input: Medical image data

[0606] Output: Analysis result data

[0607] Step 4:

[0608] The device uses an emotion engine to analyze the emotional state of medical staff, specifically by analyzing their facial expressions and working speed to detect stress and fatigue.

[0609] Input: facial expression of medical staff, working speed

[0610] Output: Emotion analysis results

[0611] Step 5:

[0612] The server adjusts the report content based on the results of the emotion analysis, for example providing more detailed explanations if stress levels are high.

[0613] Input: Sentiment analysis results, analysis result data

[0614] Output: Reconciled report

[0615] Step 6:

[0616] The device displays the adjusted report to the medical professional and prompts them to confirm it. Specific actions include displaying and confirming the analysis results.

[0617] Input: Reconciled Report

[0618] Output: Confirmed report

[0619] ---

[0620] Diagnosis system based on multiple patient information

[0621] Step 1:

[0622] A medical professional enters information about multiple patients into a terminal. For example, medical records and vital data for Patient A, Patient B, and Patient C.

[0623] Input: Medical information data for each patient

[0624] Output: The input data

[0625] Step 2:

[0626] The device sends the data to the server.

[0627] Input: Medical information data for each patient

[0628] Output: The data sent.

[0629] Step 3:

[0630] The server uses generated AI to perform diagnostic analysis for each patient based on the input data.

[0631] Input: Medical information data for each patient

[0632] Output: Diagnostic analysis results

[0633] Step 4:

[0634] The device uses an emotion engine to analyze the emotional state of medical staff in real time.

[0635] Input: facial expression of medical staff, working speed

[0636] Output: Emotion analysis results

[0637] Step 5:

[0638] The server adjusts the diagnosis report based on the sentiment analysis results, for example adding a brief explanation if the healthcare worker is tired.

[0639] Input: Sentiment analysis results, diagnostic analysis results

[0640] Output: Tailored diagnostic results report

[0641] Step 6:

[0642] The device displays the adjusted report to the medical professional for confirmation and final diagnosis.

[0643] Input: Adjusted diagnostic results report

[0644] Output: Confirmed diagnostic results report

[0645] ---

[0646] Pharmaceutical research and design support

[0647] Step 1:

[0648] Researchers input the conditions necessary for new drug design, such as active ingredients and stability conditions, into the terminal.

[0649] Input: Condition data for new drug design

[0650] Output: The input data

[0651] Step 2:

[0652] The terminal transmits the condition data to the server.

[0653] Input: Condition data for new drug design

[0654] Output: The data sent.

[0655] Step 3:

[0656] The server uses a generative AI to generate candidate molecular structures that meet the conditions.

[0657] Input: Condition data for new drug design

[0658] Output: List of candidate molecular structures

[0659] Step 4:

[0660] The device uses an emotion engine to analyze the researcher's emotional state, specifically by analyzing facial expressions and typing speed to detect stress and fatigue.

[0661] Input: Researcher's facial expression, input speed

[0662] Output: Emotion analysis results

[0663] Step 5:

[0664] The server then adjusts the list of molecular structures based on the results of the sentiment analysis, for example adding detailed annotations to complex molecular structures.

[0665] Input: Sentiment analysis results, candidate list of molecular structures

[0666] Output: Adjusted molecular structure list

[0667] Step 6:

[0668] The terminal displays the adjusted molecular structure list to the researcher and prompts them to confirm it. Specific actions include displaying and confirming the molecular structure.

[0669] Input: Adjusted molecular structure list

[0670] Output: List of confirmed molecular structures

[0671] (Application example 2)

[0672] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0673] While conventional medical operation support systems have improved the efficiency of medical personnel's work, they have not been able to integrate emotion analysis or automatic patrol functions in the security field. As a result, safety inside and outside medical facilities has not been ensured sufficiently, and the burden on medical personnel remains heavy. The present invention aims to add security functions to medical operation support systems, thereby improving safety while performing medical operations efficiently.

[0674] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes [means for receiving input medical information], [means for creating a draft of a clinical document using a generation AI based on the received medical information], and [means for displaying the created clinical document and correcting and confirming it]. This makes it possible to improve the efficiency of medical work and also to increase safety within medical facilities.

[0675] "Medical information" refers to data such as a patient's medical treatment, examination results, health status, medical history, and prescription information.

[0676] "Generative AI" is an algorithm that learns patterns from large amounts of data and automates tasks such as natural language generation, image recognition, and sentiment analysis.

[0677] "Clinical documents" are documents such as medical records, reports, and diagnosis certificates that are created based on medical information.

[0678] An "emotion engine" is a technology that analyzes and recognizes human emotions from input data.

[0679] "Security measures" are systems or devices that patrol a target area and detect suspicious behavior or unusual events.

[0680] A "patrol route" is the route that a security robot travels to check for safety.

[0681] "Emotion analysis" is the process of determining a subject's emotional state from facial expressions, voice, text, etc.

[0682] "Abnormality Report" is a function that notifies the administrator of abnormal situations detected by the system.

[0683] The present invention is an integrated system that uses generative AI and emotion engines to support healthcare professionals in their work, improve communication with patients, and enhance security both inside and outside healthcare facilities. The following describes in detail the embodiments of the present invention.

[0684] Hardware and software used

[0685] Hardware

[0686] 1. Security Robot: A robot equipped with autonomous mobility and a high-resolution camera, this robot patrols inside and outside medical facilities.

[0687] 2. Camera: The security robot will be equipped with a high-resolution camera to capture footage in real time.

[0688] 3. Computer: A computer equipped with a CPU / GPU for real-time data processing. Small computers such as Raspberry Pi are used.

[0689] software

[0690] 1. OpenCV: Image processing library. Performs image processing such as face recognition and anomaly detection.

[0691] 2. Keras: A deep learning library for running emotion recognition models using generative AI.

[0692] 3. Python: A programming language. All processing logic of the present invention is written in Python.

[0693] Specific operation of the system

[0694] 1. Receiving medical information

[0695] The server receives medical information entered by a medical professional, such as the patient's medical history and health status, which is used to create a draft clinical document.

[0696] 2. Generative AI for clinical documentation

[0697] The server analyzes the received medical information and uses a generative AI model to automatically create draft clinical documents that can be reviewed and revised by medical professionals.

[0698] 3. Emotion analysis using an emotion engine

[0699] The server uses an emotion engine to analyze the emotions of medical professionals and patients, which then fine-tunes the content of the documents created and improves the quality of communication.

[0700] 4. Automatic patrols by security robots

[0701] The security robot automatically moves along a set patrol route, checking the safety of designated areas, capturing video footage in real time with a camera and performing facial recognition and emotion analysis.

[0702] 5. Anomaly Detection and Reporting

[0703] The server analyzes the data sent from the security robot in real time to detect abnormal behavior or emotional changes. If an abnormality is detected, it is immediately reported to the administrator and appropriate countermeasures are provided using generative AI.

[0704] Specific examples

[0705] For example, if a security robot encounters a person expressing negative emotions such as "anxiety" or "fear" while patrolling, it will recognize the person's face and analyze their emotion. If an abnormality is detected, it will notify the administrator in real time and send a video of the situation. Furthermore, it will automatically update its security route and patrol areas to ensure greater safety.

[0706] Prompt Sentence Examples

[0707] You may also find the following prompt examples helpful:

[0708] Use emotion recognition models to determine the emotions of people in a security area in real time, identify suspicious behavior or emotional changes, and report them to management, including the ability to automatically adjust patrol routes.

[0709] As described above, the present invention provides an integrated system that achieves both improved efficiency in medical care and enhanced security.

[0710] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0711] Step 1:

[0712] The user inputs medical information such as the patient's medical treatment details and health condition into the terminal. The terminal then sends this information to the server. The input here is text data such as the patient's symptoms and examination results, and the server receives and stores this information.

[0713] Step 2:

[0714] The server uses a generative AI model to create a draft clinical document based on the received medical information. The generated draft includes medical records and reports based on the medical information, automating tasks that would otherwise be performed manually by medical professionals.

[0715] Step 3:

[0716] The server sends the generated draft of the clinical document to the terminal, which displays it. The user checks the displayed document and makes corrections as necessary. After corrections are made, the user confirms the document, and the terminal sends the confirmed document back to the server.

[0717] Step 4:

[0718] The server stores the confirmed clinical documents in a database for future reference and analysis. The documents are stored in text or PDF files.

[0719] Step 5:

[0720] The security robot begins patrolling automatically along a set route. The robot captures real-time video with its built-in camera and transmits it to a terminal. This video is used for facial recognition and emotion analysis.

[0721] Step 6:

[0722] The server performs facial recognition on the received video data to detect the subject's face. The detected facial image is input into the emotion engine, which analyzes the subject's emotions. The input here is a facial image, and the output is an emotional state such as "joy," "fear," or "anxiety."

[0723] Step 7:

[0724] The server detects anomalies based on the analysis results. For example, if negative emotions such as anxiety or fear are detected, the server notifies the administrator in real time. The notification includes the subject's emotional state and a link to the captured video.

[0725] Step 8:

[0726] The server automatically adjusts the patrol route based on the emotion analysis results, for example, changing the route to patrol more frequently in areas where abnormalities are detected. This configuration information is sent to the security robot, which then continues patrolling along the new route.

[0727] The above are the specific processing steps of the system that realizes the application example. The input and output, data processing and calculation contents, and specific operations at each step are explained in detail.

[0728] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0729] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0730] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0731] [Second embodiment]

[0732] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0733] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0734] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0735] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0736] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0737] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0738] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0739] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0740] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0741] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0742] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0743] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0744] This invention is a system that uses generative AI to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[0745] Clinical Documentation Support

[0746] explanation

[0747] This function allows medical professionals to input medical treatment details and examination results, and the generation AI automatically creates a draft of the clinical document, assisting medical professionals in the process of reviewing, correcting, and confirming the content.

[0748] Specific examples

[0749] The user (healthcare worker) inputs the details of the patient's medical treatment. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[0750] The server receives this information and uses generative AI to generate a draft clinical document stating, "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[0751] The terminal displays the generated document, and the user checks it, makes corrections as necessary, and finalizes it as the final document.

[0752] The server stores the established clinical documents in a database.

[0753] Automated medical interview system

[0754] explanation

[0755] This function allows patients to answer questions via their device, and the generative AI then suggests disease names as candidates for initial diagnosis, enabling efficient initial diagnosis.

[0756] Specific examples

[0757] The user (patient) answers the medical interview questions displayed on the terminal. For example, they might enter, "I have had a headache, fever, and cough for three days."

[0758] The server analyzes this information and generates potential disease names such as "cold, flu, COVID-19."

[0759] The device presents these potential disease names to the patient and then passes on the necessary information to medical professionals.

[0760] Diagnostic Imaging System

[0761] explanation

[0762] This feature allows medical professionals to upload medical images (e.g., X-rays, MRIs, etc.), and image recognition AI analyzes abnormalities and generates a report.

[0763] Specific examples

[0764] The user (healthcare worker) inputs the patient's chest X-ray image into the terminal.

[0765] The server analyzes the received images and uses image recognition AI to identify abnormalities (e.g., shadows in the lungs).

[0766] The server generates and displays the analysis results as a report.

[0767] The terminal displays the generated report, which the user (healthcare professional) can check.

[0768] Diagnosis system based on multiple patient information

[0769] explanation

[0770] This feature integrates data collected from multiple patients and uses generative AI to perform highly accurate diagnostic analysis.

[0771] Specific examples

[0772] The user (healthcare professional) inputs the medical records, vital data, and diagnostic imaging data of Patients A, B, and C into the terminal.

[0773] The server integrates this data and uses generative AI to perform a diagnostic analysis for each patient.

[0774] The server compiles diagnostic results into a report, such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having a decline in kidney function."

[0775] The terminal displays the generated report, and the user (medical professional) checks it and makes a final diagnosis.

[0776] Pharmaceutical research and design support

[0777] explanation

[0778] This feature uses generative AI to generate and predict molecular structures of new drugs, supporting the drug design process.

[0779] Specific examples

[0780] The user (researcher) inputs the conditions necessary for designing a new drug into the terminal.

[0781] The server uses generative AI to generate and predict candidate molecular structures.

[0782] The server stores the generated molecular structures in a database, allowing users (researchers) to search and use them as needed.

[0783] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI to various medical tasks.

[0784] The processing flow will be explained below.

[0785] Clinical Documentation Support

[0786] Processing Steps

[0787] Step 1:

[0788] The user (healthcare worker) uses a terminal to input details of the patient's medical treatment, including symptoms, examination results, and treatment details.

[0789] Step 2:

[0790] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[0791] Step 3:

[0792] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, including the diagnosis and treatment plan.

[0793] Step 4:

[0794] The server transmits the generated draft of the clinical document to the terminal.

[0795] Step 5:

[0796] The terminal displays the received draft of the clinical document on the screen, and the user (healthcare professional) can check and correct it.

[0797] Step 6:

[0798] The user inputs the corrections and presses the confirm button.

[0799] Step 7:

[0800] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[0801] Automated medical interview system

[0802] Processing Steps

[0803] Step 1:

[0804] The terminal displays medical interview questions to the patient on the screen.

[0805] Step 2:

[0806] The user (patient) answers questions and inputs their symptoms and physical condition.

[0807] Step 3:

[0808] The terminal transmits the input answer to the server.

[0809] Step 4:

[0810] The server analyzes the received responses and compares them with a medical database to generate relevant disease name candidates.

[0811] Step 5:

[0812] The server sends the generated disease name candidates to the terminal.

[0813] Step 6:

[0814] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[0815] Diagnostic Imaging System

[0816] Processing Steps

[0817] Step 1:

[0818] The user (medical professional) inputs medical images (e.g., X-rays, MRIs, etc.) into the terminal.

[0819] Step 2:

[0820] The terminal transmits the input image data to the server.

[0821] Step 3:

[0822] The server analyzes the received image data and identifies any abnormalities using image recognition AI.

[0823] Step 4:

[0824] The server generates a report based on the analysis results.

[0825] Step 5:

[0826] The server sends the generated report to the terminal.

[0827] Step 6:

[0828] The terminal displays the report, which the user (healthcare professional) can review.

[0829] Diagnosis system based on multiple patient information

[0830] Processing Steps

[0831] Step 1:

[0832] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[0833] Step 2:

[0834] The terminal transmits the input data to the server.

[0835] Step 3:

[0836] The server integrates the received data and performs diagnostic analysis using generative AI.

[0837] Step 4:

[0838] The server generates a diagnostic report based on the analysis results.

[0839] Step 5:

[0840] The server transmits the generated diagnostic report to the terminal.

[0841] Step 6:

[0842] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[0843] Pharmaceutical research and design support

[0844] Processing Steps

[0845] Step 1:

[0846] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[0847] Step 2:

[0848] The terminal transmits the input conditions to the server.

[0849] Step 3:

[0850] The server uses generative AI to generate and predict candidate molecular structures.

[0851] Step 4:

[0852] The server stores the generated molecular structures in a database.

[0853] Step 5:

[0854] The user (researcher) searches for the information they need and displays it on their terminal.

[0855] Example 1

[0856] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0857] The purpose of this invention is to reduce the wide-ranging workload of medical professionals and improve efficiency in the medical field. Conventional systems require a lot of manual work, such as document creation, medical interviews, image diagnosis, diagnostic analysis of multiple patients, and pharmaceutical research, which is time-consuming and labor-intensive, and also has a high risk of human error. This has resulted in an excessive workload for medical professionals, making it difficult to provide prompt and accurate medical care.

[0858] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0859] In this invention, the server includes: [means for receiving input medical data;] [means for creating a draft of a clinical document using a generative AI model based on the received medical data;] [means for displaying the created clinical document and correcting and confirming it; and] [means for saving the confirmed clinical document in a database. This automates document creation work in medical settings, reducing time and effort and human error.

[0860] "Medical data" refers to a wide range of data related to medical care, such as patient medical information, vital signs, diagnostic imaging data, and medical records.

[0861] A "generative AI model" refers to an artificial intelligence algorithm that uses technologies such as natural language processing and image recognition to generate output tailored to a specific purpose from input data.

[0862] "Clinical documents" refer to important documents in the medical field that contain information such as medical treatment details and examination results.

[0863] "Medical interview items" refer to questions presented to understand the patient's symptoms, medical history, and current health condition.

[0864] "Candidate disease names" refers to a list of possible disease names generated based on medical interview responses and other medical data.

[0865] "Medical images" refers to medical image data such as X-rays, MRIs, and CT scans.

[0866] "Image analysis AI" refers to artificial intelligence algorithms that analyze medical images and identify or diagnose abnormalities.

[0867] "Patient data" refers to all medical information related to individual patients, including medical information, vital signs, and diagnostic imaging data.

[0868] "Diagnosis result" refers to a diagnostic conclusion generated through analysis based on medical data and patient data.

[0869] "Drug design criteria" refers to the scientific and clinical requirements and specifications for designing new drugs.

[0870] "Molecular structure" refers to the internal structure of a drug or other compound, represented as a chemical formula or 3D model.

[0871] A "report" is a document that documents the results of analysis or diagnosis and is compiled in a format that can be reviewed and used by medical professionals.

[0872] "Database" refers to a computer system for systematically storing and managing information such as medical data, clinical documents, and molecular structures.

[0873] This invention is a system that uses generative AI models to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[0874] Hardware and software used

[0875] The system's main hardware consists of a server and a terminal. The server receives and analyzes data, runs the generative AI model, and stores the data. The terminal provides an interface with the user, allowing data input and display of results.

[0876] Specific software includes:

[0877] Generative AI models: Natural language processing models such as OpenAI's GPT-4

[0878] Image analysis AI: Image recognition services such as Google Cloud Vision and AWS Rekognition

[0879] Database: A data management system such as Firebase or PostgreSQL

[0880] Clinical Documentation Support

[0881] The user (healthcare professional) inputs the details of the medical treatment from the terminal. For example, they input data such as "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal then sends the input details of the medical treatment to the server. The server inputs the received data as prompts into the generative AI model, generating a draft clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal displays the generated document, and the user can review it, make corrections as necessary, and finalize it as the final document. The server then saves the finalized clinical document in a database.

[0882] Example prompt: "Please use this clinical content to draft a clinical document."

[0883] Automated medical interview system

[0884] The user (patient) answers the medical interview questions displayed on the device. For example, they might enter, "I have had a headache, fever, and cough for three days." The device then sends the entered medical interview data to the server. The server then inputs the received data as prompts into the generative AI model, generating disease name candidates such as "cold, flu, COVID-19." The device then presents these disease name candidates to the patient and then conveys the necessary information to medical professionals.

[0885] Example prompt: "Please list possible diseases based on this interview data."

[0886] Diagnostic Imaging System

[0887] The user (healthcare professional) uploads an image file (e.g., a chest X-ray image) to the device. The device sends the image file to the server. The server inputs the received image into the image analysis AI and identifies abnormal areas (e.g., shadows in the lungs). The image analysis AI generates a report of the analysis results. The server sends the generated report to the device, which displays the report and the user (healthcare professional) confirms it.

[0888] Example prompt: "Analyze this image to identify anomalies and generate a report."

[0889] Diagnosis system based on multiple patient information

[0890] The user (healthcare professional) inputs multiple patient information (e.g., medical record information, vital signs, diagnostic imaging data) into the terminal. The terminal sends all input data to the server. The server integrates the received data and inputs it as prompts into the generative AI model, performing diagnostic analysis for each patient. For example, it generates a diagnosis result such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having decreased renal function." The server then sends the generated diagnosis result as a report to the terminal, which displays the report and the user confirms it.

[0891] Example prompt: "Generate a diagnosis based on these patient data."

[0892] Pharmaceutical research and design support

[0893] The user (researcher) inputs the conditions necessary for designing a new drug into a terminal. The terminal then sends the input design conditions to the server. The server inputs the received conditions as prompts into the generative AI model, generating and predicting candidate molecular structures. The server stores the generated molecular structures in a database, allowing users to search and use them as needed.

[0894] Example prompt: "Generate molecular structure candidates based on these conditions."

[0895] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI models to various medical tasks.

[0896] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0897] Clinical Documentation Support

[0898] Step 1: The user inputs the details of the medical treatment from the terminal.

[0899] Specifically, the user uses a keyboard or voice input to input, "The patient complains of pain in the right shoulder, and examination reveals no fractures."

[0900] Step 2: The terminal sends the entered medical details to the server.

[0901] The input is the medical data, and the output is the transmitted data. HTTPS is used as the communication protocol.

[0902] Step 3: The server inputs the received data into the generative AI model as prompts.

[0903] The input is medical content data, and the output is a prompt: "Please use this medical content to draft a clinical document." The generative AI model analyzes the received data and generates an appropriate draft clinical document.

[0904] Step 4: The generative AI model returns the generated draft clinical document to the server.

[0905] The input is the analysis result data, and the output is the generated draft clinical document.

[0906] Step 5: The server sends the generated draft to the device.

[0907] The input is the draft data, and the output is the submitted draft data.

[0908] Step 6: The terminal displays the generated document, which the user can review and modify if necessary.

[0909] As a specific operation, the user uses a text editor to edit a document on the screen of the terminal.

[0910] Step 7: The user finalizes the document.

[0911] The input is the corrected document and the output is the finalized document.

[0912] Step 8: The terminal sends the confirmed document to the server.

[0913] The input is the finalized document data, and the output is the transmitted data.

[0914] Step 9: The server saves the finalized document to the database.

[0915] The input is the finalized document data, and the output is the data stored in the database.

[0916] Automated medical interview system

[0917] Step 1: The user (patient) answers the medical interview questions displayed on the terminal.

[0918] Specifically, in response to the question, enter "I have had a headache, fever, and cough for three days" in the text box.

[0919] Step 2: The terminal transmits the entered medical interview data to the server.

[0920] The input is the medical interview response data, and the output is the transmitted data.

[0921] Step 3: The server inputs the received data into the generative AI model as prompts.

[0922] The input is the patient's interview response data, and the output is a prompt statement: "Please list possible disease names based on this interview data."

[0923] Step 4: The generative AI model generates disease name candidates based on the received data.

[0924] The input is medical interview response data, and the output is disease name candidate data.

[0925] Step 5: The server sends the generated disease name candidates to the terminal.

[0926] The input is the disease name candidate data, and the output is the transmitted data.

[0927] Step 6: The terminal presents the patient with possible disease names and then conveys the necessary information to the medical professional.

[0928] Specifically, the system displays possible disease names on the screen, such as "cold, influenza, COVID-19," and the user confirms them.

[0929] Diagnostic Imaging System

[0930] Step 1: The user (healthcare professional) uploads an image file to the device.

[0931] As a specific operation, a file selection dialog is used to select a chest X-ray image.

[0932] Step 2: The device sends the image file to the server.

[0933] The input is the image file and the output is the transmitted data.

[0934] Step 3: The server inputs the received image into the image analysis AI.

[0935] The input is an image file, and the output is a prompt: "Analyze this image to identify any abnormalities and generate a report."

[0936] Step 4: Image analysis AI analyzes the pixel data of the image and identifies abnormalities.

[0937] The input is an image file, and the output is the analysis result data.

[0938] Step 5: The image analysis AI generates a report of the analysis results.

[0939] The input is analysis result data, and the output is report data.

[0940] Step 6: The server sends the generated report to the terminal.

[0941] The input is the report data and the output is the transmitted data.

[0942] Step 7: The terminal displays the report and the user (healthcare professional) confirms it.

[0943] Specifically, the generated report is displayed on the screen and checked by a medical professional.

[0944] Diagnosis system based on multiple patient information

[0945] Step 1: A user (healthcare worker) inputs multiple patient information into a terminal.

[0946] Specifically, the patient enters medical record information and vital data into a form.

[0947] Step 2: The device sends all entered data to the server.

[0948] The input is a plurality of patient information data, and the output is the transmitted data.

[0949] Step 3: The server consolidates the received data.

[0950] The input is multiple patient information data, and the output is the integrated data.

[0951] Step 4: The server inputs the integrated data as prompts into the generative AI model.

[0952] The input is the integrated data, and the output is the prompt statement "Please generate a diagnosis based on these patient data."

[0953] Step 5: The generative AI model generates a diagnosis based on the integrated data.

[0954] The input is the integrated data, and the output is the diagnostic result data.

[0955] Step 6: The server sends the generated diagnostic results to the terminal as a report.

[0956] The input is the diagnostic result data, and the output is the transmitted data.

[0957] Step 7: The terminal displays the report, which the user confirms and makes a final diagnosis.

[0958] Specifically, the report is displayed and a medical professional makes a final diagnosis.

[0959] Pharmaceutical research and design support

[0960] Step 1: The user (researcher) inputs the design conditions for a new drug into the terminal.

[0961] As a specific operation, the design conditions are input into the text box.

[0962] Step 2: The terminal transmits the input design conditions to the server.

[0963] The input is the design condition data, and the output is the transmitted data.

[0964] Step 3: The server inputs the received conditions into the generative AI model as prompts.

[0965] The input is design condition data, and the output is a prompt statement: "Generate molecular structure candidates based on these conditions."

[0966] Step 4: The generative AI model generates candidate molecular structures based on the conditions.

[0967] The input is design condition data, and the output is molecular structure candidate data.

[0968] Step 5: The server stores the generated molecular structure in a database.

[0969] The input is molecular structure candidate data, and the output is the saved data.

[0970] Step 6: The user searches for and uses molecular structures from the terminal as needed.

[0971] Specifically, the molecular structure search interface is used to search and evaluate stored molecular structures.

[0972] The specific processing steps for each function have been described above.

[0973] (Application example 1)

[0974] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0975] Conventional medical support systems require medical personnel to be present at all times, making it difficult to respond quickly in emergencies. In particular, emergency medical support systems using mobile means such as autonomous vehicles have difficulty providing first aid and diagnosis when medical personnel are not present on board, making it impossible to provide appropriate medical support.

[0976] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0977] In this invention, the server includes a means for receiving input medical information, a means for creating a draft clinical document using a generative artificial intelligence based on the received medical information, a means for displaying, correcting, and confirming the created clinical document, a means for inputting patient symptoms in an emergency and starting processing, and a means for using the generative artificial intelligence to create instructions for first aid in real time based on the input symptoms. This enables the autonomous vehicle to provide prompt and appropriate first aid and safely transport the patient even when medical personnel are not on-site.

[0978] "Medical information" refers to all data related to patients, such as medical treatment details, symptoms, examination results, and medical images.

[0979] "Generative AI" is an AI technology that automatically generates natural language sentences and reports from given data.

[0980] "Clinical documents" are official documents for medical professionals that contain information such as examination results and treatment plans.

[0981] "Emergency" refers to situations where medical personnel are not available or where immediate first aid is required.

[0982] "Real-time first aid instructions" are specific instructions for performing emergency medical procedures that are instantly generated using signal processing and generative AI.

[0983] A "medical history" is a series of questions asked to understand a patient's symptoms and medical history.

[0984] A "medical database" is a system that accumulates and manages medical information and case data.

[0985] "Medical images" are image data obtained using medical equipment such as X-rays and MRIs.

[0986] "Image recognition artificial intelligence" is an artificial intelligence technology used to analyze medical images and identify abnormalities.

[0987] "Instructions for first aid based on diagnostic results" are specific treatment procedures and treatment details created by the generative artificial intelligence based on the diagnostic results.

[0988]

[0989] This invention provides a concrete implementation of a generative AI-based emergency medical support system to be installed in an autonomous vehicle. This system enables rapid first aid in emergencies and is primarily composed of the following main components:

[0990] 1. Server:

[0991] The server is the central system that performs various processes using generative AI models. The server is responsible for the following processes:

[0992] Receiving and generating medical information. Drafting clinical documentation using AI.

[0993] Generate emergency first aid instructions based on patient symptom information.

[0994] An initial diagnosis is made based on the results of the interview, after comparing them with a medical database.

[0995] The server software includes generative AI models using TensorFlow and Keras, and a database management system.

[0996] 2. Terminal:

[0997] The terminal is a device such as a smartphone or smart glasses that provides an interface for patients and medical professionals to input and view information. The terminal has the following functions:

[0998] The user (patient or healthcare professional) enters symptoms and medical questionnaire answers.

[0999] Displays generated documents and diagnostic results from the server.

[1000] Displays emergency procedure instructions as needed.

[1001] For example, a patient may type into their smartphone, "I have severe chest pain and difficulty breathing." The device can also automatically input information using voice commands.

[1002] 3. User:

[1003] The user is a patient or a medical professional who uses the system. The user performs the following operations:

[1004] Enter your symptoms in case of an emergency.

[1005] Respond based on emergency instructions from the server.

[1006] Review and correct clinical documentation as needed.

[1007] The user can utilize the system through an intuitive interface, maximizing the effectiveness of the invention.

[1008] The following are examples of prompt sentences:

[1009] Example prompt sentence:

[1010] "The patient is experiencing chest pain and difficulty breathing. Urgent treatment is needed. What are the possible diagnoses? What first aid would be needed?"

[1011] Specific examples of hardware include self-driving vehicles, smartphones, smart glasses, and IoT medical devices (e.g., robotic arms, first aid kits).

[1012] Specific examples of software include generative AI models (TensorFlow / Keras models), real-time image analysis software, and control software for IoT devices.

[1013] In this way, the present invention realizes a system that can provide prompt and appropriate first aid in an autonomous vehicle even when medical personnel are not present.

[1014] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1015] Step 1:

[1016] The user inputs symptoms into a smartphone or smart glasses. For example, if the user inputs "severe chest pain and difficulty breathing," the device receives this input. The input data includes the patient's symptoms and condition.

[1017] Step 2:

[1018] The device sends the symptom information it receives to the server. At this time, the symptom information is transferred to the server as text data. The server receives this text data and inputs it into the generative AI model.

[1019] Step 3:

[1020] The server uses generative AI to create a draft clinical document from the received symptom information. The generative AI model generates an initial diagnosis based on the input data. Here, the prompt used is "The patient complains of severe chest pain and difficulty breathing. What are some possible diagnosis candidates?" As a result, a candidate diagnosis and first aid instructions are generated.

[1021] Step 4:

[1022] The server sends the generated clinical document and candidate diagnoses to the terminal, which displays the generated document for the user to review. The user can then review the displayed information and make corrections as necessary.

[1023] Step 5:

[1024] The server stores the clinical document in a database after the user has confirmed and corrected it. The confirmed document is stored in the medical database for future reference and use.

[1025] Step 6:

[1026] The server generates emergency first aid instructions based on the initial diagnosis. Using generative artificial intelligence, it generates instructions in real time, including specific first aid procedures. For example, it creates "operation instructions for the robot arm to begin cardiac massage."

[1027] Step 7:

[1028] The server sends the generated first aid instructions to the IoT device. The terminal receives the instructions and sends a signal to control a nearby medical device (e.g., a robotic arm). Emergency first aid is then performed based on the specific control procedure.

[1029] Step 8:

[1030] The terminal displays the final instructions sent from the server and the manufactured treatment details to the user, who can then continue responding to the emergency based on this information.

[1031] In this way, the system can provide fast and accurate real-time first aid in emergencies.

[1032] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1033] This invention is an integrated system that uses generative AI and an emotion engine to support medical professionals in their work and improve communication with patients. This system combines clinical documentation support, an automated medical interview system, an image diagnosis system, a diagnosis system using multiple patient information, and pharmaceutical research and design support functions with an emotion engine.

[1034] Clinical Documentation Support

[1035] explanation

[1036] This function allows medical professionals to input medical treatment details and examination results, and the generative AI automatically creates a draft of the clinical document, uses an emotion engine to recognize the medical professional's emotions, and assists in the process of creating an appropriate document based on the input content.

[1037] Specific examples

[1038] The user (healthcare worker) inputs the details of the patient's medical treatment into the terminal, for example, "The patient complains of pain in the right shoulder, and the examination reveals no fractures."

[1039] The device sends this information to a server, which then uses a generative AI to generate a draft clinical document stating, "The patient's main complaint was pain in the right shoulder, and examination revealed no fractures."

[1040] Additionally, the device uses an emotion engine to analyze the user's emotions as they type and detect signs of stress or fatigue.

[1041] The server adjusts the relevance of document content depending on the emotional state, for example, producing quicker and more concise documents.

[1042] The terminal displays the generated document, and the user checks and corrects it, and then presses the confirm button to confirm it.

[1043] The server stores the established clinical documents in a database.

[1044] Automated medical interview system

[1045] explanation

[1046] This function allows patients to answer questions via their device, and the generative AI then suggests possible disease names for initial diagnosis, using an emotion engine to recognize the patient's emotions and adjust the interview process accordingly.

[1047] Specific examples

[1048] The patient enters into the terminal, "I have had a headache, fever, and cough for three days."

[1049] The device sends this information to a server, which then uses generative AI to generate potential disease names such as "cold, flu, COVID-19."

[1050] At the same time, the device uses an emotion engine to check the patient's emotions and detect, for example, anxiety or tension.

[1051] The server adjusts the display of the medical questionnaire based on the results of the emotion analysis, changing the explanation of the possible disease name to easier-to-understand language and asking additional questions.

[1052] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[1053] Diagnostic Imaging System

[1054] explanation

[1055] This function allows medical professionals to upload medical images and uses image recognition AI and an emotion engine to analyze abnormalities, supporting diagnosis while reducing stress.

[1056] Specific examples

[1057] A medical professional inputs the patient's chest X-ray image into a terminal.

[1058] The device sends image data to a server, which then uses image recognition AI to identify abnormalities.

[1059] At the same time, the device uses an emotion engine to check the current emotional state of medical workers and detect high levels of stress or fatigue.

[1060] The server generates a report based on the analysis results and tailors the report content depending on the emotional state, for example providing additional analytical details to help reduce stress.

[1061] The terminal displays the report, which is then reviewed by medical personnel.

[1062] Diagnosis system based on multiple patient information

[1063] explanation

[1064] This function integrates data collected from multiple patients, uses generative AI and an emotion engine to perform highly accurate diagnostic analysis, and presents results based on the emotional state of medical professionals.

[1065] Specific examples

[1066] Medical staff enter the medical records, vital signs, and diagnostic imaging data of patients A, B, and C into the terminal.

[1067] The device sends this data to a server, which uses generative AI to perform a diagnostic analysis for each patient.

[1068] The emotion engine monitors the emotional state of healthcare professionals during input and detects high levels of fatigue and stress.

[1069] The server compiles the diagnostic results into a report, adapts the content depending on the emotional state, and provides additional information and analysis results to, for example, reduce the burden on medical professionals.

[1070] The terminal displays the report, which is then reviewed and a final diagnosis is made by a medical professional.

[1071] Pharmaceutical research and design support

[1072] explanation

[1073] This feature uses generative AI to generate and predict molecular structures of new drugs, and an emotion engine to recognize researchers' emotions while supporting the drug design process.

[1074] Specific examples

[1075] Researchers input the conditions necessary for new drug design into the terminal.

[1076] The device sends this information to a server, which then uses generative AI to generate and predict candidate molecular structures.

[1077] At the same time, the device uses an emotion engine to analyze the researcher's emotional state and detect stress and fatigue.

[1078] The server stores the generated list of molecular structures in a database and presents the researcher with the most appropriate information based on their emotional state, for example by adding detailed annotations for complex molecular structures.

[1079] Researchers use the terminal to search for the necessary information and check the displayed molecular structure.

[1080] As described above, by applying generative AI and an emotion engine to various medical tasks, the present invention reduces the burden on medical professionals, improves efficiency in the medical field, and enables smooth communication with patients.

[1081] The processing flow will be explained below.

[1082] Clinical Documentation Support

[1083] Processing Steps

[1084] Step 1:

[1085] The user (healthcare worker) inputs the details of the patient's medical treatment into the terminal. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[1086] Step 2:

[1087] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[1088] Step 3:

[1089] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, such as "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[1090] Step 4:

[1091] The device uses an emotion engine to analyze the user's emotions, for example, detecting signs of stress or fatigue through facial recognition and voice tone analysis.

[1092] Step 5:

[1093] The server adjusts document content to take into account the user's emotional state, for example by generating a more concise document if the user is feeling stressed.

[1094] Step 6:

[1095] The server transmits the generated document to the terminal.

[1096] Step 7:

[1097] The terminal displays the generated clinical document on the screen, allowing the user to check and correct it.

[1098] Step 8:

[1099] After completing the corrections, the user presses the confirm button.

[1100] Step 9:

[1101] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[1102] Automated medical interview system

[1103] Processing Steps

[1104] Step 1:

[1105] The terminal displays medical interview questions to the patient on the screen.

[1106] Step 2:

[1107] The user (patient) answers questions and inputs their symptoms and physical condition. For example, they might input "I have had a headache, fever, and cough for three days."

[1108] Step 3:

[1109] The terminal receives the response and transmits the data to the server.

[1110] Step 4:

[1111] The server analyzes the received response and compares it with a medical database to generate relevant disease name candidates, such as "cold, flu, COVID-19."

[1112] Step 5:

[1113] The device uses an emotion engine to analyze the patient's emotions and detect feelings such as anxiety and tension.

[1114] Step 6:

[1115] The server takes into account the patient's emotional state and adjusts the content of the interview, for example by making the explanation of potential illnesses more concise and easy to understand if anxiety is detected.

[1116] Step 7:

[1117] The server sends the generated disease name candidates to the terminal and displays them to the patient.

[1118] Step 8:

[1119] The terminal displays possible disease names and then sends the necessary information to medical professionals.

[1120] Diagnostic Imaging System

[1121] Processing Steps

[1122] Step 1:

[1123] A user (medical worker) inputs a medical image into the terminal, for example, uploading a chest X-ray image.

[1124] Step 2:

[1125] The terminal receives the image data and transmits the data to the server.

[1126] Step 3:

[1127] The server analyzes the received image data and uses image recognition AI to identify abnormalities, such as shadows in the lungs.

[1128] Step 4:

[1129] The device uses an emotion engine to analyze the emotions of medical staff, for example, to detect fatigue and stress.

[1130] Step 5:

[1131] The server generates a report of the analysis results, taking into account the emotional state, for example, if fatigue is detected, it will briefly summarize the analysis results.

[1132] Step 6:

[1133] The server sends the generated report to the terminal.

[1134] Step 7:

[1135] The terminal displays the report, which the user (healthcare professional) can review.

[1136] Diagnosis system based on multiple patient information

[1137] Processing Steps

[1138] Step 1:

[1139] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[1140] Step 2:

[1141] The terminal transmits the input data to the server.

[1142] Step 3:

[1143] The server integrates the received data and performs diagnostic analysis using generative AI, generating diagnostic results such as, for example, that Patient A is at high risk of myocardial infarction, Patient B is suspected of having pneumonia, and Patient C is suspected of having declining kidney function.

[1144] Step 4:

[1145] The device uses an emotion engine to analyze the emotions of medical workers, for example, to detect stress or fatigue.

[1146] Step 5:

[1147] The server generates a report of the analysis results, taking into account the emotional state, for example, if stress is detected, it will summarize the diagnosis in a concise and easy-to-understand manner.

[1148] Step 6:

[1149] The server transmits the generated diagnostic report to the terminal.

[1150] Step 7:

[1151] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[1152] Pharmaceutical research and design support

[1153] Processing Steps

[1154] Step 1:

[1155] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[1156] Step 2:

[1157] The terminal transmits the input conditions to the server.

[1158] Step 3:

[1159] The server uses generative AI to generate and predict candidate molecular structures.

[1160] Step 4:

[1161] The device uses an emotion engine to analyze the researcher's emotions, for example, to detect stress or fatigue.

[1162] Step 5:

[1163] The server takes emotional state into account to tailor the molecular structure generation results, for example by adding detailed annotations.

[1164] Step 6:

[1165] The server stores the generated molecular structures in a database and makes them available for user retrieval.

[1166] Step 7:

[1167] The terminal displays the search results, which the user (researcher) can confirm.

[1168] Example 2

[1169] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1170] In the current medical system, medical professionals are burdened with a heavy workload and are unable to properly manage their emotions and stress, resulting in problems with smooth communication with patients. Furthermore, emotional adjustments are not made during interviews or diagnostic imaging, which can lead to reduced diagnostic accuracy and work efficiency. It is necessary to solve these problems and provide a better environment for both medical professionals and patients.

[1171] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: [means for receiving input medical information]; [means for creating a draft of a clinical document using a generation AI based on the received medical information]; [means for analyzing the user's emotions using an emotion engine]; [means for adjusting the clinical document content based on the emotion analysis results]; [means for displaying the created clinical document, correcting and confirming it]; and [means for saving the confirmed clinical document in a database]. This enables document creation that takes into account the emotional state of medical professionals, thereby realizing effective communication.

[1172] The server also includes means for presenting medical interview questions to the patient and receiving the patient's responses, means for comparing the received responses with a medical database and generating relevant disease name candidates, means for analyzing the patient's emotions using an emotion engine, means for adjusting the method for presenting the disease name candidates based on the emotion analysis results, and means for presenting the generated disease name candidates to a medical professional. This enables medical interviews that take the patient's emotions into consideration, resulting in more accurate initial diagnoses.

[1173] Furthermore, the server includes: [means for inputting medical images and analyzing abnormalities using image recognition AI; [means for analyzing the emotions of medical professionals using an emotion engine;] [means for adjusting the report content based on the emotion analysis results; and [means for generating and displaying the analysis results as a report.] This enables image diagnosis that takes into account the emotional state of medical professionals, thereby improving diagnostic accuracy.

[1174] "Inputted medical information" refers to medical-related information such as medical treatment details, symptoms, and vital data that medical professionals and patients provide to the system.

[1175] "Generative AI" is a system that uses artificial intelligence technology to automatically generate text, images, etc., and specifically refers to a model that uses natural language processing technology.

[1176] "Clinical documents" are documents in which medical professionals record the details of medical treatment and examinations, including the patient's medical history, symptoms, and examination results.

[1177] An "emotion engine" is a technology that analyzes the emotional state of a user or patient and adjusts the system's operation and output based on that emotion.

[1178] The "emotion analysis result" is the result of the emotion analyzed by the emotion engine, and is data indicating the state of stress, fatigue, anxiety, etc. of the user or patient.

[1179] "Medical history questions" are a set of questions presented by the system to understand the patient's symptoms and medical history.

[1180] "Responses" are responses given by patients to medical interview questions, and include information about symptoms and physical conditions.

[1181] A "medical database" is a database that stores medical information, diagnosis results, disease name information, etc., and is used as reference information for medical interviews and diagnoses.

[1182] "Candidate disease names" is a list of possible disease names generated by the system based on the patient's answers and data received.

[1183] "Medical images" are medical image data taken for diagnosis or treatment, such as X-ray images, CT scan images, and MRI images.

[1184] "Image recognition AI" is an artificial intelligence technology that automatically analyzes medical images and identifies abnormalities and features.

[1185] A "report" is a document of diagnostic and analytical results generated by the system, intended to provide information to medical professionals.

[1186] A "database" is a system for efficiently storing, searching, and managing large amounts of data, and in this case is used to store clinical documents and medical information.

[1187] "Display" refers to the act of showing the data or documents generated by the system to the user or patient, and means outputting them on a screen.

[1188] "Modification" refers to the act of a user making changes or corrections to a generated document or data.

[1189] "Confirming" refers to the act of the user finally confirming the corrections and saving them as official data.

[1190] This invention is an integrated system that uses generative AI and an emotion engine to support medical professionals in their work and improve communication with patients. This system combines the emotion engine with clinical documentation support, an automated medical interview system, an image diagnosis system, a diagnosis system using multiple patient information, and pharmaceutical research and design support functions.

[1191] Clinical Documentation Support

[1192] explanation

[1193] With this function, the user (medical professional) inputs the details of medical treatment and examination results into the device, and the generative AI automatically creates a draft of the clinical document based on that information.The emotion engine analyzes the user's emotional state and adjusts the appropriateness of the document according to the input content.

[1194] Specific examples

[1195] The user enters "The patient complains of pain in the right shoulder, and examination revealed no fractures" into the terminal. The terminal sends this information to the server, which uses a generative AI to generate a draft clinical document stating, "The patient complains of pain in the right shoulder, and examination revealed no fractures." At the same time, the terminal uses an emotion engine to analyze the user's emotions as they type and detect signs of stress or fatigue. The server adjusts the content of the document according to the user's emotional state, creating a concise document. The terminal displays the generated document, and the user can review and edit it, then press the confirm button to confirm. The server then saves the confirmed clinical document in its database.

[1196] Prompt Sentence Examples

[1197] The patient complained of pain in the right shoulder, and examination revealed no fractures.

[1198] Automated medical interview system

[1199] explanation

[1200] In the automated medical interview system, patients answer questions via a device, and the generative AI then suggests possible initial diagnosis names. It also uses an emotion engine to recognize the patient's emotions and adjust the interview process accordingly.

[1201] Specific examples

[1202] The patient enters into the device, "I have had a headache, fever, and cough for three days." The device sends this information to the server, which uses generative AI to generate disease name candidates such as "cold, flu, and COVID-19." At the same time, the device uses an emotion engine to check the patient's emotional state and detect anxiety or tension. Based on the emotion analysis results, the server adjusts the display of the medical interview content, changing the explanation of the disease name candidates to easier-to-understand language or asking additional questions. The device displays the disease name candidates to the patient and then conveys the necessary information to medical professionals.

[1203] Prompt Sentence Examples

[1204] Headache, fever, and cough for 3 days

[1205] Diagnostic Imaging System

[1206] explanation

[1207] In the image diagnosis system, when medical professionals upload medical images, the image recognition AI analyzes abnormalities. An emotion engine is also used to analyze the emotional state of the medical professionals and support the diagnosis.

[1208] Specific examples

[1209] A medical professional inputs a patient's chest X-ray image into a terminal. The terminal then sends the image data to a server, which uses image recognition AI to identify abnormalities. At the same time, the terminal uses an emotion engine to analyze the medical professional's emotional state and detect high levels of stress or fatigue. The server generates a report based on the analysis results and adjusts the report content according to the medical professional's emotional state. The terminal then displays the generated report for the medical professional to review.

[1210] Prompt Sentence Examples

[1211] Analysis of chest X-ray images

[1212] Diagnosis system based on multiple patient information

[1213] explanation

[1214] The diagnostic system for multiple patient information integrates data collected from multiple patients, and generative AI performs highly accurate diagnostic analysis. An emotion engine is used to present results according to the emotional state of medical professionals.

[1215] Specific examples

[1216] A medical professional inputs the medical records, vital signs, and diagnostic imaging data of Patients A, B, and C into a terminal. The terminal then transmits this data to a server, which then uses generative AI to perform diagnostic analysis for each patient. At the same time, an emotion engine monitors the emotional state of the medical professional and detects high levels of fatigue or stress. The server then compiles the diagnostic results into a report, adjusts the content according to the emotional state, and provides additional information and analysis results. The terminal then displays the report, allowing the medical professional to review and make a final diagnosis.

[1217] Prompt Sentence Examples

[1218] Analyze the medical records of Patient A, Patient B, and Patient C

[1219] Pharmaceutical research and design support

[1220] explanation

[1221] The pharmaceutical research and design support function uses generative AI to generate and predict the molecular structure of new drugs, and an emotion engine to analyze researchers' emotions while supporting the pharmaceutical design process.

[1222] Specific examples

[1223] Researchers input the conditions required for new drug design into a device. The device sends this information to a server, which uses generative AI to generate and predict candidate molecular structures. At the same time, the device uses an emotion engine to analyze the researcher's emotional state and detect stress or fatigue. The server stores the list of generated molecular structures in a database and presents the researcher with the most appropriate information based on their emotional state. For example, detailed annotations are added for complex molecular structures. The researcher then uses the device to search for the required information and review the displayed molecular structures.

[1224] Prompt Sentence Examples

[1225] Enter the conditions for new drug design

[1226] As described above, by applying generative AI and an emotion engine to various medical tasks, the present invention reduces the burden on medical professionals, improves efficiency in the medical field, and enables smooth communication with patients.

[1227] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1228] Clinical Documentation Support

[1229] Step 1:

[1230] The user inputs the details of the medical treatment into the terminal. Specifically, the user inputs the text "The patient complained of pain in the right shoulder, and the examination revealed no fractures."

[1231] Input: Medical treatment text

[1232] Output: Input data

[1233] Step 2:

[1234] The terminal sends the input data to the server using a secure communication protocol such as HTTPS.

[1235] Input: Text data entered

[1236] Output: The data sent.

[1237] Step 3:

[1238] The server uses a generation AI to generate a draft of a clinical document from the input data. Specifically, the generation AI (for example, GPT-3) receives a prompt sentence, "The patient complained of pain in the right shoulder, and examination revealed no fractures," and generates a clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures."

[1239] Input: Text data entered

[1240] Output: Draft clinical document

[1241] Step 4:

[1242] The device uses an emotion engine to analyze the user's emotional state. Specifically, the emotion engine analyzes facial expressions and typing speed to detect signs of stress or fatigue.

[1243] Input: User facial expression, input speed

[1244] Output: Emotion analysis results

[1245] Step 5:

[1246] The server adjusts the content of the generated clinical document based on the results of the sentiment analysis, for example, shortening the document if the user is fatigued.

[1247] Input: Sentiment analysis results, clinical document draft

[1248] Output: Reconciled clinical documentation

[1249] Step 6:

[1250] The terminal displays the adjusted clinical document to the user, who reviews it and makes corrections as necessary. Specific actions include correcting and reviewing the document.

[1251] Input: Reconciled clinical documentation

[1252] Output: Verified clinical documentation

[1253] Step 7:

[1254] When the user has finished correcting the document, he or she presses the confirm button to confirm the document.

[1255] Input: Verified clinical documentation

[1256] Output: Confirmed clinical documentation

[1257] Step 8:

[1258] The server stores the confirmed clinical documents in a database, taking appropriate measures to ensure data integrity and security.

[1259] Input: Confirmed clinical documentation

[1260] Output: Documents stored in a database

[1261] ---

[1262] Automated medical interview system

[1263] Step 1:

[1264] The patient enters their symptoms into the terminal, specifically, "I have had a headache, fever, and cough for three days."

[1265] Input: Patient symptom text

[1266] Output: Input data

[1267] Step 2:

[1268] The terminal transmits the input data to the server.

[1269] Input: Symptom text data

[1270] Output: The data sent.

[1271] Step 3:

[1272] The server uses generation AI to generate disease name candidates from the input data. For example, based on the information "headache, fever, and cough that has continued for three days," it generates candidate disease names such as "cold, flu, and COVID-19."

[1273] Input: Text data of patient symptoms

[1274] Output: List of disease candidates

[1275] Step 4:

[1276] The device uses an emotion engine to analyze the patient's emotional state, specifically detecting anxiety or tension by analyzing facial expressions and voice during input.

[1277] Input: Patient's facial expression, voice

[1278] Output: Emotion analysis results

[1279] Step 5:

[1280] The server adjusts the way it presents disease candidates based on the results of emotion analysis. For example, if anxiety is high, it changes the language to simpler, easier-to-understand terms.

[1281] Input: Sentiment analysis results, disease name candidate list

[1282] Output: Adjusted disease name candidates

[1283] Step 6:

[1284] The device displays the adjusted disease candidate list to the patient, who can review it and provide additional information if necessary.

[1285] Input: Adjusted disease name candidate list

[1286] Output: List of confirmed disease candidates

[1287] Step 7:

[1288] The device compiles the patient's responses and sends the results to medical professionals.

[1289] Input: List of confirmed disease candidates, additional information

[1290] Output: Interview results data for medical professionals

[1291] ---

[1292] Diagnostic Imaging System

[1293] Step 1:

[1294] A medical professional uploads a patient's medical image to a terminal, for example, a chest X-ray image.

[1295] Input: Medical image data

[1296] Output: Uploaded data

[1297] Step 2:

[1298] The terminal transmits the image data to the server.

[1299] Input: Medical image data

[1300] Output: The data sent.

[1301] Step 3:

[1302] The server uses image recognition AI to analyze medical images and identify abnormalities. For example, AI can analyze X-ray images and detect abnormalities.

[1303] Input: Medical image data

[1304] Output: Analysis result data

[1305] Step 4:

[1306] The device uses an emotion engine to analyze the emotional state of medical staff, specifically by analyzing their facial expressions and working speed to detect stress and fatigue.

[1307] Input: facial expression of medical staff, working speed

[1308] Output: Emotion analysis results

[1309] Step 5:

[1310] The server adjusts the report content based on the results of the emotion analysis, for example providing more detailed explanations if stress levels are high.

[1311] Input: Sentiment analysis results, analysis result data

[1312] Output: Reconciled report

[1313] Step 6:

[1314] The device displays the adjusted report to the medical professional and prompts them to confirm it. Specific actions include displaying and confirming the analysis results.

[1315] Input: Reconciled Report

[1316] Output: Confirmed report

[1317] ---

[1318] Diagnosis system based on multiple patient information

[1319] Step 1:

[1320] A medical professional enters information about multiple patients into a terminal. For example, medical records and vital data for Patient A, Patient B, and Patient C.

[1321] Input: Medical information data for each patient

[1322] Output: The input data

[1323] Step 2:

[1324] The device sends the data to the server.

[1325] Input: Medical information data for each patient

[1326] Output: The data sent.

[1327] Step 3:

[1328] The server uses generated AI to perform diagnostic analysis for each patient based on the input data.

[1329] Input: Medical information data for each patient

[1330] Output: Diagnostic analysis results

[1331] Step 4:

[1332] The device uses an emotion engine to analyze the emotional state of medical staff in real time.

[1333] Input: facial expression of medical staff, working speed

[1334] Output: Emotion analysis results

[1335] Step 5:

[1336] The server adjusts the diagnosis report based on the sentiment analysis results, for example adding a brief explanation if the healthcare worker is tired.

[1337] Input: Sentiment analysis results, diagnostic analysis results

[1338] Output: Tailored diagnostic results report

[1339] Step 6:

[1340] The device displays the adjusted report to the medical professional for confirmation and final diagnosis.

[1341] Input: Adjusted diagnostic results report

[1342] Output: Confirmed diagnostic results report

[1343] ---

[1344] Pharmaceutical research and design support

[1345] Step 1:

[1346] Researchers input the conditions necessary for new drug design, such as active ingredients and stability conditions, into the terminal.

[1347] Input: Condition data for new drug design

[1348] Output: The input data

[1349] Step 2:

[1350] The terminal transmits the condition data to the server.

[1351] Input: Condition data for new drug design

[1352] Output: The data sent.

[1353] Step 3:

[1354] The server uses a generative AI to generate candidate molecular structures that meet the conditions.

[1355] Input: Condition data for new drug design

[1356] Output: List of candidate molecular structures

[1357] Step 4:

[1358] The device uses an emotion engine to analyze the researcher's emotional state, specifically by analyzing facial expressions and typing speed to detect stress and fatigue.

[1359] Input: Researcher's facial expression, input speed

[1360] Output: Emotion analysis results

[1361] Step 5:

[1362] The server then adjusts the list of molecular structures based on the results of the sentiment analysis, for example adding detailed annotations to complex molecular structures.

[1363] Input: Sentiment analysis results, candidate list of molecular structures

[1364] Output: Adjusted molecular structure list

[1365] Step 6:

[1366] The terminal displays the adjusted molecular structure list to the researcher and prompts them to confirm it. Specific actions include displaying and confirming the molecular structure.

[1367] Input: Adjusted molecular structure list

[1368] Output: List of confirmed molecular structures

[1369] (Application example 2)

[1370] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1371] While conventional medical operation support systems have improved the efficiency of medical personnel's work, they have not been able to integrate emotion analysis or automatic patrol functions in the security field. As a result, safety inside and outside medical facilities has not been ensured sufficiently, and the burden on medical personnel remains heavy. The present invention aims to add security functions to medical operation support systems, thereby improving safety while performing medical operations efficiently.

[1372] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes [means for receiving input medical information], [means for creating a draft of a clinical document using a generation AI based on the received medical information], and [means for displaying the created clinical document and correcting and confirming it]. This makes it possible to improve the efficiency of medical work and also to increase safety within medical facilities.

[1373] "Medical information" refers to data such as a patient's medical treatment, examination results, health status, medical history, and prescription information.

[1374] "Generative AI" is an algorithm that learns patterns from large amounts of data and automates tasks such as natural language generation, image recognition, and sentiment analysis.

[1375] "Clinical documents" are documents such as medical records, reports, and diagnosis certificates that are created based on medical information.

[1376] An "emotion engine" is a technology that analyzes and recognizes human emotions from input data.

[1377] "Security measures" are systems or devices that patrol a target area and detect suspicious behavior or unusual events.

[1378] A "patrol route" is the route that a security robot travels to check for safety.

[1379] "Emotion analysis" is the process of determining a subject's emotional state from facial expressions, voice, text, etc.

[1380] "Abnormality Report" is a function that notifies the administrator of abnormal situations detected by the system.

[1381] The present invention is an integrated system that uses generative AI and emotion engines to support healthcare professionals in their work, improve communication with patients, and enhance security both inside and outside healthcare facilities. The following describes in detail the embodiments of the present invention.

[1382] Hardware and software used

[1383] Hardware

[1384] 1. Security Robot: A robot equipped with autonomous mobility and a high-resolution camera, this robot patrols inside and outside medical facilities.

[1385] 2. Camera: The security robot will be equipped with a high-resolution camera to capture footage in real time.

[1386] 3. Computer: A computer equipped with a CPU / GPU for real-time data processing. Small computers such as Raspberry Pi are used.

[1387] software

[1388] 1. OpenCV: Image processing library. Performs image processing such as face recognition and anomaly detection.

[1389] 2. Keras: A deep learning library for running emotion recognition models using generative AI.

[1390] 3. Python: A programming language. All processing logic of the present invention is written in Python.

[1391] Specific operation of the system

[1392] 1. Receiving medical information

[1393] The server receives medical information entered by a medical professional, such as the patient's medical history and health status, which is used to create a draft clinical document.

[1394] 2. Generative AI for clinical documentation

[1395] The server analyzes the received medical information and uses a generative AI model to automatically create draft clinical documents that can be reviewed and revised by medical professionals.

[1396] 3. Emotion analysis using an emotion engine

[1397] The server uses an emotion engine to analyze the emotions of medical professionals and patients, which then fine-tunes the content of the documents created and improves the quality of communication.

[1398] 4. Automatic patrols by security robots

[1399] The security robot automatically moves along a set patrol route, checking the safety of designated areas, capturing video footage in real time with a camera and performing facial recognition and emotion analysis.

[1400] 5. Anomaly Detection and Reporting

[1401] The server analyzes the data sent from the security robot in real time to detect abnormal behavior or emotional changes. If an abnormality is detected, it is immediately reported to the administrator and appropriate countermeasures are provided using generative AI.

[1402] Specific examples

[1403] For example, if a security robot encounters a person expressing negative emotions such as "anxiety" or "fear" while patrolling, it will recognize the person's face and analyze their emotion. If an abnormality is detected, it will notify the administrator in real time and send a video of the situation. Furthermore, it will automatically update its security route and patrol areas to ensure greater safety.

[1404] Prompt Sentence Examples

[1405] You may also find the following prompt examples helpful:

[1406] Use emotion recognition models to determine the emotions of people in a security area in real time, identify suspicious behavior or emotional changes, and report them to management, including the ability to automatically adjust patrol routes.

[1407] As described above, the present invention provides an integrated system that achieves both improved efficiency in medical care and enhanced security.

[1408] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1409] Step 1:

[1410] The user inputs medical information such as the patient's medical treatment details and health condition into the terminal. The terminal then sends this information to the server. The input here is text data such as the patient's symptoms and examination results, and the server receives and stores this information.

[1411] Step 2:

[1412] The server uses a generative AI model to create a draft clinical document based on the received medical information. The generated draft includes medical records and reports based on the medical information, automating tasks that would otherwise be performed manually by medical professionals.

[1413] Step 3:

[1414] The server sends the generated draft of the clinical document to the terminal, which displays it. The user checks the displayed document and makes corrections as necessary. After corrections are made, the user confirms the document, and the terminal sends the confirmed document back to the server.

[1415] Step 4:

[1416] The server stores the confirmed clinical documents in a database for future reference and analysis. The documents are stored in text or PDF files.

[1417] Step 5:

[1418] The security robot begins patrolling automatically along a set route. The robot captures real-time video with its built-in camera and transmits it to a terminal. This video is used for facial recognition and emotion analysis.

[1419] Step 6:

[1420] The server performs facial recognition on the received video data to detect the subject's face. The detected facial image is input into the emotion engine, which analyzes the subject's emotions. The input here is a facial image, and the output is an emotional state such as "joy," "fear," or "anxiety."

[1421] Step 7:

[1422] The server detects anomalies based on the analysis results. For example, if negative emotions such as anxiety or fear are detected, the server notifies the administrator in real time. The notification includes the subject's emotional state and a link to the captured video.

[1423] Step 8:

[1424] The server automatically adjusts the patrol route based on the emotion analysis results, for example, changing the route to patrol more frequently in areas where abnormalities are detected. This configuration information is sent to the security robot, which then continues patrolling along the new route.

[1425] The above are the specific processing steps of the system that realizes the application example. The input and output, data processing and calculation contents, and specific operations at each step are explained in detail.

[1426] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1428] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1429] [Third embodiment]

[1430] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1431] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1433] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1437] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1440] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1442] This invention is a system that uses generative AI to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[1443] Clinical Documentation Support

[1444] explanation

[1445] This function allows medical professionals to input medical treatment details and examination results, and the generation AI automatically creates a draft of the clinical document, assisting medical professionals in the process of reviewing, correcting, and confirming the content.

[1446] Specific examples

[1447] The user (healthcare worker) inputs the details of the patient's medical treatment. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[1448] The server receives this information and uses generative AI to generate a draft clinical document stating, "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[1449] The terminal displays the generated document, and the user checks it, makes corrections as necessary, and finalizes it as the final document.

[1450] The server stores the established clinical documents in a database.

[1451] Automated medical interview system

[1452] explanation

[1453] This function allows patients to answer questions via their device, and the generative AI then suggests disease names as candidates for initial diagnosis, enabling efficient initial diagnosis.

[1454] Specific examples

[1455] The user (patient) answers the medical interview questions displayed on the terminal. For example, they might enter, "I have had a headache, fever, and cough for three days."

[1456] The server analyzes this information and generates potential disease names such as "cold, flu, COVID-19."

[1457] The device presents these potential disease names to the patient and then passes on the necessary information to medical professionals.

[1458] Diagnostic Imaging System

[1459] explanation

[1460] This feature allows medical professionals to upload medical images (e.g., X-rays, MRIs, etc.), and image recognition AI analyzes abnormalities and generates a report.

[1461] Specific examples

[1462] The user (healthcare worker) inputs the patient's chest X-ray image into the terminal.

[1463] The server analyzes the received images and uses image recognition AI to identify abnormalities (e.g., shadows in the lungs).

[1464] The server generates and displays the analysis results as a report.

[1465] The terminal displays the generated report, which the user (healthcare professional) can check.

[1466] Diagnosis system based on multiple patient information

[1467] explanation

[1468] This feature integrates data collected from multiple patients and uses generative AI to perform highly accurate diagnostic analysis.

[1469] Specific examples

[1470] The user (healthcare professional) inputs the medical records, vital data, and diagnostic imaging data of Patients A, B, and C into the terminal.

[1471] The server integrates this data and uses generative AI to perform a diagnostic analysis for each patient.

[1472] The server compiles diagnostic results into a report, such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having a decline in kidney function."

[1473] The terminal displays the generated report, and the user (medical professional) checks it and makes a final diagnosis.

[1474] Pharmaceutical research and design support

[1475] explanation

[1476] This feature uses generative AI to generate and predict molecular structures of new drugs, supporting the drug design process.

[1477] Specific examples

[1478] The user (researcher) inputs the conditions necessary for designing a new drug into the terminal.

[1479] The server uses generative AI to generate and predict candidate molecular structures.

[1480] The server stores the generated molecular structures in a database, allowing users (researchers) to search and use them as needed.

[1481] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI to various medical tasks.

[1482] The processing flow will be explained below.

[1483] Clinical Documentation Support

[1484] Processing Steps

[1485] Step 1:

[1486] The user (healthcare worker) uses a terminal to input details of the patient's medical treatment, including symptoms, examination results, and treatment details.

[1487] Step 2:

[1488] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[1489] Step 3:

[1490] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, including the diagnosis and treatment plan.

[1491] Step 4:

[1492] The server transmits the generated draft of the clinical document to the terminal.

[1493] Step 5:

[1494] The terminal displays the received draft of the clinical document on the screen, and the user (healthcare professional) can check and correct it.

[1495] Step 6:

[1496] The user inputs the corrections and presses the confirm button.

[1497] Step 7:

[1498] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[1499] Automated medical interview system

[1500] Processing Steps

[1501] Step 1:

[1502] The terminal displays medical interview questions to the patient on the screen.

[1503] Step 2:

[1504] The user (patient) answers questions and inputs their symptoms and physical condition.

[1505] Step 3:

[1506] The terminal transmits the input answer to the server.

[1507] Step 4:

[1508] The server analyzes the received responses and compares them with a medical database to generate relevant disease name candidates.

[1509] Step 5:

[1510] The server sends the generated disease name candidates to the terminal.

[1511] Step 6:

[1512] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[1513] Diagnostic Imaging System

[1514] Processing Steps

[1515] Step 1:

[1516] The user (medical professional) inputs medical images (e.g., X-rays, MRIs, etc.) into the terminal.

[1517] Step 2:

[1518] The terminal transmits the input image data to the server.

[1519] Step 3:

[1520] The server analyzes the received image data and identifies any abnormalities using image recognition AI.

[1521] Step 4:

[1522] The server generates a report based on the analysis results.

[1523] Step 5:

[1524] The server sends the generated report to the terminal.

[1525] Step 6:

[1526] The terminal displays the report, which the user (healthcare professional) can review.

[1527] Diagnosis system based on multiple patient information

[1528] Processing Steps

[1529] Step 1:

[1530] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[1531] Step 2:

[1532] The terminal transmits the input data to the server.

[1533] Step 3:

[1534] The server integrates the received data and performs diagnostic analysis using generative AI.

[1535] Step 4:

[1536] The server generates a diagnostic report based on the analysis results.

[1537] Step 5:

[1538] The server transmits the generated diagnostic report to the terminal.

[1539] Step 6:

[1540] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[1541] Pharmaceutical research and design support

[1542] Processing Steps

[1543] Step 1:

[1544] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[1545] Step 2:

[1546] The terminal transmits the input conditions to the server.

[1547] Step 3:

[1548] The server uses generative AI to generate and predict candidate molecular structures.

[1549] Step 4:

[1550] The server stores the generated molecular structures in a database.

[1551] Step 5:

[1552] The user (researcher) searches for the information they need and displays it on their terminal.

[1553] Example 1

[1554] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1555] The purpose of this invention is to reduce the wide-ranging workload of medical professionals and improve efficiency in the medical field. Conventional systems require a lot of manual work, such as document creation, medical interviews, image diagnosis, diagnostic analysis of multiple patients, and pharmaceutical research, which is time-consuming and labor-intensive, and also has a high risk of human error. This has resulted in an excessive workload for medical professionals, making it difficult to provide prompt and accurate medical care.

[1556] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1557] In this invention, the server includes: [means for receiving input medical data;] [means for creating a draft of a clinical document using a generative AI model based on the received medical data;] [means for displaying the created clinical document and correcting and confirming it; and] [means for saving the confirmed clinical document in a database. This automates document creation work in medical settings, reducing time and effort and human error.

[1558] "Medical data" refers to a wide range of data related to medical care, such as patient medical information, vital signs, diagnostic imaging data, and medical records.

[1559] A "generative AI model" refers to an artificial intelligence algorithm that uses technologies such as natural language processing and image recognition to generate output tailored to a specific purpose from input data.

[1560] "Clinical documents" refer to important documents in the medical field that contain information such as medical treatment details and examination results.

[1561] "Medical interview items" refer to questions presented to understand the patient's symptoms, medical history, and current health condition.

[1562] "Candidate disease names" refers to a list of possible disease names generated based on medical interview responses and other medical data.

[1563] "Medical images" refers to medical image data such as X-rays, MRIs, and CT scans.

[1564] "Image analysis AI" refers to artificial intelligence algorithms that analyze medical images and identify or diagnose abnormalities.

[1565] "Patient data" refers to all medical information related to individual patients, including medical information, vital signs, and diagnostic imaging data.

[1566] "Diagnosis result" refers to a diagnostic conclusion generated through analysis based on medical data and patient data.

[1567] "Drug design criteria" refers to the scientific and clinical requirements and specifications for designing new drugs.

[1568] "Molecular structure" refers to the internal structure of a drug or other compound, represented as a chemical formula or 3D model.

[1569] A "report" is a document that documents the results of analysis or diagnosis and is compiled in a format that can be reviewed and used by medical professionals.

[1570] "Database" refers to a computer system for systematically storing and managing information such as medical data, clinical documents, and molecular structures.

[1571] This invention is a system that uses generative AI models to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[1572] Hardware and software used

[1573] The system's main hardware consists of a server and a terminal. The server receives and analyzes data, runs the generative AI model, and stores the data. The terminal provides an interface with the user, allowing data input and display of results.

[1574] Specific software includes:

[1575] Generative AI models: Natural language processing models such as OpenAI's GPT-4

[1576] Image analysis AI: Image recognition services such as Google Cloud Vision and AWS Rekognition

[1577] Database: A data management system such as Firebase or PostgreSQL

[1578] Clinical Documentation Support

[1579] The user (healthcare professional) inputs the details of the medical treatment from the terminal. For example, they input data such as "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal then sends the input details of the medical treatment to the server. The server inputs the received data as prompts into the generative AI model, generating a draft clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal displays the generated document, and the user can review it, make corrections as necessary, and finalize it as the final document. The server then saves the finalized clinical document in a database.

[1580] Example prompt: "Please use this clinical content to draft a clinical document."

[1581] Automated medical interview system

[1582] The user (patient) answers the medical interview questions displayed on the device. For example, they might enter, "I have had a headache, fever, and cough for three days." The device then sends the entered medical interview data to the server. The server then inputs the received data as prompts into the generative AI model, generating disease name candidates such as "cold, flu, COVID-19." The device then presents these disease name candidates to the patient and then conveys the necessary information to medical professionals.

[1583] Example prompt: "Please list possible diseases based on this interview data."

[1584] Diagnostic Imaging System

[1585] The user (healthcare professional) uploads an image file (e.g., a chest X-ray image) to the device. The device sends the image file to the server. The server inputs the received image into the image analysis AI and identifies abnormal areas (e.g., shadows in the lungs). The image analysis AI generates a report of the analysis results. The server sends the generated report to the device, which displays the report and the user (healthcare professional) confirms it.

[1586] Example prompt: "Analyze this image to identify anomalies and generate a report."

[1587] Diagnosis system based on multiple patient information

[1588] The user (healthcare professional) inputs multiple patient information (e.g., medical record information, vital signs, diagnostic imaging data) into the terminal. The terminal sends all input data to the server. The server integrates the received data and inputs it as prompts into the generative AI model, performing diagnostic analysis for each patient. For example, it generates a diagnosis result such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having decreased renal function." The server then sends the generated diagnosis result as a report to the terminal, which displays the report and the user confirms it.

[1589] Example prompt: "Generate a diagnosis based on these patient data."

[1590] Pharmaceutical research and design support

[1591] The user (researcher) inputs the conditions necessary for designing a new drug into a terminal. The terminal then sends the input design conditions to the server. The server inputs the received conditions as prompts into the generative AI model, generating and predicting candidate molecular structures. The server stores the generated molecular structures in a database, allowing users to search and use them as needed.

[1592] Example prompt: "Generate molecular structure candidates based on these conditions."

[1593] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI models to various medical tasks.

[1594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1595] Clinical Documentation Support

[1596] Step 1: The user inputs the details of the medical treatment from the terminal.

[1597] Specifically, the user uses a keyboard or voice input to input, "The patient complains of pain in the right shoulder, and examination reveals no fractures."

[1598] Step 2: The terminal sends the entered medical details to the server.

[1599] The input is the medical data, and the output is the transmitted data. HTTPS is used as the communication protocol.

[1600] Step 3: The server inputs the received data into the generative AI model as prompts.

[1601] The input is medical content data, and the output is a prompt: "Please use this medical content to draft a clinical document." The generative AI model analyzes the received data and generates an appropriate draft clinical document.

[1602] Step 4: The generative AI model returns the generated draft clinical document to the server.

[1603] The input is the analysis result data, and the output is the generated draft clinical document.

[1604] Step 5: The server sends the generated draft to the device.

[1605] The input is the draft data, and the output is the submitted draft data.

[1606] Step 6: The terminal displays the generated document, which the user can review and modify if necessary.

[1607] As a specific operation, the user uses a text editor to edit a document on the screen of the terminal.

[1608] Step 7: The user finalizes the document.

[1609] The input is the corrected document and the output is the finalized document.

[1610] Step 8: The terminal sends the confirmed document to the server.

[1611] The input is the finalized document data, and the output is the transmitted data.

[1612] Step 9: The server saves the finalized document to the database.

[1613] The input is the finalized document data, and the output is the data stored in the database.

[1614] Automated medical interview system

[1615] Step 1: The user (patient) answers the medical interview questions displayed on the terminal.

[1616] Specifically, in response to the question, enter "I have had a headache, fever, and cough for three days" in the text box.

[1617] Step 2: The terminal transmits the entered medical interview data to the server.

[1618] The input is the medical interview response data, and the output is the transmitted data.

[1619] Step 3: The server inputs the received data into the generative AI model as prompts.

[1620] The input is the patient's interview response data, and the output is a prompt statement: "Please list possible disease names based on this interview data."

[1621] Step 4: The generative AI model generates disease name candidates based on the received data.

[1622] The input is medical interview response data, and the output is disease name candidate data.

[1623] Step 5: The server sends the generated disease name candidates to the terminal.

[1624] The input is the disease name candidate data, and the output is the transmitted data.

[1625] Step 6: The terminal presents the patient with possible disease names and then conveys the necessary information to the medical professional.

[1626] Specifically, the system displays possible disease names on the screen, such as "cold, influenza, COVID-19," and the user confirms them.

[1627] Diagnostic Imaging System

[1628] Step 1: The user (healthcare professional) uploads an image file to the device.

[1629] As a specific operation, a file selection dialog is used to select a chest X-ray image.

[1630] Step 2: The device sends the image file to the server.

[1631] The input is the image file and the output is the transmitted data.

[1632] Step 3: The server inputs the received image into the image analysis AI.

[1633] The input is an image file, and the output is a prompt: "Analyze this image to identify any abnormalities and generate a report."

[1634] Step 4: Image analysis AI analyzes the pixel data of the image and identifies abnormalities.

[1635] The input is an image file, and the output is the analysis result data.

[1636] Step 5: The image analysis AI generates a report of the analysis results.

[1637] The input is analysis result data, and the output is report data.

[1638] Step 6: The server sends the generated report to the terminal.

[1639] The input is the report data and the output is the transmitted data.

[1640] Step 7: The terminal displays the report and the user (healthcare professional) confirms it.

[1641] Specifically, the generated report is displayed on the screen and checked by a medical professional.

[1642] Diagnosis system based on multiple patient information

[1643] Step 1: A user (healthcare worker) inputs multiple patient information into a terminal.

[1644] Specifically, the patient enters medical record information and vital data into a form.

[1645] Step 2: The device sends all entered data to the server.

[1646] The input is a plurality of patient information data, and the output is the transmitted data.

[1647] Step 3: The server consolidates the received data.

[1648] The input is multiple patient information data, and the output is the integrated data.

[1649] Step 4: The server inputs the integrated data as prompts into the generative AI model.

[1650] The input is the integrated data, and the output is the prompt statement "Please generate a diagnosis based on these patient data."

[1651] Step 5: The generative AI model generates a diagnosis based on the integrated data.

[1652] The input is the integrated data, and the output is the diagnostic result data.

[1653] Step 6: The server sends the generated diagnostic results to the terminal as a report.

[1654] The input is the diagnostic result data, and the output is the transmitted data.

[1655] Step 7: The terminal displays the report, which the user confirms and makes a final diagnosis.

[1656] Specifically, the report is displayed and a medical professional makes a final diagnosis.

[1657] Pharmaceutical research and design support

[1658] Step 1: The user (researcher) inputs the design conditions for a new drug into the terminal.

[1659] As a specific operation, the design conditions are input into the text box.

[1660] Step 2: The terminal transmits the input design conditions to the server.

[1661] The input is the design condition data, and the output is the transmitted data.

[1662] Step 3: The server inputs the received conditions into the generative AI model as prompts.

[1663] The input is design condition data, and the output is a prompt statement: "Generate molecular structure candidates based on these conditions."

[1664] Step 4: The generative AI model generates candidate molecular structures based on the conditions.

[1665] The input is design condition data, and the output is molecular structure candidate data.

[1666] Step 5: The server stores the generated molecular structure in a database.

[1667] The input is molecular structure candidate data, and the output is the saved data.

[1668] Step 6: The user searches for and uses molecular structures from the terminal as needed.

[1669] Specifically, the molecular structure search interface is used to search and evaluate stored molecular structures.

[1670] The specific processing steps for each function have been described above.

[1671] (Application example 1)

[1672] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1673] Conventional medical support systems require medical personnel to be present at all times, making it difficult to respond quickly in emergencies. In particular, emergency medical support systems using mobile means such as autonomous vehicles have difficulty providing first aid and diagnosis when medical personnel are not present on board, making it impossible to provide appropriate medical support.

[1674] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1675] In this invention, the server includes a means for receiving input medical information, a means for creating a draft clinical document using a generative artificial intelligence based on the received medical information, a means for displaying, correcting, and confirming the created clinical document, a means for inputting patient symptoms in an emergency and starting processing, and a means for using the generative artificial intelligence to create instructions for first aid in real time based on the input symptoms. This enables the autonomous vehicle to provide prompt and appropriate first aid and safely transport the patient even when medical personnel are not on-site.

[1676] "Medical information" refers to all data related to patients, such as medical treatment details, symptoms, examination results, and medical images.

[1677] "Generative AI" is an AI technology that automatically generates natural language sentences and reports from given data.

[1678] "Clinical documents" are official documents for medical professionals that contain information such as examination results and treatment plans.

[1679] "Emergency" refers to situations where medical personnel are not available or where immediate first aid is required.

[1680] "Real-time first aid instructions" are specific instructions for performing emergency medical procedures that are instantly generated using signal processing and generative AI.

[1681] A "medical history" is a series of questions asked to understand a patient's symptoms and medical history.

[1682] A "medical database" is a system that accumulates and manages medical information and case data.

[1683] "Medical images" are image data obtained using medical equipment such as X-rays and MRIs.

[1684] "Image recognition artificial intelligence" is an artificial intelligence technology used to analyze medical images and identify abnormalities.

[1685] "Instructions for first aid based on diagnostic results" are specific treatment procedures and treatment details created by the generative artificial intelligence based on the diagnostic results.

[1686]

[1687] This invention provides a concrete implementation of a generative AI-based emergency medical support system to be installed in an autonomous vehicle. This system enables rapid first aid in emergencies and is primarily composed of the following main components:

[1688] 1. Server:

[1689] The server is the central system that performs various processes using generative AI models. The server is responsible for the following processes:

[1690] Receiving and generating medical information. Drafting clinical documentation using AI.

[1691] Generate emergency first aid instructions based on patient symptom information.

[1692] An initial diagnosis is made based on the results of the interview, after comparing them with a medical database.

[1693] The server software includes generative AI models using TensorFlow and Keras, and a database management system.

[1694] 2. Terminal:

[1695] The terminal is a device such as a smartphone or smart glasses that provides an interface for patients and medical professionals to input and view information. The terminal has the following functions:

[1696] The user (patient or healthcare professional) enters symptoms and medical questionnaire answers.

[1697] Displays generated documents and diagnostic results from the server.

[1698] Displays emergency procedure instructions as needed.

[1699] For example, a patient may type into their smartphone, "I have severe chest pain and difficulty breathing." The device can also automatically input information using voice commands.

[1700] 3. User:

[1701] The user is a patient or a medical professional who uses the system. The user performs the following operations:

[1702] Enter your symptoms in case of an emergency.

[1703] Respond based on emergency instructions from the server.

[1704] Review and correct clinical documentation as needed.

[1705] The user can utilize the system through an intuitive interface, maximizing the effectiveness of the invention.

[1706] The following are examples of prompt sentences:

[1707] Example prompt sentence:

[1708] "The patient is experiencing chest pain and difficulty breathing. Urgent treatment is needed. What are the possible diagnoses? What first aid would be needed?"

[1709] Specific examples of hardware include self-driving vehicles, smartphones, smart glasses, and IoT medical devices (e.g., robotic arms, first aid kits).

[1710] Specific examples of software include generative AI models (TensorFlow / Keras models), real-time image analysis software, and control software for IoT devices.

[1711] In this way, the present invention realizes a system that can provide prompt and appropriate first aid in an autonomous vehicle even when medical personnel are not present.

[1712] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1713] Step 1:

[1714] The user inputs symptoms into a smartphone or smart glasses. For example, if the user inputs "severe chest pain and difficulty breathing," the device receives this input. The input data includes the patient's symptoms and condition.

[1715] Step 2:

[1716] The device sends the symptom information it receives to the server. At this time, the symptom information is transferred to the server as text data. The server receives this text data and inputs it into the generative AI model.

[1717] Step 3:

[1718] The server uses generative AI to create a draft clinical document from the received symptom information. The generative AI model generates an initial diagnosis based on the input data. Here, the prompt used is "The patient complains of severe chest pain and difficulty breathing. What are some possible diagnosis candidates?" As a result, a candidate diagnosis and first aid instructions are generated.

[1719] Step 4:

[1720] The server sends the generated clinical document and candidate diagnoses to the terminal, which displays the generated document for the user to review. The user can then review the displayed information and make corrections as necessary.

[1721] Step 5:

[1722] The server stores the clinical document in a database after the user has confirmed and corrected it. The confirmed document is stored in the medical database for future reference and use.

[1723] Step 6:

[1724] The server generates emergency first aid instructions based on the initial diagnosis. Using generative artificial intelligence, it generates instructions in real time, including specific first aid procedures. For example, it creates "operation instructions for the robot arm to begin cardiac massage."

[1725] Step 7:

[1726] The server sends the generated first aid instructions to the IoT device. The terminal receives the instructions and sends a signal to control a nearby medical device (e.g., a robotic arm). Emergency first aid is then performed based on the specific control procedure.

[1727] Step 8:

[1728] The terminal displays the final instructions sent from the server and the manufactured treatment details to the user, who can then continue responding to the emergency based on this information.

[1729] In this way, the system can provide fast and accurate real-time first aid in emergencies.

[1730] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1731] This invention is an integrated system that uses generative AI and an emotion engine to support medical professionals in their work and improve communication with patients. This system combines clinical documentation support, an automated medical interview system, an image diagnosis system, a diagnosis system using multiple patient information, and pharmaceutical research and design support functions with an emotion engine.

[1732] Clinical Documentation Support

[1733] explanation

[1734] This function allows medical professionals to input medical treatment details and examination results, and the generative AI automatically creates a draft of the clinical document, uses an emotion engine to recognize the medical professional's emotions, and assists in the process of creating an appropriate document based on the input content.

[1735] Specific examples

[1736] The user (healthcare worker) inputs the details of the patient's medical treatment into the terminal, for example, "The patient complains of pain in the right shoulder, and the examination reveals no fractures."

[1737] The device sends this information to a server, which then uses a generative AI to generate a draft clinical document stating, "The patient's main complaint was pain in the right shoulder, and examination revealed no fractures."

[1738] Additionally, the device uses an emotion engine to analyze the user's emotions as they type and detect signs of stress or fatigue.

[1739] The server adjusts the relevance of document content depending on the emotional state, for example, producing quicker and more concise documents.

[1740] The terminal displays the generated document, and the user checks and corrects it, and then presses the confirm button to confirm it.

[1741] The server stores the established clinical documents in a database.

[1742] Automated medical interview system

[1743] explanation

[1744] This function allows patients to answer questions via their device, and the generative AI then suggests possible disease names for initial diagnosis, using an emotion engine to recognize the patient's emotions and adjust the interview process accordingly.

[1745] Specific examples

[1746] The patient enters into the terminal, "I have had a headache, fever, and cough for three days."

[1747] The device sends this information to a server, which then uses generative AI to generate potential disease names such as "cold, flu, COVID-19."

[1748] At the same time, the device uses an emotion engine to check the patient's emotions and detect, for example, anxiety or tension.

[1749] The server adjusts the display of the medical questionnaire based on the results of the emotion analysis, changing the explanation of the possible disease name to easier-to-understand language and asking additional questions.

[1750] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[1751] Diagnostic Imaging System

[1752] explanation

[1753] This function allows medical professionals to upload medical images and uses image recognition AI and an emotion engine to analyze abnormalities, supporting diagnosis while reducing stress.

[1754] Specific examples

[1755] A medical professional inputs the patient's chest X-ray image into a terminal.

[1756] The device sends image data to a server, which then uses image recognition AI to identify abnormalities.

[1757] At the same time, the device uses an emotion engine to check the current emotional state of medical workers and detect high levels of stress or fatigue.

[1758] The server generates a report based on the analysis results and tailors the report content depending on the emotional state, for example providing additional analytical details to help reduce stress.

[1759] The terminal displays the report, which is then reviewed by medical personnel.

[1760] Diagnosis system based on multiple patient information

[1761] explanation

[1762] This function integrates data collected from multiple patients, uses generative AI and an emotion engine to perform highly accurate diagnostic analysis, and presents results based on the emotional state of medical professionals.

[1763] Specific examples

[1764] Medical staff enter the medical records, vital signs, and diagnostic imaging data of patients A, B, and C into the terminal.

[1765] The device sends this data to a server, which uses generative AI to perform a diagnostic analysis for each patient.

[1766] The emotion engine monitors the emotional state of healthcare professionals during input and detects high levels of fatigue and stress.

[1767] The server compiles the diagnostic results into a report, adapts the content depending on the emotional state, and provides additional information and analysis results to, for example, reduce the burden on medical professionals.

[1768] The terminal displays the report, which is then reviewed and a final diagnosis is made by a medical professional.

[1769] Pharmaceutical research and design support

[1770] explanation

[1771] This feature uses generative AI to generate and predict molecular structures of new drugs, and an emotion engine to recognize researchers' emotions while supporting the drug design process.

[1772] Specific examples

[1773] Researchers input the conditions necessary for new drug design into the terminal.

[1774] The device sends this information to a server, which then uses generative AI to generate and predict candidate molecular structures.

[1775] At the same time, the device uses an emotion engine to analyze the researcher's emotional state and detect stress and fatigue.

[1776] The server stores the generated list of molecular structures in a database and presents the researcher with the most appropriate information based on their emotional state, for example by adding detailed annotations for complex molecular structures.

[1777] Researchers use the terminal to search for the necessary information and check the displayed molecular structure.

[1778] As described above, by applying generative AI and an emotion engine to various medical tasks, the present invention reduces the burden on medical professionals, improves efficiency in the medical field, and enables smooth communication with patients.

[1779] The processing flow will be explained below.

[1780] Clinical Documentation Support

[1781] Processing Steps

[1782] Step 1:

[1783] The user (healthcare worker) inputs the details of the patient's medical treatment into the terminal. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[1784] Step 2:

[1785] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[1786] Step 3:

[1787] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, such as "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[1788] Step 4:

[1789] The device uses an emotion engine to analyze the user's emotions, for example, detecting signs of stress or fatigue through facial recognition and voice tone analysis.

[1790] Step 5:

[1791] The server adjusts document content to take into account the user's emotional state, for example by generating a more concise document if the user is feeling stressed.

[1792] Step 6:

[1793] The server transmits the generated document to the terminal.

[1794] Step 7:

[1795] The terminal displays the generated clinical document on the screen, allowing the user to check and correct it.

[1796] Step 8:

[1797] After completing the corrections, the user presses the confirm button.

[1798] Step 9:

[1799] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[1800] Automated medical interview system

[1801] Processing Steps

[1802] Step 1:

[1803] The terminal displays medical interview questions to the patient on the screen.

[1804] Step 2:

[1805] The user (patient) answers questions and inputs their symptoms and physical condition. For example, they might input "I have had a headache, fever, and cough for three days."

[1806] Step 3:

[1807] The terminal receives the response and transmits the data to the server.

[1808] Step 4:

[1809] The server analyzes the received response and compares it with a medical database to generate relevant disease name candidates, such as "cold, flu, COVID-19."

[1810] Step 5:

[1811] The device uses an emotion engine to analyze the patient's emotions and detect feelings such as anxiety and tension.

[1812] Step 6:

[1813] The server takes into account the patient's emotional state and adjusts the content of the interview, for example by making the explanation of potential illnesses more concise and easy to understand if anxiety is detected.

[1814] Step 7:

[1815] The server sends the generated disease name candidates to the terminal and displays them to the patient.

[1816] Step 8:

[1817] The terminal displays possible disease names and then sends the necessary information to medical professionals.

[1818] Diagnostic Imaging System

[1819] Processing Steps

[1820] Step 1:

[1821] A user (medical worker) inputs a medical image into the terminal, for example, uploading a chest X-ray image.

[1822] Step 2:

[1823] The terminal receives the image data and transmits the data to the server.

[1824] Step 3:

[1825] The server analyzes the received image data and uses image recognition AI to identify abnormalities, such as shadows in the lungs.

[1826] Step 4:

[1827] The device uses an emotion engine to analyze the emotions of medical staff, for example, to detect fatigue and stress.

[1828] Step 5:

[1829] The server generates a report of the analysis results, taking into account the emotional state, for example, if fatigue is detected, it will briefly summarize the analysis results.

[1830] Step 6:

[1831] The server sends the generated report to the terminal.

[1832] Step 7:

[1833] The terminal displays the report, which the user (healthcare professional) can review.

[1834] Diagnosis system based on multiple patient information

[1835] Processing Steps

[1836] Step 1:

[1837] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[1838] Step 2:

[1839] The terminal transmits the input data to the server.

[1840] Step 3:

[1841] The server integrates the received data and performs diagnostic analysis using generative AI, generating diagnostic results such as, for example, that Patient A is at high risk of myocardial infarction, Patient B is suspected of having pneumonia, and Patient C is suspected of having declining kidney function.

[1842] Step 4:

[1843] The device uses an emotion engine to analyze the emotions of medical workers, for example, to detect stress or fatigue.

[1844] Step 5:

[1845] The server generates a report of the analysis results, taking into account the emotional state, for example, if stress is detected, it will summarize the diagnosis in a concise and easy-to-understand manner.

[1846] Step 6:

[1847] The server transmits the generated diagnostic report to the terminal.

[1848] Step 7:

[1849] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[1850] Pharmaceutical research and design support

[1851] Processing Steps

[1852] Step 1:

[1853] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[1854] Step 2:

[1855] The terminal transmits the input conditions to the server.

[1856] Step 3:

[1857] The server uses generative AI to generate and predict candidate molecular structures.

[1858] Step 4:

[1859] The device uses an emotion engine to analyze the researcher's emotions, for example, to detect stress or fatigue.

[1860] Step 5:

[1861] The server takes emotional state into account to tailor the molecular structure generation results, for example by adding detailed annotations.

[1862] Step 6:

[1863] The server stores the generated molecular structures in a database and makes them available for user retrieval.

[1864] Step 7:

[1865] The terminal displays the search results, which the user (researcher) can confirm.

[1866] Example 2

[1867] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1868] In the current medical system, medical professionals are burdened with a heavy workload and are unable to properly manage their emotions and stress, resulting in problems with smooth communication with patients. Furthermore, emotional adjustments are not made during interviews or diagnostic imaging, which can lead to reduced diagnostic accuracy and work efficiency. It is necessary to solve these problems and provide a better environment for both medical professionals and patients.

[1869] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: [means for receiving input medical information]; [means for creating a draft of a clinical document using a generation AI based on the received medical information]; [means for analyzing the user's emotions using an emotion engine]; [means for adjusting the clinical document content based on the emotion analysis results]; [means for displaying the created clinical document, correcting and confirming it]; and [means for saving the confirmed clinical document in a database]. This enables document creation that takes into account the emotional state of medical professionals, thereby realizing effective communication.

[1870] The server also includes means for presenting medical interview questions to the patient and receiving the patient's responses, means for comparing the received responses with a medical database and generating relevant disease name candidates, means for analyzing the patient's emotions using an emotion engine, means for adjusting the method for presenting the disease name candidates based on the emotion analysis results, and means for presenting the generated disease name candidates to a medical professional. This enables medical interviews that take the patient's emotions into consideration, resulting in more accurate initial diagnoses.

[1871] Furthermore, the server includes: [means for inputting medical images and analyzing abnormalities using image recognition AI; [means for analyzing the emotions of medical professionals using an emotion engine;] [means for adjusting the report content based on the emotion analysis results; and [means for generating and displaying the analysis results as a report.] This enables image diagnosis that takes into account the emotional state of medical professionals, thereby improving diagnostic accuracy.

[1872] "Inputted medical information" refers to medical-related information such as medical treatment details, symptoms, and vital data that medical professionals and patients provide to the system.

[1873] "Generative AI" is a system that uses artificial intelligence technology to automatically generate text, images, etc., and specifically refers to a model that uses natural language processing technology.

[1874] "Clinical documents" are documents in which medical professionals record the details of medical treatment and examinations, including the patient's medical history, symptoms, and examination results.

[1875] An "emotion engine" is a technology that analyzes the emotional state of a user or patient and adjusts the system's operation and output based on that emotion.

[1876] The "emotion analysis result" is the result of the emotion analyzed by the emotion engine, and is data indicating the state of stress, fatigue, anxiety, etc. of the user or patient.

[1877] "Medical history questions" are a set of questions presented by the system to understand the patient's symptoms and medical history.

[1878] "Responses" are responses given by patients to medical interview questions, and include information about symptoms and physical conditions.

[1879] A "medical database" is a database that stores medical information, diagnosis results, disease name information, etc., and is used as reference information for medical interviews and diagnoses.

[1880] "Candidate disease names" is a list of possible disease names generated by the system based on the patient's answers and data received.

[1881] "Medical images" are medical image data taken for diagnosis or treatment, such as X-ray images, CT scan images, and MRI images.

[1882] "Image recognition AI" is an artificial intelligence technology that automatically analyzes medical images and identifies abnormalities and features.

[1883] A "report" is a document of diagnostic and analytical results generated by the system, intended to provide information to medical professionals.

[1884] A "database" is a system for efficiently storing, searching, and managing large amounts of data, and in this case is used to store clinical documents and medical information.

[1885] "Display" refers to the act of showing the data or documents generated by the system to the user or patient, and means outputting them on a screen.

[1886] "Modification" refers to the act of a user making changes or corrections to a generated document or data.

[1887] "Confirming" refers to the act of the user finally confirming the corrections and saving them as official data.

[1888] This invention is an integrated system that uses generative AI and an emotion engine to support medical professionals in their work and improve communication with patients. This system combines the emotion engine with clinical documentation support, an automated medical interview system, an image diagnosis system, a diagnosis system using multiple patient information, and pharmaceutical research and design support functions.

[1889] Clinical Documentation Support

[1890] explanation

[1891] With this function, the user (medical professional) inputs the details of medical treatment and examination results into the device, and the generative AI automatically creates a draft of the clinical document based on that information.The emotion engine analyzes the user's emotional state and adjusts the appropriateness of the document according to the input content.

[1892] Specific examples

[1893] The user enters "The patient complains of pain in the right shoulder, and examination revealed no fractures" into the terminal. The terminal sends this information to the server, which uses a generative AI to generate a draft clinical document stating, "The patient complains of pain in the right shoulder, and examination revealed no fractures." At the same time, the terminal uses an emotion engine to analyze the user's emotions as they type and detect signs of stress or fatigue. The server adjusts the content of the document according to the user's emotional state, creating a concise document. The terminal displays the generated document, and the user can review and edit it, then press the confirm button to confirm. The server then saves the confirmed clinical document in its database.

[1894] Prompt Sentence Examples

[1895] The patient complained of pain in the right shoulder, and examination revealed no fractures.

[1896] Automated medical interview system

[1897] explanation

[1898] In the automated medical interview system, patients answer questions via a device, and the generative AI then suggests possible initial diagnosis names. It also uses an emotion engine to recognize the patient's emotions and adjust the interview process accordingly.

[1899] Specific examples

[1900] The patient enters into the device, "I have had a headache, fever, and cough for three days." The device sends this information to the server, which uses generative AI to generate disease name candidates such as "cold, flu, and COVID-19." At the same time, the device uses an emotion engine to check the patient's emotional state and detect anxiety or tension. Based on the emotion analysis results, the server adjusts the display of the medical interview content, changing the explanation of the disease name candidates to easier-to-understand language or asking additional questions. The device displays the disease name candidates to the patient and then conveys the necessary information to medical professionals.

[1901] Prompt Sentence Examples

[1902] Headache, fever, and cough for 3 days

[1903] Diagnostic Imaging System

[1904] explanation

[1905] In the image diagnosis system, when medical professionals upload medical images, the image recognition AI analyzes abnormalities. An emotion engine is also used to analyze the emotional state of the medical professionals and support the diagnosis.

[1906] Specific examples

[1907] A medical professional inputs a patient's chest X-ray image into a terminal. The terminal then sends the image data to a server, which uses image recognition AI to identify abnormalities. At the same time, the terminal uses an emotion engine to analyze the medical professional's emotional state and detect high levels of stress or fatigue. The server generates a report based on the analysis results and adjusts the report content according to the medical professional's emotional state. The terminal then displays the generated report for the medical professional to review.

[1908] Prompt Sentence Examples

[1909] Analysis of chest X-ray images

[1910] Diagnosis system based on multiple patient information

[1911] explanation

[1912] The diagnostic system for multiple patient information integrates data collected from multiple patients, and generative AI performs highly accurate diagnostic analysis. An emotion engine is used to present results according to the emotional state of medical professionals.

[1913] Specific examples

[1914] A medical professional inputs the medical records, vital signs, and diagnostic imaging data of Patients A, B, and C into a terminal. The terminal then transmits this data to a server, which then uses generative AI to perform diagnostic analysis for each patient. At the same time, an emotion engine monitors the emotional state of the medical professional and detects high levels of fatigue or stress. The server then compiles the diagnostic results into a report, adjusts the content according to the emotional state, and provides additional information and analysis results. The terminal then displays the report, allowing the medical professional to review and make a final diagnosis.

[1915] Prompt Sentence Examples

[1916] Analyze the medical records of Patient A, Patient B, and Patient C

[1917] Pharmaceutical research and design support

[1918] explanation

[1919] The pharmaceutical research and design support function uses generative AI to generate and predict the molecular structure of new drugs, and an emotion engine to analyze researchers' emotions while supporting the pharmaceutical design process.

[1920] Specific examples

[1921] Researchers input the conditions required for new drug design into a device. The device sends this information to a server, which uses generative AI to generate and predict candidate molecular structures. At the same time, the device uses an emotion engine to analyze the researcher's emotional state and detect stress or fatigue. The server stores the list of generated molecular structures in a database and presents the researcher with the most appropriate information based on their emotional state. For example, detailed annotations are added for complex molecular structures. The researcher then uses the device to search for the required information and review the displayed molecular structures.

[1922] Prompt Sentence Examples

[1923] Enter the conditions for new drug design

[1924] As described above, by applying generative AI and an emotion engine to various medical tasks, the present invention reduces the burden on medical professionals, improves efficiency in the medical field, and enables smooth communication with patients.

[1925] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1926] Clinical Documentation Support

[1927] Step 1:

[1928] The user inputs the details of the medical treatment into the terminal. Specifically, the user inputs the text "The patient complained of pain in the right shoulder, and the examination revealed no fractures."

[1929] Input: Medical treatment text

[1930] Output: Input data

[1931] Step 2:

[1932] The terminal sends the input data to the server using a secure communication protocol such as HTTPS.

[1933] Input: Text data entered

[1934] Output: The data sent.

[1935] Step 3:

[1936] The server uses a generation AI to generate a draft of a clinical document from the input data. Specifically, the generation AI (for example, GPT-3) receives a prompt sentence, "The patient complained of pain in the right shoulder, and examination revealed no fractures," and generates a clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures."

[1937] Input: Text data entered

[1938] Output: Draft clinical document

[1939] Step 4:

[1940] The device uses an emotion engine to analyze the user's emotional state. Specifically, the emotion engine analyzes facial expressions and typing speed to detect signs of stress or fatigue.

[1941] Input: User facial expression, input speed

[1942] Output: Emotion analysis results

[1943] Step 5:

[1944] The server adjusts the content of the generated clinical document based on the results of the sentiment analysis, for example, shortening the document if the user is fatigued.

[1945] Input: Sentiment analysis results, clinical document draft

[1946] Output: Reconciled clinical documentation

[1947] Step 6:

[1948] The terminal displays the adjusted clinical document to the user, who reviews it and makes corrections as necessary. Specific actions include correcting and reviewing the document.

[1949] Input: Reconciled clinical documentation

[1950] Output: Verified clinical documentation

[1951] Step 7:

[1952] When the user has finished correcting the document, he or she presses the confirm button to confirm the document.

[1953] Input: Verified clinical documentation

[1954] Output: Confirmed clinical documentation

[1955] Step 8:

[1956] The server stores the confirmed clinical documents in a database, taking appropriate measures to ensure data integrity and security.

[1957] Input: Confirmed clinical documentation

[1958] Output: Documents stored in a database

[1959] ---

[1960] Automated medical interview system

[1961] Step 1:

[1962] The patient enters their symptoms into the terminal, specifically, "I have had a headache, fever, and cough for three days."

[1963] Input: Patient symptom text

[1964] Output: Input data

[1965] Step 2:

[1966] The terminal transmits the input data to the server.

[1967] Input: Symptom text data

[1968] Output: The data sent.

[1969] Step 3:

[1970] The server uses generation AI to generate disease name candidates from the input data. For example, based on the information "headache, fever, and cough that has continued for three days," it generates candidate disease names such as "cold, flu, and COVID-19."

[1971] Input: Text data of patient symptoms

[1972] Output: List of disease candidates

[1973] Step 4:

[1974] The device uses an emotion engine to analyze the patient's emotional state, specifically detecting anxiety or tension by analyzing facial expressions and voice during input.

[1975] Input: Patient's facial expression, voice

[1976] Output: Emotion analysis results

[1977] Step 5:

[1978] The server adjusts the way it presents disease candidates based on the results of emotion analysis. For example, if anxiety is high, it changes the language to simpler, easier-to-understand terms.

[1979] Input: Sentiment analysis results, disease name candidate list

[1980] Output: Adjusted disease name candidates

[1981] Step 6:

[1982] The device displays the adjusted disease candidate list to the patient, who can review it and provide additional information if necessary.

[1983] Input: Adjusted disease name candidate list

[1984] Output: List of confirmed disease candidates

[1985] Step 7:

[1986] The device compiles the patient's responses and sends the results to medical professionals.

[1987] Input: List of confirmed disease candidates, additional information

[1988] Output: Interview results data for medical professionals

[1989] ---

[1990] Diagnostic Imaging System

[1991] Step 1:

[1992] A medical professional uploads a patient's medical image to a terminal, for example, a chest X-ray image.

[1993] Input: Medical image data

[1994] Output: Uploaded data

[1995] Step 2:

[1996] The terminal transmits the image data to the server.

[1997] Input: Medical image data

[1998] Output: The data sent.

[1999] Step 3:

[2000] The server uses image recognition AI to analyze medical images and identify abnormalities. For example, AI can analyze X-ray images and detect abnormalities.

[2001] Input: Medical image data

[2002] Output: Analysis result data

[2003] Step 4:

[2004] The device uses an emotion engine to analyze the emotional state of medical staff, specifically by analyzing their facial expressions and working speed to detect stress and fatigue.

[2005] Input: facial expression of medical staff, working speed

[2006] Output: Emotion analysis results

[2007] Step 5:

[2008] The server adjusts the report content based on the results of the emotion analysis, for example providing more detailed explanations if stress levels are high.

[2009] Input: Sentiment analysis results, analysis result data

[2010] Output: Reconciled report

[2011] Step 6:

[2012] The device displays the adjusted report to the medical professional and prompts them to confirm it. Specific actions include displaying and confirming the analysis results.

[2013] Input: Reconciled Report

[2014] Output: Confirmed report

[2015] ---

[2016] Diagnosis system based on multiple patient information

[2017] Step 1:

[2018] A medical professional enters information about multiple patients into a terminal. For example, medical records and vital data for Patient A, Patient B, and Patient C.

[2019] Input: Medical information data for each patient

[2020] Output: The input data

[2021] Step 2:

[2022] The device sends the data to the server.

[2023] Input: Medical information data for each patient

[2024] Output: The data sent.

[2025] Step 3:

[2026] The server uses generated AI to perform diagnostic analysis for each patient based on the input data.

[2027] Input: Medical information data for each patient

[2028] Output: Diagnostic analysis results

[2029] Step 4:

[2030] The device uses an emotion engine to analyze the emotional state of medical staff in real time.

[2031] Input: facial expression of medical staff, working speed

[2032] Output: Emotion analysis results

[2033] Step 5:

[2034] The server adjusts the diagnosis report based on the sentiment analysis results, for example adding a brief explanation if the healthcare worker is tired.

[2035] Input: Sentiment analysis results, diagnostic analysis results

[2036] Output: Tailored diagnostic results report

[2037] Step 6:

[2038] The device displays the adjusted report to the medical professional for confirmation and final diagnosis.

[2039] Input: Adjusted diagnostic results report

[2040] Output: Confirmed diagnostic results report

[2041] ---

[2042] Pharmaceutical research and design support

[2043] Step 1:

[2044] Researchers input the conditions necessary for new drug design, such as active ingredients and stability conditions, into the terminal.

[2045] Input: Condition data for new drug design

[2046] Output: The input data

[2047] Step 2:

[2048] The terminal transmits the condition data to the server.

[2049] Input: Condition data for new drug design

[2050] Output: The data sent.

[2051] Step 3:

[2052] The server uses a generative AI to generate candidate molecular structures that meet the conditions.

[2053] Input: Condition data for new drug design

[2054] Output: List of candidate molecular structures

[2055] Step 4:

[2056] The device uses an emotion engine to analyze the researcher's emotional state, specifically by analyzing facial expressions and typing speed to detect stress and fatigue.

[2057] Input: Researcher's facial expression, input speed

[2058] Output: Emotion analysis results

[2059] Step 5:

[2060] The server then adjusts the list of molecular structures based on the results of the sentiment analysis, for example adding detailed annotations to complex molecular structures.

[2061] Input: Sentiment analysis results, candidate list of molecular structures

[2062] Output: Adjusted molecular structure list

[2063] Step 6:

[2064] The terminal displays the adjusted molecular structure list to the researcher and prompts them to confirm it. Specific actions include displaying and confirming the molecular structure.

[2065] Input: Adjusted molecular structure list

[2066] Output: List of confirmed molecular structures

[2067] (Application example 2)

[2068] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2069] While conventional medical operation support systems have improved the efficiency of medical personnel's work, they have not been able to integrate emotion analysis or automatic patrol functions in the security field. As a result, safety inside and outside medical facilities has not been ensured sufficiently, and the burden on medical personnel remains heavy. The present invention aims to add security functions to medical operation support systems, thereby improving safety while performing medical operations efficiently.

[2070] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes [means for receiving input medical information], [means for creating a draft of a clinical document using a generation AI based on the received medical information], and [means for displaying the created clinical document and correcting and confirming it]. This makes it possible to improve the efficiency of medical work and also to increase safety within medical facilities.

[2071] "Medical information" refers to data such as a patient's medical treatment, examination results, health status, medical history, and prescription information.

[2072] "Generative AI" is an algorithm that learns patterns from large amounts of data and automates tasks such as natural language generation, image recognition, and sentiment analysis.

[2073] "Clinical documents" are documents such as medical records, reports, and diagnosis certificates that are created based on medical information.

[2074] An "emotion engine" is a technology that analyzes and recognizes human emotions from input data.

[2075] "Security measures" are systems or devices that patrol a target area and detect suspicious behavior or unusual events.

[2076] A "patrol route" is the route that a security robot travels to check for safety.

[2077] "Emotion analysis" is the process of determining a subject's emotional state from facial expressions, voice, text, etc.

[2078] "Abnormality Report" is a function that notifies the administrator of abnormal situations detected by the system.

[2079] The present invention is an integrated system that uses generative AI and emotion engines to support healthcare professionals in their work, improve communication with patients, and enhance security both inside and outside healthcare facilities. The following describes in detail the embodiments of the present invention.

[2080] Hardware and software used

[2081] Hardware

[2082] 1. Security Robot: A robot equipped with autonomous mobility and a high-resolution camera, this robot patrols inside and outside medical facilities.

[2083] 2. Camera: The security robot will be equipped with a high-resolution camera to capture footage in real time.

[2084] 3. Computer: A computer equipped with a CPU / GPU for real-time data processing. Small computers such as Raspberry Pi are used.

[2085] software

[2086] 1. OpenCV: Image processing library. Performs image processing such as face recognition and anomaly detection.

[2087] 2. Keras: A deep learning library for running emotion recognition models using generative AI.

[2088] 3. Python: A programming language. All processing logic of the present invention is written in Python.

[2089] Specific operation of the system

[2090] 1. Receiving medical information

[2091] The server receives medical information entered by a medical professional, such as the patient's medical history and health status, which is used to create a draft clinical document.

[2092] 2. Generative AI for clinical documentation

[2093] The server analyzes the received medical information and uses a generative AI model to automatically create draft clinical documents that can be reviewed and revised by medical professionals.

[2094] 3. Emotion analysis using an emotion engine

[2095] The server uses an emotion engine to analyze the emotions of medical professionals and patients, which then fine-tunes the content of the documents created and improves the quality of communication.

[2096] 4. Automatic patrols by security robots

[2097] The security robot automatically moves along a set patrol route, checking the safety of designated areas, capturing video footage in real time with a camera and performing facial recognition and emotion analysis.

[2098] 5. Anomaly Detection and Reporting

[2099] The server analyzes the data sent from the security robot in real time to detect abnormal behavior or emotional changes. If an abnormality is detected, it is immediately reported to the administrator and appropriate countermeasures are provided using generative AI.

[2100] Specific examples

[2101] For example, if a security robot encounters a person expressing negative emotions such as "anxiety" or "fear" while patrolling, it will recognize the person's face and analyze their emotion. If an abnormality is detected, it will notify the administrator in real time and send a video of the situation. Furthermore, it will automatically update its security route and patrol areas to ensure greater safety.

[2102] Prompt Sentence Examples

[2103] You may also find the following prompt examples helpful:

[2104] Use emotion recognition models to determine the emotions of people in a security area in real time, identify suspicious behavior or emotional changes, and report them to management, including the ability to automatically adjust patrol routes.

[2105] As described above, the present invention provides an integrated system that achieves both improved efficiency in medical care and enhanced security.

[2106] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2107] Step 1:

[2108] The user inputs medical information such as the patient's medical treatment details and health condition into the terminal. The terminal then sends this information to the server. The input here is text data such as the patient's symptoms and examination results, and the server receives and stores this information.

[2109] Step 2:

[2110] The server uses a generative AI model to create a draft clinical document based on the received medical information. The generated draft includes medical records and reports based on the medical information, automating tasks that would otherwise be performed manually by medical professionals.

[2111] Step 3:

[2112] The server sends the generated draft of the clinical document to the terminal, which displays it. The user checks the displayed document and makes corrections as necessary. After corrections are made, the user confirms the document, and the terminal sends the confirmed document back to the server.

[2113] Step 4:

[2114] The server stores the confirmed clinical documents in a database for future reference and analysis. The documents are stored in text or PDF files.

[2115] Step 5:

[2116] The security robot begins patrolling automatically along a set route. The robot captures real-time video with its built-in camera and transmits it to a terminal. This video is used for facial recognition and emotion analysis.

[2117] Step 6:

[2118] The server performs facial recognition on the received video data to detect the subject's face. The detected facial image is input into the emotion engine, which analyzes the subject's emotions. The input here is a facial image, and the output is an emotional state such as "joy," "fear," or "anxiety."

[2119] Step 7:

[2120] The server detects anomalies based on the analysis results. For example, if negative emotions such as anxiety or fear are detected, the server notifies the administrator in real time. The notification includes the subject's emotional state and a link to the captured video.

[2121] Step 8:

[2122] The server automatically adjusts the patrol route based on the emotion analysis results, for example, changing the route to patrol more frequently in areas where abnormalities are detected. This configuration information is sent to the security robot, which then continues patrolling along the new route.

[2123] The above are the specific processing steps of the system that realizes the application example. The input and output, data processing and calculation contents, and specific operations at each step are explained in detail.

[2124] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2125] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2126] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2127] [Fourth embodiment]

[2128] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2129] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2130] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2131] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2132] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2133] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2134] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2135] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2136] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2137] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2138] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2139] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2140] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2141] This invention is a system that uses generative AI to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[2142] Clinical Documentation Support

[2143] explanation

[2144] This function allows medical professionals to input medical treatment details and examination results, and the generation AI automatically creates a draft of the clinical document, assisting medical professionals in the process of reviewing, correcting, and confirming the content.

[2145] Specific examples

[2146] The user (healthcare worker) inputs the details of the patient's medical treatment. For example, the user inputs data such as "The patient complains of pain in the right shoulder, and the examination revealed no fractures."

[2147] The server receives this information and uses generative AI to generate a draft clinical document stating, "The patient's main complaint is pain in the right shoulder, and examination revealed no fractures."

[2148] The terminal displays the generated document, and the user checks it, makes corrections as necessary, and finalizes it as the final document.

[2149] The server stores the established clinical documents in a database.

[2150] Automated medical interview system

[2151] explanation

[2152] This function allows patients to answer questions via their device, and the generative AI then suggests disease names as candidates for initial diagnosis, enabling efficient initial diagnosis.

[2153] Specific examples

[2154] The user (patient) answers the medical interview questions displayed on the terminal. For example, they might enter, "I have had a headache, fever, and cough for three days."

[2155] The server analyzes this information and generates potential disease names such as "cold, flu, COVID-19."

[2156] The device presents these potential disease names to the patient and then passes on the necessary information to medical professionals.

[2157] Diagnostic Imaging System

[2158] explanation

[2159] This feature allows medical professionals to upload medical images (e.g., X-rays, MRIs, etc.), and image recognition AI analyzes abnormalities and generates a report.

[2160] Specific examples

[2161] The user (healthcare worker) inputs the patient's chest X-ray image into the terminal.

[2162] The server analyzes the received images and uses image recognition AI to identify abnormalities (e.g., shadows in the lungs).

[2163] The server generates and displays the analysis results as a report.

[2164] The terminal displays the generated report, which the user (healthcare professional) can check.

[2165] Diagnosis system based on multiple patient information

[2166] explanation

[2167] This feature integrates data collected from multiple patients and uses generative AI to perform highly accurate diagnostic analysis.

[2168] Specific examples

[2169] The user (healthcare professional) inputs the medical records, vital data, and diagnostic imaging data of Patients A, B, and C into the terminal.

[2170] The server integrates this data and uses generative AI to perform a diagnostic analysis for each patient.

[2171] The server compiles diagnostic results into a report, such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having a decline in kidney function."

[2172] The terminal displays the generated report, and the user (medical professional) checks it and makes a final diagnosis.

[2173] Pharmaceutical research and design support

[2174] explanation

[2175] This feature uses generative AI to generate and predict molecular structures of new drugs, supporting the drug design process.

[2176] Specific examples

[2177] The user (researcher) inputs the conditions necessary for designing a new drug into the terminal.

[2178] The server uses generative AI to generate and predict candidate molecular structures.

[2179] The server stores the generated molecular structures in a database, allowing users (researchers) to search and use them as needed.

[2180] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI to various medical tasks.

[2181] The processing flow will be explained below.

[2182] Clinical Documentation Support

[2183] Processing Steps

[2184] Step 1:

[2185] The user (healthcare worker) uses a terminal to input details of the patient's medical treatment, including symptoms, examination results, and treatment details.

[2186] Step 2:

[2187] The device receives the entered medical information and transmits the data to the server using a secure communication protocol (e.g., HTTPS).

[2188] Step 3:

[2189] The server analyzes the received medical information and uses generative AI to generate a draft clinical document, including the diagnosis and treatment plan.

[2190] Step 4:

[2191] The server transmits the generated draft of the clinical document to the terminal.

[2192] Step 5:

[2193] The terminal displays the received draft of the clinical document on the screen, and the user (healthcare professional) can check and correct it.

[2194] Step 6:

[2195] The user inputs the corrections and presses the confirm button.

[2196] Step 7:

[2197] The terminal sends the confirmed clinical document to the server, which stores the data in a database.

[2198] Automated medical interview system

[2199] Processing Steps

[2200] Step 1:

[2201] The terminal displays medical interview questions to the patient on the screen.

[2202] Step 2:

[2203] The user (patient) answers questions and inputs their symptoms and physical condition.

[2204] Step 3:

[2205] The terminal transmits the input answer to the server.

[2206] Step 4:

[2207] The server analyzes the received responses and compares them with a medical database to generate relevant disease name candidates.

[2208] Step 5:

[2209] The server sends the generated disease name candidates to the terminal.

[2210] Step 6:

[2211] The terminal displays possible disease names to the patient and then provides the necessary information to medical professionals.

[2212] Diagnostic Imaging System

[2213] Processing Steps

[2214] Step 1:

[2215] The user (medical professional) inputs medical images (e.g., X-rays, MRIs, etc.) into the terminal.

[2216] Step 2:

[2217] The terminal transmits the input image data to the server.

[2218] Step 3:

[2219] The server analyzes the received image data and identifies any abnormalities using image recognition AI.

[2220] Step 4:

[2221] The server generates a report based on the analysis results.

[2222] Step 5:

[2223] The server sends the generated report to the terminal.

[2224] Step 6:

[2225] The terminal displays the report, which the user (healthcare professional) can review.

[2226] Diagnosis system based on multiple patient information

[2227] Processing Steps

[2228] Step 1:

[2229] The user (healthcare professional) inputs multiple patient data (vital signs, medical record information, diagnostic imaging data, etc.) into the terminal.

[2230] Step 2:

[2231] The terminal transmits the input data to the server.

[2232] Step 3:

[2233] The server integrates the received data and performs diagnostic analysis using generative AI.

[2234] Step 4:

[2235] The server generates a diagnostic report based on the analysis results.

[2236] Step 5:

[2237] The server transmits the generated diagnostic report to the terminal.

[2238] Step 6:

[2239] The terminal displays the diagnostic report, and the user (medical professional) checks it and makes a final diagnosis.

[2240] Pharmaceutical research and design support

[2241] Processing Steps

[2242] Step 1:

[2243] The user (researcher) inputs the conditions necessary for new drug design into the terminal.

[2244] Step 2:

[2245] The terminal transmits the input conditions to the server.

[2246] Step 3:

[2247] The server uses generative AI to generate and predict candidate molecular structures.

[2248] Step 4:

[2249] The server stores the generated molecular structures in a database.

[2250] Step 5:

[2251] The user (researcher) searches for the information they need and displays it on their terminal.

[2252] Example 1

[2253] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2254] The purpose of this invention is to reduce the wide-ranging workload of medical professionals and improve efficiency in the medical field. Conventional systems require a lot of manual work, such as document creation, medical interviews, image diagnosis, diagnostic analysis of multiple patients, and pharmaceutical research, which is time-consuming and labor-intensive, and also has a high risk of human error. This has resulted in an excessive workload for medical professionals, making it difficult to provide prompt and accurate medical care.

[2255] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2256] In this invention, the server includes: [means for receiving input medical data;] [means for creating a draft of a clinical document using a generative AI model based on the received medical data;] [means for displaying the created clinical document and correcting and confirming it; and] [means for saving the confirmed clinical document in a database. This automates document creation work in medical settings, reducing time and effort and human error.

[2257] "Medical data" refers to a wide range of data related to medical care, such as patient medical information, vital signs, diagnostic imaging data, and medical records.

[2258] A "generative AI model" refers to an artificial intelligence algorithm that uses technologies such as natural language processing and image recognition to generate output tailored to a specific purpose from input data.

[2259] "Clinical documents" refer to important documents in the medical field that contain information such as medical treatment details and examination results.

[2260] "Medical interview items" refer to questions presented to understand the patient's symptoms, medical history, and current health condition.

[2261] "Candidate disease names" refers to a list of possible disease names generated based on medical interview responses and other medical data.

[2262] "Medical images" refers to medical image data such as X-rays, MRIs, and CT scans.

[2263] "Image analysis AI" refers to artificial intelligence algorithms that analyze medical images and identify or diagnose abnormalities.

[2264] "Patient data" refers to all medical information related to individual patients, including medical information, vital signs, and diagnostic imaging data.

[2265] "Diagnosis result" refers to a diagnostic conclusion generated through analysis based on medical data and patient data.

[2266] "Drug design criteria" refers to the scientific and clinical requirements and specifications for designing new drugs.

[2267] "Molecular structure" refers to the internal structure of a drug or other compound, represented as a chemical formula or 3D model.

[2268] A "report" is a document that documents the results of analysis or diagnosis and is compiled in a format that can be reviewed and used by medical professionals.

[2269] "Database" refers to a computer system for systematically storing and managing information such as medical data, clinical documents, and molecular structures.

[2270] This invention is a system that uses generative AI models to support the work of medical professionals and improve efficiency in the medical field. This system integrates the following functions: clinical document creation support, automated medical interview system, image diagnosis system, diagnosis system using multiple patient information, and pharmaceutical research and design support.

[2271] Hardware and software used

[2272] The system's main hardware consists of a server and a terminal. The server receives and analyzes data, runs the generative AI model, and stores the data. The terminal provides an interface with the user, allowing data input and display of results.

[2273] Specific software includes:

[2274] Generative AI models: Natural language processing models such as OpenAI's GPT-4

[2275] Image analysis AI: Image recognition services such as Google Cloud Vision and AWS Rekognition

[2276] Database: A data management system such as Firebase or PostgreSQL

[2277] Clinical Documentation Support

[2278] The user (healthcare professional) inputs the details of the medical treatment from the terminal. For example, they input data such as "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal then sends the input details of the medical treatment to the server. The server inputs the received data as prompts into the generative AI model, generating a draft clinical document stating, "The patient complained of pain in the right shoulder, and examination revealed no fractures." The terminal displays the generated document, and the user can review it, make corrections as necessary, and finalize it as the final document. The server then saves the finalized clinical document in a database.

[2279] Example prompt: "Please use this clinical content to draft a clinical document."

[2280] Automated medical interview system

[2281] The user (patient) answers the medical interview questions displayed on the device. For example, they might enter, "I have had a headache, fever, and cough for three days." The device then sends the entered medical interview data to the server. The server then inputs the received data as prompts into the generative AI model, generating disease name candidates such as "cold, flu, COVID-19." The device then presents these disease name candidates to the patient and then conveys the necessary information to medical professionals.

[2282] Example prompt: "Please list possible diseases based on this interview data."

[2283] Diagnostic Imaging System

[2284] The user (healthcare professional) uploads an image file (e.g., a chest X-ray image) to the device. The device sends the image file to the server. The server inputs the received image into the image analysis AI and identifies abnormal areas (e.g., shadows in the lungs). The image analysis AI generates a report of the analysis results. The server sends the generated report to the device, which displays the report and the user (healthcare professional) confirms it.

[2285] Example prompt: "Analyze this image to identify anomalies and generate a report."

[2286] Diagnosis system based on multiple patient information

[2287] The user (healthcare professional) inputs multiple patient information (e.g., medical record information, vital signs, diagnostic imaging data) into the terminal. The terminal sends all input data to the server. The server integrates the received data and inputs it as prompts into the generative AI model, performing diagnostic analysis for each patient. For example, it generates a diagnosis result such as "Patient A is at high risk of myocardial infarction. Patient B is suspected of having pneumonia. Patient C is suspected of having decreased renal function." The server then sends the generated diagnosis result as a report to the terminal, which displays the report and the user confirms it.

[2288] Example prompt: "Generate a diagnosis based on these patient data."

[2289] Pharmaceutical research and design support

[2290] The user (researcher) inputs the conditions necessary for designing a new drug into a terminal. The terminal then sends the input design conditions to the server. The server inputs the received conditions as prompts into the generative AI model, generating and predicting candidate molecular structures. The server stores the generated molecular structures in a database, allowing users to search and use them as needed.

[2291] Example prompt: "Generate molecular structure candidates based on these conditions."

[2292] As described above, the present invention reduces the burden on medical professionals and improves efficiency in the medical field by applying generative AI models to various medical tasks.

[2293] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2294] Clinical Documentation Support

[2295] Step 1: The user inputs the details of the medical treatment from the terminal.

[2296] Specifically, the user uses a keyboard or voice input to input, "The patient complains of pain in the right shoulder, and examination reveals no fractures."

[2297] Step 2: The terminal sends the entered medical details to the server.

[2298] The input is the medical data, and the output is the transmitted data. HTTPS is used as the communication protocol.

[2299] Step 3: The server inputs the received data into the generative AI model as prompts.

[2300] The input is medical content data, and the output is a prompt: "Please use this medical content to draft a clinical document." The generative AI model analyzes the received data and generates an appropriate draft clinical document.

[2301] Step 4: The generative AI model returns the generated draft clinical document to the server.

[2302] The input is the analysis result data, and the output is the generated draft clinical document.

[2303] Step 5: The server sends the generated draft to the device.

[2304] The input is the draft data, and the output is the submitted draft data.

[2305] Step 6: The terminal displays the generated document, which the user can review and modify if necessary.

[2306] As a specific operation, the user uses a text editor to edit a document on the screen of the terminal.

[2307] Step 7: The user finalizes the document.

[2308] The input is the corrected document and the output is the finalized document.

[2309] Step 8: The terminal sends the confirmed document to the server.

[2310] The input is the finalized document data, and the output is the transmitted data.

[2311] Step 9: The server saves the finalized document to the database.

[2312] The input is the finalized document data, and the output is the data stored in the database.

[2313] Automated medical interview system

[2314] Step 1: The user (patient) answers the medical interview questions displayed on the terminal.

[2315] Specifically, in response to the question, enter "I have had a headache, fever, and cough for three days" in the text box.

[2316] Step 2: The terminal transmits the entered medical interview data to the server.

[2317] The input is the medical interview response data, and the output is the transmitted data.

[2318] Step 3: The server inputs the received data into the generative AI model as prompts.

[2319] The input is the patient's interview response data, and the output is a prompt statement: "Please list possible disease names based on this interview data."

[2320] Step 4: The generative AI model generates disease name candidates based on the received data.

[2321] The input is medical interview response data, and the output is disease name candidate data.

[2322] Step 5: The server sends the generated disease name candidates to the terminal.

[2323] The input is the disease name candidate data, and the output is the transmitted data.

[2324] Step 6: The terminal presents the patient with possible disease names and then conveys the necessary information to the medical professional.

[2325] Specifically, the system displays possible disease names on the screen, such as "cold, influenza, COVID-19," and the user confirms them.

[2326] Diagnostic Imaging System

[2327] Step 1: The user (healthcare professional) uploads an image file to the device.

[2328] As a specific operation, a file selection dialog is used to select a chest X-ray image.

[2329] Step 2: The device sends the image file to the server.

[2330] The input is the image file and the output is the transmitted data.

[2331] Step 3: The server inputs the received image into the image analysis AI.

[2332] The input is an image file, and the output is a prompt: "Analyze this image to identify any abnormalities and generate a report."

[2333] Step 4: Image analysis AI analyzes the pixel data of the image and identifies abnormalities.

[2334] The input is an image file, and the output is the analysis result data.

[2335] Step 5: The image analysis AI generates a report of the analysis results.

[2336] The input is analysis result data, and the output is report data.

[2337] Step 6: The server sends the generated report to the terminal.

[2338] The input is the report data and the output is the transmitted data.

[2339] Step 7: The terminal displays the report and the user (healthcare professional) confirms it.

[2340] Specifically, the generated report is displayed on the screen and checked by a medical professional.

[2341] Diagnosis system based on multiple patient information

[2342] Step 1: A user (healthcare worker) inputs multiple patient information into a terminal.

[2343] Specifically, the patient enters medical record information and vital data into a form.

[2344] Step 2: The device sends all entered data to the server.

[2345] The input is a plurality of patient information data, and the output is the transmitted data.

[2346] Step 3: The server consolidates the received data.

[2347] The input is multiple patient information data, and the output is the integrated data.

[2348] Step 4: The server inputs the integrated data as prompts into the generative AI model.

[2349] The input is the integrated data, and the output is the prompt statement "Please generate a diagnosis based on these patient data."

[2350] Step 5: The generative AI model generates a diagnosis based on the integrated data.

[2351] The input is the integrated data, and the output is the diagnostic result data.

[2352] Step 6: The server sends the generated diagnostic results to the terminal as a report.

[2353] The input is the diagnostic result data, and the output is the transmitted data.

[2354] Step 7: The terminal displays the report, which the user confirms and makes a final diagnosis.

[2355] Specifically, the report is displayed and a medical professional makes a final diagnosis.

[2356] Pharmaceutical research and design support

[2357] Step 1: The user (researcher) inputs the design conditions for a new drug into the terminal.

[2358] As a specific operation, the design conditions are input into the text box.

[2359] Step 2: The terminal transmits the input design conditions to the server.

[2360] The input is the design condition data, and the output is the transmitted data.

[2361] Step 3: The server inputs the received conditions into the generative AI model as prompts.

[2362] The input is design condition data, and the output is a prompt statement: "Generate molecular structure candidates based on these conditions."

[2363] Step 4: The generative AI model generates candidate molecular structures based on the conditions.

[2364] The input is design condition data, and the output is molecular structure candidate data.

[2365] Step 5: The server stores the generated molecular structure in a database.

[2366] The input is molecular structure candidate data, and the output is the saved data.

[2367] Step 6: The user searches for and uses molecular structures from the terminal as needed.

[2368] Specifically, the molecular structure search interface is used to search and evaluate stored molecular structures.

[2369] The specific processing steps for each function have been described above.

[2370] (Application example 1)

[2371] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2372] Conventional medical support systems require medical personnel to be present at all times, making it difficult to respond quickly in emergencies. In particular, emergency medical support systems using mobile means such as autonomous vehicles have difficulty providing first aid and diagnosis when medical personnel are not present on board, making it impossible to provide appropriate medical support.

[2373] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2374] In this invention, the server includes a means for receiving input medical information, a means for creating a draft clinical document using a generative artificial intelligence based on the received medical information, a means for displaying, correcting, and confirming the created clinical document, a means for inputting patient symptoms in an emergency and starting processing, and a means for using the generative artificial intelligence to create instructions for first aid in real time based on the input symptoms. This enables the autonomous vehicle to provide prompt and appropriate first aid and safely transport the patient even when medical personnel are not on-site.

[2375] "Medical information" refers to all data related to patients, such as medical treatment details, symptoms, examination results, and medical images.

[2376] "Generative AI" is an AI technology that automatically generates natural language sentences and reports from given data.

[2377] "Clinical documents" are official documents for medical professionals that contain information such as examination results and treatment plans.

[2378] "Emergency" refers to situations where medical personnel are not available or where immediate first aid is required.

[2379] "Real-time first aid instructions" are specific instructions for performing emergency medical procedures that are instantly generated using signal processing and generative AI.

[2380] A "medical history" is a series of questions asked to understand a patient's symptoms and medical history.

[2381] A "medical database" is a system that accumulates and manages medical information and case data.

[2382] "Medical images" are image data obtained using medical equipment such as X-rays and MRIs.

[2383] "Image recognition artificial intelligence" is an artificial intelligence technology used to analyze medical images and identify abnormalities.

[2384] "Instructions for first aid based on diagnostic results" are specific treatment procedures and treatment details created by the generative artificial intelligence based on the diagnostic results.

[2385]

[2386] This invention provides a concrete implementation of a generative AI-based emergency medical support system to be installed in an autonomous vehicle. This system enables rapid first aid in emergencies and is primarily composed of the following main components:

[2387] 1. Server:

[2388] The server is the central system that performs various processes using generative AI models. The server is responsible for the following processes:

[2389] Receiving and generating medical information. Drafting clinical documentation using AI.

[2390] Generate emergency first aid instructions based on patient symptom information.

[2391] An initial diagnosis is made based on the results of the interview, after comparing them with a medical database.

[2392] The server software includes generative AI models using TensorFlow and Keras, and a database management system.

[2393] 2. Terminal:

[2394] The terminal is a device such as a smartphone or smart glasses that provides an interface for patients and medical professionals to input and view information. The terminal has the following functions:

[2395] The user (patient or healthcare professional) enters symptoms and medical questionnaire answers.

[2396] Displays generated documents and diagnostic results from the server.

[2397] Displays emergency procedure instructions as needed.

[2398] For example, a patient may type into their smartphone, "I have severe chest pain and difficulty breathing." The device can also automatically input information using voice commands.

[2399] 3. User:

[2400] The user is a patient or a medical professional who uses the system. The user performs the following operations:

[2401] Enter your symptoms in case of an emergency.

[2402] Respond based on emergency instructions from the server.

[2403] Review and correct clinical documentation as needed.

[2404] The user can utilize the system through an intuitive interface, maximizing the effectiveness of the invention.

[2405] The following are examples of prompt sentences:

[2406] Example prompt sentence:

[2407] "The patient is experiencing chest pain and difficulty breathing. Urgent treatment is needed. What are the possible diagnoses? What first aid would be needed?"

[2408] Specific examples of hardware include self-driving vehicles, smartphones, smart glasses, and IoT medical devices (e.g., robotic arms, first aid kits).

[2409] Specific examples of software include generative AI models (TensorFlow / Keras models), real-time image analysis software, and control software for IoT devices.

[2410] In this way, the present invention realizes a system that can provide prompt and appropriate first aid in an autonomous vehicle even when medical personnel are not present.

[2411] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2412] Step 1:

[2413] The user inputs symptoms into a smartphone or smart glasses. For example, if the user inputs "severe chest pain and difficulty breathing," the device receives this input. The input data includes the patient's symptoms and condition.

[2414] Step 2:

[2415] The device sends the symptom information it receives to the server. At this time, the symptom information is transferred to the server as text data. The server receives this text data and inputs it into the generative AI model.

[2416] Step 3:

[2417] The server uses generative AI to create a draft clinical document from the received symptom information. The generative AI model generates an initial diagnosis based on the input data. Here, the prompt used is "The patient complains of severe chest pain and difficulty breathing. What are some possible diagnosis candidates?...

Claims

1. means for receiving input medical information; A means for creating a draft clinical document using generative AI based on the received medical information; means for displaying, correcting and finalizing the generated clinical documentation; a means for storing the confirmed clinical documentation in a database; A system including:

2. means for presenting medical history questions to the patient and receiving responses from the patient; means for matching the received responses with a medical database to generate relevant disease name candidates; A means to present the generated disease name candidates and hand them over to medical professionals; The system of claim 1 , comprising:

3. A method for inputting medical images and analyzing abnormalities using image recognition AI, means for generating and displaying the analysis results as a report; The system of claim 1 , comprising:

4. A means of integrating data collected from multiple patients and performing diagnostic analysis using generative AI; means for generating and displaying the diagnostic analysis results as a report; The system of claim 1 , comprising:

5. A means for generating and predicting molecular structures using generative AI for drug design; a means for storing the generated molecular structures in a database and making them searchable; The system of claim 1 , comprising:

Images