system

Abstract

We provide the system. [Solution] A means of collecting treatment data and test data from each medical institution in an anonymized form, A means for storing collected treatment data and test data in a database, A means for training a generative artificial intelligence model using treatment data and examination data stored in a database, A means for calculating a diagnostic result for a disease condition using a generative artificial intelligence model, A system that includes means for displaying diagnostic results corresponding to the patient's condition on a terminal.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the current electronic medical record system, although the diagnosis template is standardized, there is a problem that it lacks flexibility and individuality for each patient's situation. Also, in the medical field, it is necessary to quickly and accurately understand new medical research and papers and reflect that knowledge in medical treatment, but in reality, it often takes a lot of time. Therefore, a new approach is needed to improve the accuracy of diagnosis and for appropriate treatment.

Means for Solving the Problems

[0005] This invention provides a system that collects anonymized treatment and examination data from various medical institutions and stores this data in a database. Subsequently, by training the database using a generative artificial intelligence model, it calculates accurate and personalized diagnostic results for medical conditions. Furthermore, by regularly adding new research paper data and retraining the model, it continuously utilizes the latest medical knowledge to achieve more accurate diagnoses. In addition, by providing a user interface that allows medical staff to easily input clinical data and quickly and accurately confirm diagnostic results, it enables appropriate treatment and drug administration.

[0006] "Medical institutions" is a general term for facilities that provide medical services, such as hospitals, clinics, and medical offices.

[0007] "Treatment data" refers to detailed information about medical procedures, treatments, and drug administrations performed on a patient.

[0008] "Test data" refers to the results and measurements of various tests performed to evaluate a patient's health status.

[0009] "Anonymization" refers to the process of removing personally identifiable information from data, thereby processing it in a way that prevents personal information from being identified.

[0010] A "database" refers to an information system used to systematically organize and store collected information.

[0011] A "generative artificial intelligence model" refers to an artificial intelligence system that uses technologies such as machine learning and deep learning to learn patterns from vast amounts of data and provide generated results.

[0012] "Learning" refers to the process by which artificial intelligence models use large amounts of data to analyze patterns and relationships, thereby improving their ability to predict the future and make decisions.

[0013] "Diagnosis results" refer to information that shows an evaluation or hypothesis of the patient's condition, output by a generative artificial intelligence model based on the patient's symptoms, test data, etc.

[0014] "Terminal" refers to devices such as computers, tablets, and smartphones that medical staff use to input information.

[0015] "User interface" refers to the interface, including screens and input forms, that users use to interact with and operate a system. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] Modes for carrying out the invention

[0038] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to learn from this data, and provides accurate diagnostic results for each medical condition. This system allows medical staff to quickly and accurately obtain personalized diagnostic information for each patient, enabling appropriate treatment and drug administration.

[0039] Server program processing

[0040] 1. Data collection and anonymization

[0041] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information (name, address, contact information, etc.) is removed and anonymized.

[0042] Specific example: Obtain "Patient X's blood test results" and "blood pressure measurement results" from Hospital A's server, and anonymize this data by removing names and addresses.

[0043] 2. Storing in a database

[0044] The server categorizes the anonymized data (diagnostic data, test data, treatment data, etc.) and stores it in a database. This database is designed to allow high-speed access to large amounts of data.

[0045] Specific example: Store blood test results and blood pressure measurement results in the "Test Data" table of the database.

[0046] 3. Training of generative artificial intelligence models

[0047] The server periodically uses database information to train generative artificial intelligence models. This improves the accuracy of disease diagnosis based on the latest treatment and examination data.

[0048] Specific example: Retrain a generative artificial intelligence model using newly added blood test data to improve the accuracy of diabetes diagnosis.

[0049] 4. Integration with research paper databases

[0050] The server periodically retrieves the latest medical papers from domestic and international journal databases, machine-translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[0051] Specific example: Obtain the latest papers on heart disease from PubMed and translate them into Japanese using natural language processing technology. Then, add the analysis results to the model and retrain it.

[0052] Terminal program processing

[0053] 1. Providing a user interface

[0054] The terminal provides a user interface that allows medical staff to easily input data and review diagnostic results. This interface is designed to be intuitive to use.

[0055] Specific example: Provide an input field for doctors to enter patient blood pressure measurement results and a dashboard to display the diagnostic results.

[0056] 2. Display of diagnostic results

[0057] The terminal displays a diagnostic result calculated by a generative artificial intelligence model based on the entered medical data. Medical staff then decide on a treatment plan based on this result.

[0058] Specific example: When a doctor inputs a patient's symptoms, a generative artificial intelligence model determines that "influenza is highly likely" and displays a list of appropriate treatments.

[0059] User program processing

[0060] 1. Data entry

[0061] The medical staff, acting as users, input patient medical data into the terminal. The entered data is sent to the server, and processing begins.

[0062] Specific example: A nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form.

[0063] 2. Input of feedback

[0064] Medical staff, acting as users, input feedback on the treatments they administered after the diagnosis and their effectiveness, and send this feedback to the server. This submitted feedback data is then used again as training data for a generative artificial intelligence model.

[0065] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[0066] In this way, this system dramatically improves diagnostic accuracy in medical settings and enables the provision of optimal treatment plans for each patient.

[0067] The following describes the processing flow.

[0068] Program processing flow

[0069] Server program processing

[0070] Step 1: Data Collection and Anonymization

[0071] The server establishes a secure communication protocol (e.g., HTTPS) with each medical institution and periodically requests treatment and test data.

[0072] Example: A server sends a request to a hospital server via an API to retrieve patient test results and treatment records.

[0073] Step 2: Data reception and verification

[0074] The server analyzes the received data and automatically checks for format consistency and the presence of required fields.

[0075] Example: Verify that JSON data has the correct structure and log any errors in the error log.

[0076] Step 3: Data anonymization

[0077] The server removes personal information from incoming data, anonymizing it to protect patient privacy.

[0078] Example: Remove patient names, addresses, and contact information from the data, and randomly generate identifiers.

[0079] Step 4: Data classification and storage

[0080] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[0081] Example: Use an SQL statement to insert the test results into the test data table.

[0082] Step 5: Training a generative artificial intelligence model

[0083] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[0084] Example: Export data in CSV format and input it into a machine learning algorithm to train a neural network.

[0085] Step 6: Integration with the research paper database

[0086] The server periodically retrieves new papers from external academic databases and performs translation and text analysis.

[0087] Example: Set up a scheduled job to download papers from PubMed via API and then process them for machine translation.

[0088] Step 7: Retrain the model

[0089] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[0090] Example: Retrain the model using a dataset that includes the latest research paper data.

[0091] Terminal program processing

[0092] Step 1: Provide the user interface

[0093] The terminal provides input forms and dashboards that medical staff can easily operate.

[0094] Example: Develop a test result input form and a diagnostic results dashboard using the React framework.

[0095] Step 2: Enter diagnostic data

[0096] The medical staff, who are the users, enter the patient's medical data (symptoms, test results, etc.) into the input form on the terminal.

[0097] Example: A doctor enters the patient's temperature, blood pressure, and pulse rate into the input fields.

[0098] Step 3: Submit a diagnostic request

[0099] The terminal sends the entered data to the server and makes a diagnostic request.

[0100] Example: By pressing the submit button, the input data is sent to the server via the HTTPS protocol.

[0101] Step 4: Displaying the diagnostic results

[0102] The server uses a generative artificial intelligence model to calculate the diagnostic results and sends them back to the terminal.

[0103] Example: A generative artificial intelligence model calculates the "likelihood of influenza" and sends the result to the terminal in JSON format.

[0104] Step 5: Confirm and display the diagnostic results.

[0105] The terminal displays the diagnostic results returned from the server on the user interface.

[0106] Example: Display the diagnostic results on the dashboard so that doctors can review them.

[0107] User program processing

[0108] Step 1: Entering medical data

[0109] The medical staff, acting as users, enter the patient's medical data into the input fields on the terminal and then transmit it.

[0110] Example: A nurse copies test results obtained from the electronic medical record system into an input form.

[0111] Step 2: Confirmation of diagnostic results

[0112] The medical staff, who are the users, review the diagnostic results displayed on the terminal and decide on a treatment plan based on those results.

[0113] Example: The doctor checks the displayed diagnosis result, "Possible influenza," and prescribes antiviral medication.

[0114] Step 3: Entering Feedback

[0115] The medical staff, who are the users, input post-treatment feedback information into their terminals and send it to the server.

[0116] Example: As follow-up data after treatment, the patient's condition improvement and the presence or absence of side effects are entered and sent to the database.

[0117] This series of processing steps allows the system to improve diagnostic accuracy in medical settings and provide personalized treatment quickly.

[0118] (Example 1)

[0119] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0120] Conventional medical systems have been insufficient in anonymizing treatment and test data collected from each medical institution, and in improving diagnostic accuracy using generative artificial intelligence models. Furthermore, the lack of retraining utilizing the latest medical papers and feedback information has made it difficult to improve the accuracy of diagnostic results and provide prompt treatment.

[0121] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0122] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model using the treatment data and test data stored in the database; means for periodically acquiring the latest medical papers, machine translating and analyzing them; means for inputting the translated paper data into the generative artificial intelligence model for retraining; and means for medical staff to input feedback on treatment effectiveness and transmit it to the server. As a result, the generative artificial intelligence model continuously retrains using the collected data and the latest medical knowledge, enabling improved diagnostic accuracy and rapid treatment.

[0123] "Treatment data" refers to a series of pieces of information about the treatments provided to a patient at a medical institution, the medications prescribed, and the treatment outcomes.

[0124] "Test data" refers to information showing the results of various medical tests (blood tests, imaging tests, urine tests, etc.) performed on a patient.

[0125] "Anonymization" is the process of removing a patient's personal information (such as name, address, and contact information) so that a specific individual cannot be identified.

[0126] A "database" is a data aggregation system that efficiently stores collected treatment and examination data and manages it so that it can be quickly accessed as needed.

[0127] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates appropriate diagnostic results or predictions for new data.

[0128] A "user interface" is a software component that provides a visual screen layout and input methods for medical staff to operate a system.

[0129] "Diagnosis" refers to a professional evaluation and opinion regarding a patient's medical condition and symptoms, based on the analysis of collected data and generative artificial intelligence models.

[0130] "Feedback" refers to information that medical staff input regarding diagnostic results and treatment effectiveness, and is used to improve the accuracy and retrain generative artificial intelligence models.

[0131] "Paper data" refers to the latest medical information published in academic papers and research results, and is used as data for training generative artificial intelligence models.

[0132] "Machine translation" is a technology that automatically converts text written in one language into another language using a computer.

[0133] This invention relates to a system that collects anonymized treatment and examination data from various medical institutions, stores this data in a database, and then uses a generative artificial intelligence model to train the database and calculate a diagnosis for a patient's condition. This system enables medical staff to make quick and accurate diagnoses. Furthermore, the system aims to further improve diagnostic accuracy by retraining the model based on the latest medical papers and feedback information.

[0134] Server program processing

[0135] 1. Data collection and anonymization

[0136] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information is removed and the data is anonymized. For example, the server sends an API request at 2 AM every day to request new data, analyzes the received data, and removes personal information. A dedicated data analysis engine is used for this process.

[0137] 2. Storing in a database

[0138] The server categorizes the anonymized data and stores it in the corresponding tables. The database is designed to allow high-speed access to large amounts of data, and database types such as MySQL are commonly used.

[0139] 3. Training of generative artificial intelligence models

[0140] The server uses periodically collected data to train a generative artificial intelligence model. For example, it checks the training schedule every Sunday night and retrains the model with new data. Machine learning frameworks such as TensorFlow are used in this process.

[0141] 4. Integration with research paper databases

[0142] The server regularly retrieves the latest medical papers from domestic and international research paper databases, machine-translates them, and analyzes them. For example, once a month, it uses the PubMed API to retrieve the latest papers on heart disease, translates them using the Google Translate API, analyzes them with a natural language processing engine, and inputs the results into a model.

[0143] Terminal program processing

[0144] 1. Providing a user interface

[0145] The terminal provides a user interface for medical staff to operate. This interface is intuitive and includes input fields such as blood pressure measurement results and a dashboard that displays diagnostic results.

[0146] 2. Display of diagnostic results

[0147] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. For example, when a doctor enters a patient's symptoms, the terminal sends a request to the server and displays a diagnostic result such as "high probability of influenza."

[0148] User program processing

[0149] 1. Data entry

[0150] Medical staff, who are the users, input patient medical data through a terminal. For example, a nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form, and the terminal transmits this information to the server in real time.

[0151] 2. Input of feedback

[0152] Medical staff, acting as users, input feedback on the effectiveness of treatments based on diagnostic results and patient responses. For example, doctors input feedback on the efficacy of prescribed medications and send it from their terminals to the server. The collected feedback is then used to retrain generative artificial intelligence models.

[0153] Specific example

[0154] For example, the following prompt statement may be used:

[0155] "Please enter the patient's blood pressure measurement results."

[0156] "The generative artificial intelligence model has determined that there is a possibility of diabetes."

[0157] This system can support improved diagnostic accuracy and faster decision-making regarding treatment plans in medical settings.

[0158] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0159] Step 1: Data Collection and Anonymization

[0160] The server periodically collects treatment and test data from each medical institution. Specifically, the server sends an API request at 2:00 AM every day to request new data. The data received from the medical institution's server is sent in JSON format. The server analyzes this data, removes personal information such as names, addresses, and contact information, and anonymizes it. The input is treatment and test data in JSON format from the medical institution, and the output is anonymized data from which personal information has been removed. The anonymized data is temporarily stored in storage.

[0161] Step 2: Storing in the database

[0162] The server reads anonymized data from temporary storage and categorizes it. Specifically, test data such as blood test results and blood pressure measurements are stored in the "Test Data" table, and treatment data such as treatment history and prescription information is stored in the "Treatment Data" table. This database uses MySQL or similar, enabling high-speed access. The input is anonymized data, and the output is structured data stored in the database. The server logs that the data has been correctly stored in the database.

[0163] Step 3: Training a generative artificial intelligence model

[0164] The server periodically retrieves new data from the database and retrains the generative artificial intelligence model. Specifically, it checks the training schedule at 3 AM every Sunday and retrains the model using data collected over the past week. Machine learning frameworks such as TensorFlow are used for this process. The input is new treatment and examination data, and the output is the updated generative artificial intelligence model. The model's accuracy is evaluated using test data, and if the accuracy meets the criteria, it is saved as a new model.

[0165] Step 4: Integration with the paper database

[0166] The server regularly retrieves the latest medical papers from domestic and international journal databases. Specifically, on the 1st of each month, it uses the PubMed API to retrieve the latest papers on heart disease. The retrieved paper data is machine-translated into Japanese using the Google Translate API. Then, it is analyzed using natural language processing techniques and converted into a format that can be input into a generative artificial intelligence model. The input is the paper data, and the output is new data that is added to the model.

[0167] Step 5: Provide the user interface

[0168] The terminal provides a user interface that allows medical staff to input data and review diagnostic results. Specifically, it includes input fields for doctors to enter patient blood pressure measurements and symptoms, and a dashboard to display diagnostic results. Input consists of medical data entered by medical staff, while output consists of data sent to the server and diagnostic results displayed on the terminal.

[0169] Step 6: Displaying the diagnostic results

[0170] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. Specifically, when a doctor enters the patient's symptoms, the terminal sends a request to the server to retrieve the diagnostic results. As a result, the dashboard displays a diagnostic result such as "high probability of influenza" and recommended treatment methods. The input is medical data, and the output is the diagnostic result.

[0171] Step 7: Data Entry

[0172] The medical staff, acting as users, input patient medical data into the terminal. Specifically, nurses enter the temperature, blood pressure, and pulse rate of patients with fever into a dedicated form, and this data is sent to the server. The input is the patient's medical data, and the output is the data sent to the server.

[0173] Step 8: Entering Feedback

[0174] Medical staff, acting as users, input feedback on the effectiveness of treatment and patient responses based on diagnostic results. Specifically, they input information on the efficacy of medications prescribed by doctors into a dedicated form and send it from their terminal to the server. The input is treatment feedback data, and the output is the data sent to the server. This feedback data is used to retrain generative artificial intelligence models.

[0175] (Application Example 1)

[0176] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0177] Conventional diagnostic support systems often required medical staff to manually input patient symptoms and test results, resulting in lengthy diagnostic timeliness. Furthermore, diagnostic results were not displayed in real time, and diagnostic accuracy was still insufficient. Additionally, there was a lack of direct feedback from patients regarding the effectiveness of post-diagnosis treatment, hindering effective retraining. To address these issues, there was a need for real-time data sharing and rapid delivery of diagnostic results.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0179] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model on the treatment data and test data stored in the database; means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model; means for displaying the diagnosis result corresponding to the medical condition on a terminal; means for creating a virtual avatar of the patient and enabling interaction with medical staff; means for inputting the patient's symptoms and test results in real time and sharing them with medical staff; means for displaying the diagnosis result and treatment method in real time; and means for the patient to input feedback on the effectiveness of treatment after diagnosis. This enables rapid data sharing and real-time display of diagnosis results, and patient feedback is also quickly reflected in the system, thereby improving diagnostic accuracy.

[0180] A "medical institution" refers to a facility that performs medical procedures, such as a hospital, clinic, or medical practice.

[0181] "Treatment data" refers to information about a patient's treatment that medical staff record during the course of their medical care. Examples include the details of drug treatment and information about prescribed medications.

[0182] "Test data" refers to data from tests performed to check a patient's health status. Examples include the results of blood tests and X-ray examinations.

[0183] "Anonymization" is the process of removing personally identifiable information (such as name, address, and contact information) from data so that individuals cannot be identified.

[0184] A "database" is a system used to store and manage collected treatment and examination data.

[0185] A "generative artificial intelligence model" is an AI model that learns from large amounts of data and has the ability to recognize patterns.

[0186] "Medical condition" refers to the symptoms a patient is exhibiting or their current health status.

[0187] "Diagnosis result" refers to the analysis results regarding the medical condition calculated by the generative artificial intelligence model, and represents the conclusion of the diagnosis.

[0188] A "terminal" is a device used by medical staff or patients, and examples include smartphones, smart glasses, and head-mounted displays.

[0189] A "virtual avatar" is a digital representation of a patient in a virtual space.

[0190] "Real-time" refers to a state where data collection, processing, and display occur instantly.

[0191] "Feedback" refers to the process of returning information about treatment effectiveness and patient response obtained after diagnosis back to the system.

[0192] "User interface" refers to the screens and methods of operation that users use to operate a system or input data.

[0193] The overall system configuration and each processing procedure for implementing this invention are described below. The system consists of three elements: a server, a terminal, and a user.

[0194] Server program processing

[0195] 1. Data collection and anonymization

[0196] The server periodically collects treatment and test data from each medical institution. During this process, it anonymizes the data by removing patients' personal information. For example, a Python script is used to remove names and addresses.

[0197] 2. Storing in a database

[0198] The collected anonymized data is categorized into diagnostic data, test data, treatment data, etc., and stored in databases such as Amazon RDS.

[0199] 3. Training of generative artificial intelligence models

[0200] The server periodically trains a generative artificial intelligence model (using TensorFlow) based on the information in the database. This improves the accuracy of diagnostics based on the latest data.

[0201] 4. Integration with research paper databases

[0202] The server periodically retrieves the latest medical papers from domestic and international medical journal databases, and then machine-translates and analyzes them. For example, it uses the Google Translation API to perform natural language processing, and then re-inputs this analyzed data into a generative artificial intelligence model for retraining.

[0203] Terminal program processing

[0204] 1. Providing a user interface

[0205] The devices (such as smart glasses, head-mounted displays, and smartphones) provide an intuitive user interface that allows medical staff to easily input data and quickly review diagnostic results. For example, this can be achieved through cross-platform applications developed using React Native.

[0206] 2. Creating a virtual avatar for the patient

[0207] The terminal application provides a function that allows patients to create virtual avatars and have those avatars interact with medical staff. The virtual avatars are implemented using Unity or Unreal Engine.

[0208] 3. Data entry and sharing

[0209] Patient symptoms and test results are entered in real time via a terminal and sent to the server. Real-time input and sharing allow medical staff to access the data immediately.

[0210] 4. Real-time display of diagnostic results

[0211] The diagnostic results and recommended treatments calculated by the generative artificial intelligence model are displayed on the device in real time. For example, the diagnostic results are projected onto smart glasses.

[0212] 5. Feedback function

[0213] After diagnosis, patients input feedback about the effectiveness of treatment via a terminal and send it to a server. This feedback is also used to retrain the generative artificial intelligence model.

[0214] User program processing

[0215] 1. Data entry

[0216] Medical staff and patients, who are the users, input medical data and test results into the terminal. The entered data is immediately sent to the server and processing begins.

[0217] 2. Input of feedback

[0218] Medical staff and patients, acting as users, input feedback on the treatments they received after a diagnosis and their effectiveness, and send this feedback to the server. This collected feedback data is then used to retrain the AI ​​model.

[0219] Examples of prompt statements

[0220] Patient's symptoms: fever of 37.8°C, sore throat, cough

[0221] Past medical history: Influenza, allergies

[0222] ---

[0223] Body temperature measurement result: 37.8°C

[0224] Blood pressure measurement result: 120 / 80

[0225] Pulse rate: 88 bpm

[0226] ---

[0227] Questions for generative AI models:

[0228] What disease might be suspected based on these symptoms?

[0229] What treatments and medications are recommended?

[0230] By inputting such prompts, the generative artificial intelligence model provides accurate diagnostic results and optimal treatment methods.

[0231] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0232] Step 1:

[0233] The server collects treatment and test data from each medical institution. During collection, patient personal information is removed and anonymized. For example, a Python script is used to remove personally identifiable information such as names and addresses.

[0234] Input: Raw treatment and test data provided by each medical institution.

[0235] Output: Anonymized treatment and laboratory data.

[0236] Step 2:

[0237] The server stores anonymized treatment and test data in a database. Here, the data is categorized into diagnostic data, test data, treatment data, etc. Amazon RDS or a similar database is recommended.

[0238] Input: Anonymized treatment and laboratory data.

[0239] Output: Categorized data stored in the database.

[0240] Step 3:

[0241] The server periodically trains a generative artificial intelligence model using treatment and examination data stored in the database. The model is updated with the latest data using AI frameworks such as TensorFlow.

[0242] Input: Treatment data and test data stored in the database.

[0243] Output: A trained generative artificial intelligence model.

[0244] Step 4:

[0245] The server periodically retrieves the latest medical papers from domestic and international medical paper databases. This paper data is machine translated using the Google Translation API and other tools, and then analyzed using natural language processing techniques. The analysis results are then added to a generative artificial intelligence model for further training.

[0246] Input: Latest medical research paper data.

[0247] Output: A retrained generative artificial intelligence model.

[0248] Step 5:

[0249] The terminal provides medical staff with a user interface for data entry. Using this interface, medical staff can input patient symptoms and test results, and the data is sent to the server. For example, React Native can be used to enable intuitive operation.

[0250] Input: Patient symptoms and test results entered by medical staff.

[0251] Output: Medical data sent to the server.

[0252] Step 6:

[0253] The device creates a virtual avatar of the patient. Using Unity or Unreal Engine, it generates a patient avatar in a virtual space, enabling interaction with medical staff.

[0254] Input: Basic patient information and image data.

[0255] Output: A virtual avatar displayed in a virtual space.

[0256] Step 7:

[0257] The terminal shares the patient's symptoms and test results entered into it with the server in real time. This allows medical staff to access the data immediately.

[0258] Input: Patient's symptoms and test results.

[0259] Output: Data sent to the server in real time.

[0260] Step 8:

[0261] The terminal displays diagnostic results and recommended treatments calculated by a generative artificial intelligence model in real time. The diagnostic results are projected using devices such as smart glasses, and medical staff determine treatment plans based on this information.

[0262] Input: Diagnostic results calculated by a generative artificial intelligence model.

[0263] Output: Diagnostic results projected onto the terminal in real time.

[0264] Step 9:

[0265] After the diagnosis, the user inputs feedback on the treatment's effectiveness into the device and sends it to the server. This allows the generative artificial intelligence model to retrain, improving the accuracy of the diagnosis.

[0266] Input: User-submitted feedback regarding treatment effectiveness.

[0267] Output: Feedback data sent to the server.

[0268] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0269] Modes for carrying out the invention

[0270] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to train on this data, and provides accurate diagnostic results for each medical condition. By further incorporating an emotion engine that recognizes the user's emotions into this system, the method of providing diagnostic results can be adjusted based on the user's emotional state.

[0271] Server program processing

[0272] 1. Data collection and anonymization:

[0273] The server establishes secure communication protocols with each medical institution and regularly collects treatment and test data. During this process, patient personal information is removed and anonymized.

[0274] Specific example: Obtain the patient's test results and treatment records from the hospital server, delete personal information, and anonymize it.

[0275] 2. Storage in the database:

[0276] The server classifies the anonymized data into categories such as "diagnosis data", "test data", "treatment data", etc., and stores it in the database.

[0277] Specific example: Classify test data into appropriate tables and store it in the database.

[0278] 3. Training of generative AI models:

[0279] The server uses the data extracted from the database to train the generative AI model. This improves the diagnostic accuracy of the medical condition.

[0280] Specific example: Retrain the generative AI model using new treatment data to improve diagnostic accuracy.

[0281] 4. Integration with the paper database:

[0282] The server regularly obtains the latest papers from an external paper database, translates and analyzes them. The obtained data is input into the generative AI model for retraining.

[0283] Specific example: Obtain new papers on heart diseases from PubMed, perform translation and analysis using natural language processing technology, and add them to the model.

[0284] Program processing on the terminal

[0285] 1. Provision of user interface:

[0286] The terminal provides a user interface that is easy for medical staff to use. This interface is designed to be intuitively operable.

[0287] Specific example: Using the React framework, construct an input form for entering a patient's medical data and a dashboard for displaying the diagnosis results.

[0288] 2. Operation of the emotion engine:

[0289] The terminal operates the emotion engine while the user is checking the diagnosis results, and analyzes the emotional state from the user's facial expression data, voice tone, etc.

[0290] Specific example: Run facial expression analysis software using the terminal's camera and microphone to determine in real time the emotions of the user, such as whether the user is surprised or at ease.

[0291] 3. Display of diagnosis results based on emotions:

[0292] The terminal adjusts the display method of the diagnosis results based on the emotion data analyzed by the emotion engine. For example, when the user feels anxious, additional explanations or step-by-step information is provided.

[0293] Specific example: When the diagnosis results are displayed, if the user is determined to be surprised, a detailed explanation of the diagnosis results is displayed in a pop-up window.

[0294] User program processing

[0295] 1. Input of medical data:

[0296] The medical staff, who is the user, enters the patient's medical data into the input field of the terminal and sends it to the server.

[0297] Specific example: A doctor enters the symptoms of a patient's fever and cough into an input form.

[0298] 2. Confirmation of diagnosis results:

[0299] The medical staff, who is the user, checks the diagnosis results displayed on the terminal and considers the explanation to the patient based on the sentiment analysis results by the sentiment engine.

[0300] Specific example: The doctor checks the diagnosis result showing "possibility of pneumonia" presented by the generative AI model and explains it carefully to the worried patient.

[0301] 3. Input of feedback:

[0302] The medical staff, who is the user, inputs the feedback on the treatment and its effect implemented after the diagnosis result into the terminal and sends it to the server. The sent feedback data is also utilized again as the learning data of the generative AI model.

[0303] Specific example: The doctor inputs the effect of the prescribed medicine and the patient's reaction as feedback and sends it to the server.

[0304] Through this series of processing steps, this system can not only improve the diagnostic accuracy in the medical field, but also enable flexible responses based on the user's sentiment and provide a better patient experience.

[0305] The following explains the process flow.

[0306] Mode for carrying out the invention

[0307] Server program processing

[0308] Step 1: Data collection and anonymization

[0309] The server establishes a secure communication protocol with each medical institution and regularly collects treatment data and examination data. At this time, patient-specific information is removed and anonymized.

[0310] Example: Obtain the patient's examination results and treatment records from the hospital server, delete the personal information and anonymize them.

[0311] Step 2: Data reception and verification

[0312] The server analyzes the received data and verifies whether the data format is correct and consistent.

[0313] Example: Check if JSON data has the correct structure and log any problems in the error log.

[0314] Step 3: Data anonymization

[0315] The server removes personal information from the received data and performs anonymization.

[0316] Specific example: Remove patient names and addresses from data and generate a unique anonymous identifier.

[0317] Step 4: Data classification and storage

[0318] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[0319] Specific example: Classify inspection data into appropriate tables and store them in a database.

[0320] Step 5: Training a generative artificial intelligence model

[0321] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[0322] Specific example: Export data in CSV format, input it into a machine learning algorithm, and train the model.

[0323] Step 6: Integration with the research paper database

[0324] The server periodically retrieves the latest papers from external research paper databases and performs translation and analysis.

[0325] Specific example: Obtain new articles on heart disease from PubMed, translate them into Japanese using machine translation, and then analyze them.

[0326] Step 7: Retrain the model

[0327] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[0328] Specific example: Retrain the model with a dataset that includes the latest research paper data to improve diagnostic accuracy.

[0329] Terminal program processing

[0330] Step 1: Provide the user interface

[0331] The terminal provides input forms and dashboards that medical staff can easily operate.

[0332] Specific example: Using the React framework, build a medical data input form and a diagnostic results display dashboard.

[0333] Step 2: Operating the Emotion Engine

[0334] While the user is reviewing the diagnostic results, the device operates an emotion engine, analyzing facial expression data and voice tone.

[0335] Specific example: Using the device's camera and microphone, facial expression analysis software is used to determine the user's emotional state in real time.

[0336] Step 3: Entering medical data

[0337] Medical staff, who are the users, input patient medical data into their terminals and send it to the server.

[0338] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form and sends a diagnostic request to the server.

[0339] Step 4: Processing diagnostic requests

[0340] The server uses a generative artificial intelligence model based on the input medical data to calculate the diagnosis.

[0341] Specific example: A generative artificial intelligence model outputs a diagnosis result stating "high suspicion of infectious disease," and sends that result to the terminal in JSON format.

[0342] Step 5: Displaying the diagnostic results

[0343] The terminal displays the diagnostic results received from the server. Additional analysis results from the emotion engine are also displayed simultaneously.

[0344] Specific example: Along with the diagnostic results, a pop-up message or additional explanation tailored to the user's emotional state will be displayed.

[0345] User program processing

[0346] Step 1: Confirming the emotionally-based diagnostic results

[0347] The medical staff, who are the users, review the diagnostic results and the analysis results of the emotion engine, and then explain them to the patient based on that information.

[0348] Specific example: The system carefully explains to the user the diagnosis that "there is a high possibility of pneumonia" and the emotional information that the user has identified as "anxiety."

[0349] Step 2: Entering emotionally-based feedback

[0350] After the diagnosis, the medical staff, acting as users, input the patient's response and treatment results as feedback into the terminal and send it to the server.

[0351] Specific example: The user inputs information about the patient's condition and post-treatment follow-up, and sends this information, along with the results of the emotion engine's analysis indicating "satisfied," to the server.

[0352] This series of processing steps not only improves diagnostic accuracy in medical settings, but also enables flexible responses based on user emotions, thereby providing a better patient experience.

[0353] (Example 2)

[0354] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0355] Traditional medical diagnostic systems required manual collection and anonymization of treatment and test data, which was time-consuming and labor-intensive. Furthermore, there was a lack of systems capable of efficiently learning from the collected data and providing rapid and accurate diagnostic results. Additionally, the presentation of diagnostic results was not tailored to the patient's emotional state, resulting in a poor patient experience.

[0356] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, and means for analyzing the user's emotions on the terminal and adjusting the method of presenting the diagnosis result based on the emotion result. This enables efficient collection and anonymization of treatment data and examination data, rapid and accurate diagnosis, and presentation of diagnosis results that respond to the patient's emotions.

[0357] "Treatment data" refers to information about treatment provided by a medical institution, including details such as the patient's treatment plan, treatment progress, and medications used.

[0358] "Test data" refers to information that includes the results of various tests performed on a patient, and includes data such as blood tests, urine tests, and imaging diagnostics.

[0359] "Anonymization" is the process of removing or concealing information that identifies an individual, and is a means of handling data securely while protecting patient privacy.

[0360] A "database" is a system for organizing and storing collected treatment and examination data, and is a collection of information that enables efficient data retrieval and management.

[0361] A "generative artificial intelligence model" is an artificial intelligence system that uses collected and learned data to perform tasks such as prediction and classification, equipped with algorithms and data structures.

[0362] "Diagnosis results" refer to information that shows judgments and conclusions about the patient's condition calculated by a generative artificial intelligence model based on collected treatment and examination data.

[0363] A "terminal" refers to a device such as a computer or tablet operated by medical staff, used for data input and display, sentiment analysis, and other similar tasks.

[0364] "Emotional analysis" is a technology that identifies a user's emotional state from their facial expressions, voice tone, and behavior, and is a means of understanding the user's psychological state.

[0365] A "user interface" refers to the interface, including screens and input forms, that medical staff use to access and operate a system through their terminals.

[0366] This invention relates to a system that provides diagnostic results by training a generative artificial intelligence model based on treatment and examination data collected from medical institutions. This system consists of three main components: a server, a terminal, and a user.

[0367] Server Processing

[0368] The server connects to each healthcare facility using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. This data is first anonymized by removing or concealing personal information. For example, personally identifiable information such as names and addresses in the collected data is replaced with "anonymous IDs."

[0369] Anonymized data is categorized into "diagnostic data," "test data," and "treatment data," and stored in a database. This database enables efficient data management and retrieval.

[0370] Next, a generative artificial intelligence model is trained using data extracted from this database. During this training process, new treatment data is added to the training set, updating the existing model. For example, retraining the model using a backpropagation algorithm can improve the accuracy of disease diagnosis.

[0371] Furthermore, the server periodically retrieves the latest research papers from external academic databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). As a result, the paper data is also incorporated into the generative artificial intelligence model for retraining.

[0372] Terminal processing

[0373] The terminal provides a user interface that is easy for medical staff to use. For example, patient information input forms and diagnostic result dashboards are built using the React framework. Users can easily input data and check diagnostic results by simply operating the system.

[0374] When displaying diagnostic results, the device captures the user's facial expressions and voice tone in real time and analyzes the user's psychological state using emotion analysis algorithms (e.g., OpenFace or IBM Watson Tone Analyzer). Once the user's emotional state, such as surprise or anxiety, is identified, the way the diagnostic results are presented is adjusted accordingly. For example, if the user is feeling anxious, a pop-up window with additional detailed explanations will be displayed.

[0375] User processing

[0376] Medical staff, as users, enter patient symptoms and medical data into input fields on their terminals and send them to the server. For example, they might enter a patient's fever or cough into an input form and submit it.

[0377] When the diagnostic results are displayed on the terminal, the emotional analysis results are also reviewed. Medical staff explain the results to the patient based on the diagnostic results presented by the generative artificial intelligence model. In doing so, the patient's emotional state is also taken into consideration to provide a more appropriate explanation.

[0378] Feedback regarding treatment results and their effectiveness is also entered into the terminal and sent to the server. The transmitted feedback data is used to retrain generative artificial intelligence models, contributing to improved diagnostic accuracy in the future.

[0379] Example of a prompt

[0380] Use the following prompts to input data into the generative AI model:

[0381] Please retrain the diagnostic accuracy for cardiac disease using recently collected treatment data. Please also incorporate the latest research papers.

[0382] This will not only improve the accuracy of diagnoses in medical settings, but also enable flexible responses based on user emotions, thereby providing a better patient experience.

[0383] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0384] Step 1:

[0385] Data collection and anonymization

[0386] The server connects with each healthcare institution using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. It receives treatment and test data sent from healthcare institutions as input. Because this data contains patient personal information, it is first anonymized by removing personal information (name, ID, address, etc.). Specifically, it scans each field of the collected data and replaces personal information with meaningless data such as an "anonymous ID." The output is the anonymized treatment and test data.

[0387] Step 2:

[0388] Storage in database

[0389] The server categorizes anonymized data into categories such as "diagnostic data," "test data," and "treatment data." It receives anonymized treatment and test data as input. This data is then organized by category for storage in the appropriate database tables. For example, if anonymized test data is collected, it is organized and stored in the "test data" table. The output is an efficiently organized database.

[0390] Step 3:

[0391] Learning of generative artificial intelligence models

[0392] The server extracts relevant data from the database and trains a generative artificial intelligence model. It extracts "diagnostic data," "test data," and "treatment data" from the database as input. Based on this data, it uses the generative AI model to train and improve the accuracy of disease diagnosis. Specifically, it adds new treatment data to the model's training set and uses a backpropagation algorithm to update the existing model. The output is the trained generative AI model.

[0393] Step 4:

[0394] Integration with research paper databases

[0395] The server periodically retrieves the latest research papers from external research paper databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). It receives paper data retrieved from external research paper databases as input. The retrieved paper text is passed through translation and analysis tools to convert it into a format suitable for input into a generative artificial intelligence model. Specifically, it uses the PubMed API to perform keyword searches, downloads new papers, and performs translation and content analysis using natural language processing techniques. The output is the analyzed paper data used for retraining.

[0396] Step 5:

[0397] Providing a user interface

[0398] The terminal provides an interface that is easy for medical staff to use. It receives data for medical staff to input patient information as input. Using the React framework, a patient information input form and a dashboard displaying diagnostic results are built. Specifically, the input form and dashboard are designed and implemented to allow users to operate them intuitively. The output is a user-friendly interface.

[0399] Step 6:

[0400] How the emotion engine works

[0401] The device operates an emotion engine while the user is reviewing the diagnostic results, analyzing the user's facial expressions and voice tone. It receives user facial and voice data acquired using the device's camera and microphone as input. This data is analyzed using an emotion analysis algorithm (e.g., OpenFace or IBM Watson Tone Analyzer) to identify the user's emotional state. Specifically, the camera captures facial expressions, which are then analyzed in real time by the algorithm. The output provides data regarding the user's emotional state.

[0402] Step 7:

[0403] Display of emotion-based diagnostic results

[0404] The device adjusts how the diagnostic results are presented based on data analyzed by the emotion engine. It receives the analysis results from the emotion engine as input. It selects an appropriate method for displaying the diagnostic results based on the analysis results, such as when the user is feeling anxious. Specifically, if the user is determined to be anxious, a pop-up window containing a detailed explanation is displayed. The output is a display of the diagnostic results adapted to the user.

[0405] Step 8:

[0406] Entering medical data

[0407] The medical staff, acting as users, input patient symptoms and medical data into the terminal's input fields. For input, they enter medical data such as the patient's fever or cough into the input form. The terminal then sends this data to the server. Specifically, when the medical staff enters data into the input form and presses the submit button, the input data is sent to the server. The output is the medical data sent to the server.

[0408] Step 9:

[0409] Checking the diagnosis results

[0410] The medical staff, acting as users, review the diagnostic results displayed on the terminal and provide explanations to the patient, taking into account the emotional analysis results from the emotion engine. The input is the diagnostic results and emotional analysis results displayed on the terminal. Based on this data, the medical staff provides the most appropriate explanation to the patient. Specifically, they review the displayed diagnostic results and emotional analysis results, paying particular attention to explanations if the patient appears anxious. The output is the most appropriate explanation for the patient.

[0411] Step 10:

[0412] Feedback Input

[0413] Medical staff, acting as users, input feedback on treatment and its effectiveness after a diagnosis into a terminal and send it to the server. The input consists of data on treatment effectiveness and patient response entered into an input form. The terminal then sends this feedback data to the server. Specifically, when medical staff enter data into the feedback form and press the submit button, the input data is sent to the server. The output is the feedback data sent to the server. This data is then used to retrain generative artificial intelligence models, improving diagnostic accuracy.

[0414] (Application Example 2)

[0415] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0416] In modern factories, accurately diagnosing machine malfunctions and maintenance needs is difficult. Furthermore, when communicating fault diagnosis results to workers, the information provided is often difficult to understand, leading to anxiety and misunderstandings. A system is needed to solve these problems and enable efficient and safe work processes.

[0417] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, means for analyzing the user's emotions using the terminal's camera and microphone, and means for adjusting the method of presenting the diagnosis result based on the analyzed emotion data. As a result, it is possible to calculate a diagnosis result using an AI model based on collected machine operation data and failure data, and to perform emotion analysis of workers using an emotion engine, thereby providing a diagnosis result in an easy-to-understand format.

[0418] "Treatment data" refers to data from a medical institution regarding the process and methods of treatment for a patient's medical condition.

[0419] "Test data" refers to data related to the results of various tests conducted at a medical institution to diagnose a patient's medical condition.

[0420] "Anonymization" is the process of removing information that can identify an individual, so that the individual cannot be identified.

[0421] A "generative artificial intelligence model" is a type of artificial intelligence that learns from data, generates new information, and provides predictions and diagnostic results.

[0422] "Learning" is the process by which a generative artificial intelligence model uses provided data to improve its prediction accuracy and performance.

[0423] A "diagnosis result" is a judgment or conclusion about a medical condition calculated by a generative artificial intelligence model based on the provided data.

[0424] A "terminal" is a device that provides a user interface, allowing medical staff to input medical data and review diagnostic results.

[0425] "Camera and microphone" refers to devices installed in a terminal for capturing video and audio.

[0426] Analyzing "emotions" involves determining the user's emotional state based on their facial expressions and voice, using data acquired from the camera and microphone.

[0427] Adjusting the "presentation method" means changing the way the diagnostic results are displayed and the content of those results to suit the user, based on the analyzed emotional data.

[0428] "Machine operation data" refers to data regarding the operating status of various machines in a factory.

[0429] "Failure data" refers to information such as the state of a machine and error logs when it malfunctions in a factory.

[0430] System Overview

[0431] This invention is a system that collects treatment and examination data anonymized from various medical institutions, trains a generative artificial intelligence model on this data, and provides accurate diagnostic results. Furthermore, it has a function that analyzes the user's emotions using the terminal's camera and microphone, and adjusts the method of presenting the diagnostic results based on the analysis results.

[0432] Hardware and software used

[0433] The following hardware and software are used for the server and terminal:

[0434] 1. Server: Performs data collection, anonymization, storage in a database, training of generative artificial intelligence models, and calculation of diagnostic results.

[0435] 2. Camera and microphone: Capture the user's facial expressions and voice in real time and use them for emotion analysis.

[0436] 3. Database: Stores treatment data and test data.

[0437] 4. Generative artificial intelligence models: These models calculate diagnostic results based on data.

[0438] 5. Emotion Analysis Engine: Analyzes the user's emotions using data acquired from the camera and microphone.

[0439] 6. User Interface (UI): This interface handles the input of medical data, the display of diagnostic results, and the presentation of sentiment analysis results.

[0440] Specific system configuration

[0441] Data collection and anonymization

[0442] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. For example, patient test results and treatment records are retrieved from the hospital server, personal information is removed, and then the data is stored in the database.

[0443] Storage in database

[0444] The collected treatment and examination data are stored in a database on the server. This database is categorized into categories such as "diagnostic data," "examination data," and "treatment data" for efficient management. For example, vibration data and temperature data are classified into appropriate tables and stored in the database.

[0445] Training and diagnostics of generative artificial intelligence models

[0446] The server uses data extracted from the database to train a generative artificial intelligence model. This improves the accuracy of diagnosing medical conditions. Furthermore, it periodically retrieves the latest research paper data to retrain the model. For example, it translates the latest papers from PubMed using natural language processing techniques and adds the analysis results to the training.

[0447] Emotional analysis and presentation of diagnostic results

[0448] The device uses its built-in camera and microphone to analyze the user's emotional state in real time. The emotion analysis engine recognizes emotions from the user's facial expressions and voice, and adjusts how the diagnostic results are displayed. For example, if the system determines that the user is surprised when the diagnostic results are displayed, a detailed explanation will be shown in a pop-up window.

[0449] Specific examples and prompt statements

[0450] Specific example:

[0451] Machine vibration and temperature data are collected, and an AI model predicts the "possibility of bearing wear." If the emotional engine detects that the worker is startled, a pop-up will display photos of bearing wear symptoms and a video explaining the replacement procedure.

[0452] Example of a prompt:

[0453] Collect failure data, analyze it using an AI model, and display the judgment results. Also, analyze the emotions of the workers using cameras. Based on the obtained emotion data, adjust the presentation method of the judgment results.

[0454] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0455] Step 1:

[0456] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. The input consists of treatment and test data received from medical institutions, and the output is anonymized data. Specific operations include data format conversion and removal of personal information fields.

[0457] Step 2:

[0458] The server stores the collected anonymized data in a database. The input consists of anonymized treatment and examination data, and the output is data categorized and stored within the database. Specifically, the server performs insertion operations into the database tables.

[0459] Step 3:

[0460] The server trains a generative artificial intelligence model using treatment and examination data stored in a database. The input is training data extracted from the database, and the output is the trained generative AI model. Specifically, the server performs tasks such as updating model parameters and applying the learning algorithm.

[0461] Step 4:

[0462] The server uses a generative artificial intelligence model to calculate a diagnosis for a medical condition. New treatment and test data are used as input, and the output is a diagnosis. Specifically, the server inputs data into a pre-trained model and outputs the diagnosis.

[0463] Step 5:

[0464] The server displays the diagnostic results on the terminal. The input is the calculated diagnostic results, and the output is the diagnostic results displayed on the terminal screen. Specific actions include updating UI components and visualizing the data.

[0465] Step 6:

[0466] The device analyzes the user's emotions in real time using a camera and microphone. Inputs include video and audio data acquired from the camera and microphone, while output is user emotion data. Specifically, it applies facial recognition algorithms and voice analysis algorithms.

[0467] Step 7:

[0468] The device adjusts how the diagnostic results are presented based on the analyzed emotional data. Inputs include the user's emotional data and diagnostic results, and output is a display of the adjusted diagnostic results. Specific actions include changing the display method of the diagnostic results and displaying pop-ups with additional explanations.

[0469] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0470] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0471] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0472] [Second Embodiment]

[0473] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0474] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0475] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0476] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0477] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0478] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0479] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0480] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0481] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0482] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0483] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0484] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0485] Modes for carrying out the invention

[0486] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to learn from this data, and provides accurate diagnostic results for each medical condition. This system allows medical staff to quickly and accurately obtain personalized diagnostic information for each patient, enabling appropriate treatment and drug administration.

[0487] Server program processing

[0488] 1. Data collection and anonymization

[0489] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information (name, address, contact information, etc.) is removed and anonymized.

[0490] Specific example: Obtain "Patient X's blood test results" and "blood pressure measurement results" from Hospital A's server, and anonymize this data by removing names and addresses.

[0491] 2. Storing in a database

[0492] The server categorizes the anonymized data (diagnostic data, test data, treatment data, etc.) and stores it in a database. This database is designed to allow high-speed access to large amounts of data.

[0493] Specific example: Store blood test results and blood pressure measurement results in the "Test Data" table of the database.

[0494] 3. Training of generative artificial intelligence models

[0495] The server periodically uses database information to train generative artificial intelligence models. This improves the accuracy of disease diagnosis based on the latest treatment and examination data.

[0496] Specific example: Retrain a generative artificial intelligence model using newly added blood test data to improve the accuracy of diabetes diagnosis.

[0497] 4. Integration with research paper databases

[0498] The server periodically retrieves the latest medical papers from domestic and international journal databases, machine-translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[0499] Specific example: Obtain the latest papers on heart disease from PubMed and translate them into Japanese using natural language processing technology. Then, add the analysis results to the model and retrain it.

[0500] Terminal program processing

[0501] 1. Providing a user interface

[0502] The terminal provides a user interface that allows medical staff to easily input data and review diagnostic results. This interface is designed to be intuitive to use.

[0503] Specific example: Provide an input field for doctors to enter patient blood pressure measurement results and a dashboard to display the diagnostic results.

[0504] 2. Display of diagnostic results

[0505] The terminal displays a diagnostic result calculated by a generative artificial intelligence model based on the entered medical data. Medical staff then decide on a treatment plan based on this result.

[0506] Specific example: When a doctor inputs a patient's symptoms, a generative artificial intelligence model determines that "influenza is highly likely" and displays a list of appropriate treatments.

[0507] User program processing

[0508] 1. Data entry

[0509] The medical staff, acting as users, input patient medical data into the terminal. The entered data is sent to the server, and processing begins.

[0510] Specific example: A nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form.

[0511] 2. Input of feedback

[0512] Medical staff, acting as users, input feedback on the treatments they administered after the diagnosis and their effectiveness, and send this feedback to the server. This submitted feedback data is then used again as training data for a generative artificial intelligence model.

[0513] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[0514] In this way, this system dramatically improves diagnostic accuracy in medical settings and enables the provision of optimal treatment plans for each patient.

[0515] The following describes the processing flow.

[0516] Program processing flow

[0517] Server program processing

[0518] Step 1: Data Collection and Anonymization

[0519] The server establishes a secure communication protocol (e.g., HTTPS) with each medical institution and periodically requests treatment and test data.

[0520] Example: A server sends a request to a hospital server via an API to retrieve patient test results and treatment records.

[0521] Step 2: Data reception and verification

[0522] The server analyzes the received data and automatically checks for format consistency and the presence of required fields.

[0523] Example: Verify that JSON data has the correct structure and log any errors in the error log.

[0524] Step 3: Data anonymization

[0525] The server removes personal information from incoming data, anonymizing it to protect patient privacy.

[0526] Example: Remove patient names, addresses, and contact information from the data, and randomly generate identifiers.

[0527] Step 4: Data classification and storage

[0528] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[0529] Example: Use an SQL statement to insert the test results into the test data table.

[0530] Step 5: Training a generative artificial intelligence model

[0531] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[0532] Example: Export data in CSV format and input it into a machine learning algorithm to train a neural network.

[0533] Step 6: Integration with the research paper database

[0534] The server periodically retrieves new papers from external academic databases and performs translation and text analysis.

[0535] Example: Set up a scheduled job to download papers from PubMed via API and then process them for machine translation.

[0536] Step 7: Retrain the model

[0537] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[0538] Example: Retrain the model using a dataset that includes the latest research paper data.

[0539] Terminal program processing

[0540] Step 1: Provide the user interface

[0541] The terminal provides input forms and dashboards that medical staff can easily operate.

[0542] Example: Develop a test result input form and a diagnostic results dashboard using the React framework.

[0543] Step 2: Enter diagnostic data

[0544] The medical staff, who are the users, enter the patient's medical data (symptoms, test results, etc.) into the input form on the terminal.

[0545] Example: A doctor enters the patient's temperature, blood pressure, and pulse rate into the input fields.

[0546] Step 3: Submit a diagnostic request

[0547] The terminal sends the entered data to the server and makes a diagnostic request.

[0548] Example: By pressing the submit button, the input data is sent to the server via the HTTPS protocol.

[0549] Step 4: Displaying the diagnostic results

[0550] The server uses a generative artificial intelligence model to calculate the diagnostic results and sends them back to the terminal.

[0551] Example: A generative artificial intelligence model calculates the "likelihood of influenza" and sends the result to the terminal in JSON format.

[0552] Step 5: Confirm and display the diagnostic results.

[0553] The terminal displays the diagnostic results returned from the server on the user interface.

[0554] Example: Display the diagnostic results on the dashboard so that doctors can review them.

[0555] User program processing

[0556] Step 1: Entering medical data

[0557] The medical staff, acting as users, enter the patient's medical data into the input fields on the terminal and then transmit it.

[0558] Example: A nurse copies test results obtained from the electronic medical record system into an input form.

[0559] Step 2: Confirmation of diagnostic results

[0560] The medical staff, who are the users, review the diagnostic results displayed on the terminal and decide on a treatment plan based on those results.

[0561] Example: The doctor checks the displayed diagnosis result, "Possible influenza," and prescribes antiviral medication.

[0562] Step 3: Entering Feedback

[0563] The medical staff, who are the users, input post-treatment feedback information into their terminals and send it to the server.

[0564] Example: As follow-up data after treatment, the patient's condition improvement and the presence or absence of side effects are entered and sent to the database.

[0565] This series of processing steps allows the system to improve diagnostic accuracy in medical settings and provide personalized treatment quickly.

[0566] (Example 1)

[0567] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0568] Conventional medical systems have been insufficient in anonymizing treatment and test data collected from each medical institution, and in improving diagnostic accuracy using generative artificial intelligence models. Furthermore, the lack of retraining utilizing the latest medical papers and feedback information has made it difficult to improve the accuracy of diagnostic results and provide prompt treatment.

[0569] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0570] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model using the treatment data and test data stored in the database; means for periodically acquiring the latest medical papers, machine translating and analyzing them; means for inputting the translated paper data into the generative artificial intelligence model for retraining; and means for medical staff to input feedback on treatment effectiveness and transmit it to the server. As a result, the generative artificial intelligence model continuously retrains using the collected data and the latest medical knowledge, enabling improved diagnostic accuracy and rapid treatment.

[0571] "Treatment data" refers to a series of pieces of information about the treatments provided to a patient at a medical institution, the medications prescribed, and the treatment outcomes.

[0572] "Test data" refers to information showing the results of various medical tests (blood tests, imaging tests, urine tests, etc.) performed on a patient.

[0573] "Anonymization" is the process of removing a patient's personal information (such as name, address, and contact information) so that a specific individual cannot be identified.

[0574] A "database" is a data aggregation system that efficiently stores collected treatment and examination data and manages it so that it can be quickly accessed as needed.

[0575] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates appropriate diagnostic results or predictions for new data.

[0576] A "user interface" is a software component that provides a visual screen layout and input methods for medical staff to operate a system.

[0577] "Diagnosis" refers to a professional evaluation and opinion regarding a patient's medical condition and symptoms, based on the analysis of collected data and generative artificial intelligence models.

[0578] "Feedback" refers to information that medical staff input regarding diagnostic results and treatment effectiveness, and is used to improve the accuracy and retrain generative artificial intelligence models.

[0579] "Paper data" refers to the latest medical information published in academic papers and research results, and is used as data for training generative artificial intelligence models.

[0580] "Machine translation" is a technology that automatically converts text written in one language into another language using a computer.

[0581] This invention relates to a system that collects anonymized treatment and examination data from various medical institutions, stores this data in a database, and then uses a generative artificial intelligence model to train the database and calculate a diagnosis for a patient's condition. This system enables medical staff to make quick and accurate diagnoses. Furthermore, the system aims to further improve diagnostic accuracy by retraining the model based on the latest medical papers and feedback information.

[0582] Server program processing

[0583] 1. Data collection and anonymization

[0584] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information is removed and the data is anonymized. For example, the server sends an API request at 2 AM every day to request new data, analyzes the received data, and removes personal information. A dedicated data analysis engine is used for this process.

[0585] 2. Storing in a database

[0586] The server categorizes the anonymized data and stores it in the corresponding tables. The database is designed to allow high-speed access to large amounts of data, and database types such as MySQL are commonly used.

[0587] 3. Training of generative artificial intelligence models

[0588] The server uses periodically collected data to train a generative artificial intelligence model. For example, it checks the training schedule every Sunday night and retrains the model with new data. Machine learning frameworks such as TensorFlow are used in this process.

[0589] 4. Integration with research paper databases

[0590] The server regularly retrieves the latest medical papers from domestic and international research paper databases, machine-translates them, and analyzes them. For example, once a month, it uses the PubMed API to retrieve the latest papers on heart disease, translates them using the Google Translate API, analyzes them with a natural language processing engine, and inputs the results into a model.

[0591] Terminal program processing

[0592] 1. Providing a user interface

[0593] The terminal provides a user interface for medical staff to operate. This interface is intuitive and includes input fields such as blood pressure measurement results and a dashboard that displays diagnostic results.

[0594] 2. Display of diagnostic results

[0595] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. For example, when a doctor enters a patient's symptoms, the terminal sends a request to the server and displays a diagnostic result such as "high probability of influenza."

[0596] User program processing

[0597] 1. Data entry

[0598] Medical staff, who are the users, input patient medical data through a terminal. For example, a nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form, and the terminal transmits this information to the server in real time.

[0599] 2. Input of feedback

[0600] Medical staff, acting as users, input feedback on the effectiveness of treatments based on diagnostic results and patient responses. For example, doctors input feedback on the efficacy of prescribed medications and send it from their terminals to the server. The collected feedback is then used to retrain generative artificial intelligence models.

[0601] Specific example

[0602] For example, the following prompt statement may be used:

[0603] "Please enter the patient's blood pressure measurement results."

[0604] "The generative artificial intelligence model has determined that there is a possibility of diabetes."

[0605] This system can support improved diagnostic accuracy and faster decision-making regarding treatment plans in medical settings.

[0606] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0607] Step 1: Data Collection and Anonymization

[0608] The server periodically collects treatment and test data from each medical institution. Specifically, the server sends an API request at 2:00 AM every day to request new data. The data received from the medical institution's server is sent in JSON format. The server analyzes this data, removes personal information such as names, addresses, and contact information, and anonymizes it. The input is treatment and test data in JSON format from the medical institution, and the output is anonymized data from which personal information has been removed. The anonymized data is temporarily stored in storage.

[0609] Step 2: Storing in the database

[0610] The server reads anonymized data from temporary storage and categorizes it. Specifically, test data such as blood test results and blood pressure measurements are stored in the "Test Data" table, and treatment data such as treatment history and prescription information is stored in the "Treatment Data" table. This database uses MySQL or similar, enabling high-speed access. The input is anonymized data, and the output is structured data stored in the database. The server logs that the data has been correctly stored in the database.

[0611] Step 3: Training a generative artificial intelligence model

[0612] The server periodically retrieves new data from the database and retrains the generative artificial intelligence model. Specifically, it checks the training schedule at 3 AM every Sunday and retrains the model using data collected over the past week. Machine learning frameworks such as TensorFlow are used for this process. The input is new treatment and examination data, and the output is the updated generative artificial intelligence model. The model's accuracy is evaluated using test data, and if the accuracy meets the criteria, it is saved as a new model.

[0613] Step 4: Integration with the paper database

[0614] The server regularly retrieves the latest medical papers from domestic and international journal databases. Specifically, on the 1st of each month, it uses the PubMed API to retrieve the latest papers on heart disease. The retrieved paper data is machine-translated into Japanese using the Google Translate API. Then, it is analyzed using natural language processing techniques and converted into a format that can be input into a generative artificial intelligence model. The input is the paper data, and the output is new data that is added to the model.

[0615] Step 5: Provide the user interface

[0616] The terminal provides a user interface that allows medical staff to input data and review diagnostic results. Specifically, it includes input fields for doctors to enter patient blood pressure measurements and symptoms, and a dashboard to display diagnostic results. Input consists of medical data entered by medical staff, while output consists of data sent to the server and diagnostic results displayed on the terminal.

[0617] Step 6: Displaying the diagnostic results

[0618] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. Specifically, when a doctor enters the patient's symptoms, the terminal sends a request to the server to retrieve the diagnostic results. As a result, the dashboard displays a diagnostic result such as "high probability of influenza" and recommended treatment methods. The input is medical data, and the output is the diagnostic result.

[0619] Step 7: Data Entry

[0620] The medical staff, acting as users, input patient medical data into the terminal. Specifically, nurses enter the temperature, blood pressure, and pulse rate of patients with fever into a dedicated form, and this data is sent to the server. The input is the patient's medical data, and the output is the data sent to the server.

[0621] Step 8: Entering Feedback

[0622] Medical staff, acting as users, input feedback on the effectiveness of treatment and patient responses based on diagnostic results. Specifically, they input information on the efficacy of medications prescribed by doctors into a dedicated form and send it from their terminal to the server. The input is treatment feedback data, and the output is the data sent to the server. This feedback data is used to retrain generative artificial intelligence models.

[0623] (Application Example 1)

[0624] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0625] Conventional diagnostic support systems often required medical staff to manually input patient symptoms and test results, resulting in lengthy diagnostic timeliness. Furthermore, diagnostic results were not displayed in real time, and diagnostic accuracy was still insufficient. Additionally, there was a lack of direct feedback from patients regarding the effectiveness of post-diagnosis treatment, hindering effective retraining. To address these issues, there was a need for real-time data sharing and rapid delivery of diagnostic results.

[0626] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0627] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model on the treatment data and test data stored in the database; means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model; means for displaying the diagnosis result corresponding to the medical condition on a terminal; means for creating a virtual avatar of the patient and enabling interaction with medical staff; means for inputting the patient's symptoms and test results in real time and sharing them with medical staff; means for displaying the diagnosis result and treatment method in real time; and means for the patient to input feedback on the effectiveness of treatment after diagnosis. This enables rapid data sharing and real-time display of diagnosis results, and patient feedback is also quickly reflected in the system, thereby improving diagnostic accuracy.

[0628] A "medical institution" refers to a facility that performs medical procedures, such as a hospital, clinic, or medical practice.

[0629] "Treatment data" refers to information about a patient's treatment that medical staff record during the course of their medical care. Examples include the details of drug treatment and information about prescribed medications.

[0630] "Test data" refers to data from tests performed to check a patient's health status. Examples include the results of blood tests and X-ray examinations.

[0631] "Anonymization" is the process of removing personally identifiable information (such as name, address, and contact information) from data so that individuals cannot be identified.

[0632] A "database" is a system used to store and manage collected treatment and examination data.

[0633] A "generative artificial intelligence model" is an AI model that learns from large amounts of data and has the ability to recognize patterns.

[0634] "Medical condition" refers to the symptoms a patient is exhibiting or their current health status.

[0635] "Diagnosis result" refers to the analysis results regarding the medical condition calculated by the generative artificial intelligence model, and represents the conclusion of the diagnosis.

[0636] A "terminal" is a device used by medical staff or patients, and examples include smartphones, smart glasses, and head-mounted displays.

[0637] A "virtual avatar" is a digital representation of a patient in a virtual space.

[0638] "Real-time" refers to a state where data collection, processing, and display occur instantly.

[0639] "Feedback" refers to the process of returning information about treatment effectiveness and patient response obtained after diagnosis back to the system.

[0640] "User interface" refers to the screens and methods of operation that users use to operate a system or input data.

[0641] The overall system configuration and each processing procedure for implementing this invention are described below. The system consists of three elements: a server, a terminal, and a user.

[0642] Server program processing

[0643] 1. Data collection and anonymization

[0644] The server periodically collects treatment and test data from each medical institution. During this process, it anonymizes the data by removing patients' personal information. For example, a Python script is used to remove names and addresses.

[0645] 2. Storing in a database

[0646] The collected anonymized data is categorized into diagnostic data, test data, treatment data, etc., and stored in databases such as Amazon RDS.

[0647] 3. Training of generative artificial intelligence models

[0648] The server periodically trains a generative artificial intelligence model (using TensorFlow) based on the information in the database. This improves the accuracy of diagnostics based on the latest data.

[0649] 4. Integration with research paper databases

[0650] The server periodically retrieves the latest medical papers from domestic and international medical journal databases, and then machine-translates and analyzes them. For example, it uses the Google Translation API to perform natural language processing, and then re-inputs this analyzed data into a generative artificial intelligence model for retraining.

[0651] Terminal program processing

[0652] 1. Providing a user interface

[0653] The devices (such as smart glasses, head-mounted displays, and smartphones) provide an intuitive user interface that allows medical staff to easily input data and quickly review diagnostic results. For example, this can be achieved through cross-platform applications developed using React Native.

[0654] 2. Creating a virtual avatar for the patient

[0655] The terminal application provides a function that allows patients to create virtual avatars and have those avatars interact with medical staff. The virtual avatars are implemented using Unity or Unreal Engine.

[0656] 3. Data entry and sharing

[0657] Patient symptoms and test results are entered in real time via a terminal and sent to the server. Real-time input and sharing allow medical staff to access the data immediately.

[0658] 4. Real-time display of diagnostic results

[0659] The diagnostic results and recommended treatments calculated by the generative artificial intelligence model are displayed on the device in real time. For example, the diagnostic results are projected onto smart glasses.

[0660] 5. Feedback function

[0661] After diagnosis, patients input feedback about the effectiveness of treatment via a terminal and send it to a server. This feedback is also used to retrain the generative artificial intelligence model.

[0662] User program processing

[0663] 1. Data entry

[0664] Medical staff and patients, who are the users, input medical data and test results into the terminal. The entered data is immediately sent to the server and processing begins.

[0665] 2. Input of feedback

[0666] Medical staff and patients, acting as users, input feedback on the treatments they received after a diagnosis and their effectiveness, and send this feedback to the server. This collected feedback data is then used to retrain the AI ​​model.

[0667] Examples of prompt statements

[0668] Patient's symptoms: fever of 37.8°C, sore throat, cough

[0669] Past medical history: Influenza, allergies

[0670] ---

[0671] Body temperature measurement result: 37.8°C

[0672] Blood pressure measurement result: 120 / 80

[0673] Pulse rate: 88 bpm

[0674] ---

[0675] Questions for generative AI models:

[0676] What disease might be suspected based on these symptoms?

[0677] What treatments and medications are recommended?

[0678] By inputting such prompts, the generative artificial intelligence model provides accurate diagnostic results and optimal treatment methods.

[0679] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0680] Step 1:

[0681] The server collects treatment and test data from each medical institution. During collection, patient personal information is removed and anonymized. For example, a Python script is used to remove personally identifiable information such as names and addresses.

[0682] Input: Raw treatment and test data provided by each medical institution.

[0683] Output: Anonymized treatment and laboratory data.

[0684] Step 2:

[0685] The server stores anonymized treatment and test data in a database. Here, the data is categorized into diagnostic data, test data, treatment data, etc. Amazon RDS or a similar database is recommended.

[0686] Input: Anonymized treatment and laboratory data.

[0687] Output: Categorized data stored in the database.

[0688] Step 3:

[0689] The server periodically trains a generative artificial intelligence model using treatment and examination data stored in the database. The model is updated with the latest data using AI frameworks such as TensorFlow.

[0690] Input: Treatment data and test data stored in the database.

[0691] Output: A trained generative artificial intelligence model.

[0692] Step 4:

[0693] The server periodically retrieves the latest medical papers from domestic and international medical paper databases. This paper data is machine translated using the Google Translation API and other tools, and then analyzed using natural language processing techniques. The analysis results are then added to a generative artificial intelligence model for further training.

[0694] Input: Latest medical research paper data.

[0695] Output: A retrained generative artificial intelligence model.

[0696] Step 5:

[0697] The terminal provides medical staff with a user interface for data entry. Using this interface, medical staff can input patient symptoms and test results, and the data is sent to the server. For example, React Native can be used to enable intuitive operation.

[0698] Input: Patient symptoms and test results entered by medical staff.

[0699] Output: Medical data sent to the server.

[0700] Step 6:

[0701] The device creates a virtual avatar of the patient. Using Unity or Unreal Engine, it generates a patient avatar in a virtual space, enabling interaction with medical staff.

[0702] Input: Basic patient information and image data.

[0703] Output: A virtual avatar displayed in a virtual space.

[0704] Step 7:

[0705] The terminal shares the patient's symptoms and test results entered into it with the server in real time. This allows medical staff to access the data immediately.

[0706] Input: Patient's symptoms and test results.

[0707] Output: Data sent to the server in real time.

[0708] Step 8:

[0709] The terminal displays diagnostic results and recommended treatments calculated by a generative artificial intelligence model in real time. The diagnostic results are projected using devices such as smart glasses, and medical staff determine treatment plans based on this information.

[0710] Input: Diagnostic results calculated by a generative artificial intelligence model.

[0711] Output: Diagnostic results projected onto the terminal in real time.

[0712] Step 9:

[0713] After the diagnosis, the user inputs feedback on the treatment's effectiveness into the device and sends it to the server. This allows the generative artificial intelligence model to retrain, improving the accuracy of the diagnosis.

[0714] Input: User-submitted feedback regarding treatment effectiveness.

[0715] Output: Feedback data sent to the server.

[0716] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0717] Modes for carrying out the invention

[0718] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to train on this data, and provides accurate diagnostic results for each medical condition. By further incorporating an emotion engine that recognizes the user's emotions into this system, the method of providing diagnostic results can be adjusted based on the user's emotional state.

[0719] Server program processing

[0720] 1. Data collection and anonymization:

[0721] The server establishes secure communication protocols with each medical institution and regularly collects treatment and test data. During this process, patient personal information is removed and anonymized.

[0722] Specific example: Retrieve patient test results and treatment records from a hospital server, and anonymize them by removing personal information.

[0723] 2. Storing in the database:

[0724] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[0725] Specific example: Classify inspection data into appropriate tables and store them in a database.

[0726] 3. Training of generative artificial intelligence models:

[0727] The server uses data extracted from the database to train a generative artificial intelligence model. This improves the accuracy of diagnosing medical conditions.

[0728] Specific example: Retrain a generative artificial intelligence model using new treatment data to improve diagnostic accuracy.

[0729] 4. Integration with research paper databases:

[0730] The server periodically retrieves the latest research papers from an external database, translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[0731] Specific example: Obtain new papers on heart disease from PubMed, translate and analyze them using natural language processing techniques, and add them to the model.

[0732] Terminal program processing

[0733] 1. Providing a user interface:

[0734] The terminal provides a user interface that is easy for medical staff to use. This interface is designed to be intuitive to operate.

[0735] Specific example: Using the React framework, build an input form for entering patient medical data and a dashboard to display diagnostic results.

[0736] 2. How the emotion engine works:

[0737] While the user is reviewing the diagnostic results, the device activates an emotion engine to analyze the user's emotional state based on facial expression data, voice tone, and other factors.

[0738] Specific example: Using the device's camera and microphone, facial expression analysis software is run to determine in real time whether the user is surprised, relieved, or otherwise experiencing other emotions.

[0739] 3. Display of emotionally-based diagnostic results:

[0740] The device adjusts how the diagnostic results are displayed based on emotional data analyzed by the emotion engine. For example, if the user is feeling anxious, it will provide additional explanations or step-by-step information.

[0741] Specific example: If the system determines that the user is surprised when the diagnostic results are displayed, a detailed explanation of the diagnostic results will be displayed in a pop-up window.

[0742] User program processing

[0743] 1. Entering medical data:

[0744] The medical staff, acting as users, enter patient medical data into the input fields on the terminal and send it to the server.

[0745] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form.

[0746] 2. Confirmation of the diagnosis results:

[0747] The medical staff, as users, review the diagnostic results displayed on the terminal and consider the emotional analysis results from the emotion engine when deciding how to explain things to the patient.

[0748] Specific example: A doctor reviews a diagnosis result displayed by a generative artificial intelligence model, which indicates "possible pneumonia," and carefully explains it to a patient who appears anxious.

[0749] 3. Entering feedback:

[0750] Medical staff, acting as users, input feedback on the treatments performed after the diagnosis and their effectiveness into a terminal and send it to the server. The transmitted feedback data is then used again as training data for a generative artificial intelligence model.

[0751] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[0752] This series of processing steps not only improves diagnostic accuracy in medical settings but also enables flexible responses based on user emotions, providing a better patient experience.

[0753] The following describes the processing flow.

[0754] Modes for carrying out the invention

[0755] Server program processing

[0756] Step 1: Data Collection and Anonymization

[0757] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient identification information is removed and the data is anonymized.

[0758] Example: Retrieve patient test results and treatment records from the hospital server, and anonymize them by removing personal information.

[0759] Step 2: Data reception and verification

[0760] The server analyzes the received data and verifies whether the data format is correct and consistent.

[0761] Example: Check if JSON data has the correct structure and log any problems in the error log.

[0762] Step 3: Data anonymization

[0763] The server removes personal information from the received data and performs anonymization.

[0764] Specific example: Remove patient names and addresses from data and generate a unique anonymous identifier.

[0765] Step 4: Data classification and storage

[0766] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[0767] Specific example: Classify inspection data into appropriate tables and store them in a database.

[0768] Step 5: Training a generative artificial intelligence model

[0769] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[0770] Specific example: Export data in CSV format, input it into a machine learning algorithm, and train the model.

[0771] Step 6: Integration with the research paper database

[0772] The server periodically retrieves the latest papers from external research paper databases and performs translation and analysis.

[0773] Specific example: Obtain new articles on heart disease from PubMed, translate them into Japanese using machine translation, and then analyze them.

[0774] Step 7: Retrain the model

[0775] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[0776] Specific example: Retrain the model with a dataset that includes the latest research paper data to improve diagnostic accuracy.

[0777] Terminal program processing

[0778] Step 1: Provide the user interface

[0779] The terminal provides input forms and dashboards that medical staff can easily operate.

[0780] Specific example: Using the React framework, build a medical data input form and a diagnostic results display dashboard.

[0781] Step 2: Operating the Emotion Engine

[0782] While the user is reviewing the diagnostic results, the device operates an emotion engine, analyzing facial expression data and voice tone.

[0783] Specific example: Using the device's camera and microphone, facial expression analysis software is used to determine the user's emotional state in real time.

[0784] Step 3: Entering medical data

[0785] Medical staff, who are the users, input patient medical data into their terminals and send it to the server.

[0786] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form and sends a diagnostic request to the server.

[0787] Step 4: Processing diagnostic requests

[0788] The server uses a generative artificial intelligence model based on the input medical data to calculate the diagnosis.

[0789] Specific example: A generative artificial intelligence model outputs a diagnosis result stating "high suspicion of infectious disease," and sends that result to the terminal in JSON format.

[0790] Step 5: Displaying the diagnostic results

[0791] The terminal displays the diagnostic results received from the server. Additional analysis results from the emotion engine are also displayed simultaneously.

[0792] Specific example: Along with the diagnostic results, a pop-up message or additional explanation tailored to the user's emotional state will be displayed.

[0793] User program processing

[0794] Step 1: Confirming the emotionally-based diagnostic results

[0795] The medical staff, who are the users, review the diagnostic results and the analysis results of the emotion engine, and then explain them to the patient based on that information.

[0796] Specific example: The system carefully explains to the user the diagnosis that "there is a high possibility of pneumonia" and the emotional information that the user has identified as "anxiety."

[0797] Step 2: Entering emotionally-based feedback

[0798] After the diagnosis, the medical staff, acting as users, input the patient's response and treatment results as feedback into the terminal and send it to the server.

[0799] Specific example: The user inputs information about the patient's condition and post-treatment follow-up, and sends this information, along with the results of the emotion engine's analysis indicating "satisfied," to the server.

[0800] This series of processing steps not only improves diagnostic accuracy in medical settings, but also enables flexible responses based on user emotions, thereby providing a better patient experience.

[0801] (Example 2)

[0802] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0803] Traditional medical diagnostic systems required manual collection and anonymization of treatment and test data, which was time-consuming and labor-intensive. Furthermore, there was a lack of systems capable of efficiently learning from the collected data and providing rapid and accurate diagnostic results. Additionally, the presentation of diagnostic results was not tailored to the patient's emotional state, resulting in a poor patient experience.

[0804] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, and means for analyzing the user's emotions on the terminal and adjusting the method of presenting the diagnosis result based on the emotion result. This enables efficient collection and anonymization of treatment data and examination data, rapid and accurate diagnosis, and presentation of diagnosis results that respond to the patient's emotions.

[0805] "Treatment data" refers to information about treatment provided by a medical institution, including details such as the patient's treatment plan, treatment progress, and medications used.

[0806] "Test data" refers to information that includes the results of various tests performed on a patient, and includes data such as blood tests, urine tests, and imaging diagnostics.

[0807] "Anonymization" is the process of removing or concealing information that identifies an individual, and is a means of handling data securely while protecting patient privacy.

[0808] A "database" is a system for organizing and storing collected treatment and examination data, and is a collection of information that enables efficient data retrieval and management.

[0809] A "generative artificial intelligence model" is an artificial intelligence system that uses collected and learned data to perform tasks such as prediction and classification, equipped with algorithms and data structures.

[0810] "Diagnosis results" refer to information that shows judgments and conclusions about the patient's condition calculated by a generative artificial intelligence model based on collected treatment and examination data.

[0811] A "terminal" refers to a device such as a computer or tablet operated by medical staff, used for data input and display, sentiment analysis, and other similar tasks.

[0812] "Emotional analysis" is a technology that identifies a user's emotional state from their facial expressions, voice tone, and behavior, and is a means of understanding the user's psychological state.

[0813] A "user interface" refers to the interface, including screens and input forms, that medical staff use to access and operate a system through their terminals.

[0814] This invention relates to a system that provides diagnostic results by training a generative artificial intelligence model based on treatment and examination data collected from medical institutions. This system consists of three main components: a server, a terminal, and a user.

[0815] Server Processing

[0816] The server connects to each healthcare facility using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. This data is first anonymized by removing or concealing personal information. For example, personally identifiable information such as names and addresses in the collected data is replaced with "anonymous IDs."

[0817] Anonymized data is categorized into "diagnostic data," "test data," and "treatment data," and stored in a database. This database enables efficient data management and retrieval.

[0818] Next, a generative artificial intelligence model is trained using data extracted from this database. During this training process, new treatment data is added to the training set, updating the existing model. For example, retraining the model using a backpropagation algorithm can improve the accuracy of disease diagnosis.

[0819] Furthermore, the server periodically retrieves the latest research papers from external academic databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). As a result, the paper data is also incorporated into the generative artificial intelligence model for retraining.

[0820] Terminal processing

[0821] The terminal provides a user interface that is easy for medical staff to use. For example, patient information input forms and diagnostic result dashboards are built using the React framework. Users can easily input data and check diagnostic results by simply operating the system.

[0822] When displaying diagnostic results, the device captures the user's facial expressions and voice tone in real time and analyzes the user's psychological state using emotion analysis algorithms (e.g., OpenFace or IBM Watson Tone Analyzer). Once the user's emotional state, such as surprise or anxiety, is identified, the way the diagnostic results are presented is adjusted accordingly. For example, if the user is feeling anxious, a pop-up window with additional detailed explanations will be displayed.

[0823] User processing

[0824] Medical staff, as users, enter patient symptoms and medical data into input fields on their terminals and send them to the server. For example, they might enter a patient's fever or cough into an input form and submit it.

[0825] When the diagnostic results are displayed on the terminal, the emotional analysis results are also reviewed. Medical staff explain the results to the patient based on the diagnostic results presented by the generative artificial intelligence model. In doing so, the patient's emotional state is also taken into consideration to provide a more appropriate explanation.

[0826] Feedback regarding treatment results and their effectiveness is also entered into the terminal and sent to the server. The transmitted feedback data is used to retrain generative artificial intelligence models, contributing to improved diagnostic accuracy in the future.

[0827] Example of a prompt

[0828] Use the following prompts to input data into the generative AI model:

[0829] Please retrain the diagnostic accuracy for cardiac disease using recently collected treatment data. Please also incorporate the latest research papers.

[0830] This will not only improve the accuracy of diagnoses in medical settings, but also enable flexible responses based on user emotions, thereby providing a better patient experience.

[0831] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0832] Step 1:

[0833] Data collection and anonymization

[0834] The server connects with each healthcare institution using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. It receives treatment and test data sent from healthcare institutions as input. Because this data contains patient personal information, it is first anonymized by removing personal information (name, ID, address, etc.). Specifically, it scans each field of the collected data and replaces personal information with meaningless data such as an "anonymous ID." The output is the anonymized treatment and test data.

[0835] Step 2:

[0836] Storage in database

[0837] The server categorizes anonymized data into categories such as "diagnostic data," "test data," and "treatment data." It receives anonymized treatment and test data as input. This data is then organized by category for storage in the appropriate database tables. For example, if anonymized test data is collected, it is organized and stored in the "test data" table. The output is an efficiently organized database.

[0838] Step 3:

[0839] Learning of generative artificial intelligence models

[0840] The server extracts relevant data from the database and trains a generative artificial intelligence model. It extracts "diagnostic data," "test data," and "treatment data" from the database as input. Based on this data, it uses the generative AI model to train and improve the accuracy of disease diagnosis. Specifically, it adds new treatment data to the model's training set and uses a backpropagation algorithm to update the existing model. The output is the trained generative AI model.

[0841] Step 4:

[0842] Integration with research paper databases

[0843] The server periodically retrieves the latest research papers from external research paper databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). It receives paper data retrieved from external research paper databases as input. The retrieved paper text is passed through translation and analysis tools to convert it into a format suitable for input into a generative artificial intelligence model. Specifically, it uses the PubMed API to perform keyword searches, downloads new papers, and performs translation and content analysis using natural language processing techniques. The output is the analyzed paper data used for retraining.

[0844] Step 5:

[0845] Providing a user interface

[0846] The terminal provides an interface that is easy for medical staff to use. It receives data for medical staff to input patient information as input. Using the React framework, a patient information input form and a dashboard displaying diagnostic results are built. Specifically, the input form and dashboard are designed and implemented to allow users to operate them intuitively. The output is a user-friendly interface.

[0847] Step 6:

[0848] How the emotion engine works

[0849] The device operates an emotion engine while the user is reviewing the diagnostic results, analyzing the user's facial expressions and voice tone. It receives user facial and voice data acquired using the device's camera and microphone as input. This data is analyzed using an emotion analysis algorithm (e.g., OpenFace or IBM Watson Tone Analyzer) to identify the user's emotional state. Specifically, the camera captures facial expressions, which are then analyzed in real time by the algorithm. The output provides data regarding the user's emotional state.

[0850] Step 7:

[0851] Display of emotion-based diagnostic results

[0852] The device adjusts how the diagnostic results are presented based on data analyzed by the emotion engine. It receives the analysis results from the emotion engine as input. It selects an appropriate method for displaying the diagnostic results based on the analysis results, such as when the user is feeling anxious. Specifically, if the user is determined to be anxious, a pop-up window containing a detailed explanation is displayed. The output is a display of the diagnostic results adapted to the user.

[0853] Step 8:

[0854] Entering medical data

[0855] The medical staff, acting as users, input patient symptoms and medical data into the terminal's input fields. For input, they enter medical data such as the patient's fever or cough into the input form. The terminal then sends this data to the server. Specifically, when the medical staff enters data into the input form and presses the submit button, the input data is sent to the server. The output is the medical data sent to the server.

[0856] Step 9:

[0857] Checking the diagnosis results

[0858] The medical staff, acting as users, review the diagnostic results displayed on the terminal and provide explanations to the patient, taking into account the emotional analysis results from the emotion engine. The input is the diagnostic results and emotional analysis results displayed on the terminal. Based on this data, the medical staff provides the most appropriate explanation to the patient. Specifically, they review the displayed diagnostic results and emotional analysis results, paying particular attention to explanations if the patient appears anxious. The output is the most appropriate explanation for the patient.

[0859] Step 10:

[0860] Feedback Input

[0861] Medical staff, acting as users, input feedback on treatment and its effectiveness after a diagnosis into a terminal and send it to the server. The input consists of data on treatment effectiveness and patient response entered into an input form. The terminal then sends this feedback data to the server. Specifically, when medical staff enter data into the feedback form and press the submit button, the input data is sent to the server. The output is the feedback data sent to the server. This data is then used to retrain generative artificial intelligence models, improving diagnostic accuracy.

[0862] (Application Example 2)

[0863] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0864] In modern factories, accurately diagnosing machine malfunctions and maintenance needs is difficult. Furthermore, when communicating fault diagnosis results to workers, the information provided is often difficult to understand, leading to anxiety and misunderstandings. A system is needed to solve these problems and enable efficient and safe work processes.

[0865] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, means for analyzing the user's emotions using the terminal's camera and microphone, and means for adjusting the method of presenting the diagnosis result based on the analyzed emotion data. As a result, it is possible to calculate a diagnosis result using an AI model based on collected machine operation data and failure data, and to perform emotion analysis of workers using an emotion engine, thereby providing a diagnosis result in an easy-to-understand format.

[0866] "Treatment data" refers to data from a medical institution regarding the process and methods of treatment for a patient's medical condition.

[0867] "Test data" refers to data related to the results of various tests conducted at a medical institution to diagnose a patient's medical condition.

[0868] "Anonymization" is the process of removing information that can identify an individual, so that the individual cannot be identified.

[0869] A "generative artificial intelligence model" is a type of artificial intelligence that learns from data, generates new information, and provides predictions and diagnostic results.

[0870] "Learning" is the process by which a generative artificial intelligence model uses provided data to improve its prediction accuracy and performance.

[0871] A "diagnosis result" is a judgment or conclusion about a medical condition calculated by a generative artificial intelligence model based on the provided data.

[0872] A "terminal" is a device that provides a user interface, allowing medical staff to input medical data and review diagnostic results.

[0873] "Camera and microphone" refers to devices installed in a terminal for capturing video and audio.

[0874] Analyzing "emotions" involves determining the user's emotional state based on their facial expressions and voice, using data acquired from the camera and microphone.

[0875] Adjusting the "presentation method" means changing the way the diagnostic results are displayed and the content of those results to suit the user, based on the analyzed emotional data.

[0876] "Machine operation data" refers to data regarding the operating status of various machines in a factory.

[0877] "Failure data" refers to information such as the state of a machine and error logs when it malfunctions in a factory.

[0878] System Overview

[0879] This invention is a system that collects treatment and examination data anonymized from various medical institutions, trains a generative artificial intelligence model on this data, and provides accurate diagnostic results. Furthermore, it has a function that analyzes the user's emotions using the terminal's camera and microphone, and adjusts the method of presenting the diagnostic results based on the analysis results.

[0880] Hardware and software used

[0881] The following hardware and software are used for the server and terminal:

[0882] 1. Server: Performs data collection, anonymization, storage in a database, training of generative artificial intelligence models, and calculation of diagnostic results.

[0883] 2. Camera and microphone: Capture the user's facial expressions and voice in real time and use them for emotion analysis.

[0884] 3. Database: Stores treatment data and test data.

[0885] 4. Generative artificial intelligence models: These models calculate diagnostic results based on data.

[0886] 5. Emotion Analysis Engine: Analyzes the user's emotions using data acquired from the camera and microphone.

[0887] 6. User Interface (UI): This interface handles the input of medical data, the display of diagnostic results, and the presentation of sentiment analysis results.

[0888] Specific system configuration

[0889] Data collection and anonymization

[0890] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. For example, patient test results and treatment records are retrieved from the hospital server, personal information is removed, and then the data is stored in the database.

[0891] Storage in database

[0892] The collected treatment and examination data are stored in a database on the server. This database is categorized into categories such as "diagnostic data," "examination data," and "treatment data" for efficient management. For example, vibration data and temperature data are classified into appropriate tables and stored in the database.

[0893] Training and diagnostics of generative artificial intelligence models

[0894] The server uses data extracted from the database to train a generative artificial intelligence model. This improves the accuracy of diagnosing medical conditions. Furthermore, it periodically retrieves the latest research paper data to retrain the model. For example, it translates the latest papers from PubMed using natural language processing techniques and adds the analysis results to the training.

[0895] Emotional analysis and presentation of diagnostic results

[0896] The device uses its built-in camera and microphone to analyze the user's emotional state in real time. The emotion analysis engine recognizes emotions from the user's facial expressions and voice, and adjusts how the diagnostic results are displayed. For example, if the system determines that the user is surprised when the diagnostic results are displayed, a detailed explanation will be shown in a pop-up window.

[0897] Specific examples and prompt statements

[0898] Specific example:

[0899] Machine vibration and temperature data are collected, and an AI model predicts the "possibility of bearing wear." If the emotional engine detects that the worker is startled, a pop-up will display photos of bearing wear symptoms and a video explaining the replacement procedure.

[0900] Example of a prompt:

[0901] Collect failure data, analyze it using an AI model, and display the judgment results. Also, analyze the emotions of the workers using cameras. Based on the obtained emotion data, adjust the presentation method of the judgment results.

[0902] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0903] Step 1:

[0904] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. The input consists of treatment and test data received from medical institutions, and the output is anonymized data. Specific operations include data format conversion and removal of personal information fields.

[0905] Step 2:

[0906] The server stores the collected anonymized data in a database. The input consists of anonymized treatment and examination data, and the output is data categorized and stored within the database. Specifically, the server performs insertion operations into the database tables.

[0907] Step 3:

[0908] The server trains a generative artificial intelligence model using treatment and examination data stored in a database. The input is training data extracted from the database, and the output is the trained generative AI model. Specifically, the server performs tasks such as updating model parameters and applying the learning algorithm.

[0909] Step 4:

[0910] The server uses a generative artificial intelligence model to calculate a diagnosis for a medical condition. New treatment and test data are used as input, and the output is a diagnosis. Specifically, the server inputs data into a pre-trained model and outputs the diagnosis.

[0911] Step 5:

[0912] The server displays the diagnostic results on the terminal. The input is the calculated diagnostic results, and the output is the diagnostic results displayed on the terminal screen. Specific actions include updating UI components and visualizing the data.

[0913] Step 6:

[0914] The device analyzes the user's emotions in real time using a camera and microphone. Inputs include video and audio data acquired from the camera and microphone, while output is user emotion data. Specifically, it applies facial recognition algorithms and voice analysis algorithms.

[0915] Step 7:

[0916] The device adjusts how the diagnostic results are presented based on the analyzed emotional data. Inputs include the user's emotional data and diagnostic results, and output is a display of the adjusted diagnostic results. Specific actions include changing the display method of the diagnostic results and displaying pop-ups with additional explanations.

[0917] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0918] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0919] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0920] [Third Embodiment]

[0921] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0922] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0923] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0924] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0925] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0926] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0927] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0928] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0929] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0930] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0931] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0932] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0933] Modes for carrying out the invention

[0934] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to learn from this data, and provides accurate diagnostic results for each medical condition. This system allows medical staff to quickly and accurately obtain personalized diagnostic information for each patient, enabling appropriate treatment and drug administration.

[0935] Server program processing

[0936] 1. Data collection and anonymization

[0937] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information (name, address, contact information, etc.) is removed and anonymized.

[0938] Specific example: Obtain "Patient X's blood test results" and "blood pressure measurement results" from Hospital A's server, and anonymize this data by removing names and addresses.

[0939] 2. Storing in a database

[0940] The server categorizes the anonymized data (diagnostic data, test data, treatment data, etc.) and stores it in a database. This database is designed to allow high-speed access to large amounts of data.

[0941] Specific example: Store blood test results and blood pressure measurement results in the "Test Data" table of the database.

[0942] 3. Training of generative artificial intelligence models

[0943] The server periodically uses database information to train generative artificial intelligence models. This improves the accuracy of disease diagnosis based on the latest treatment and examination data.

[0944] Specific example: Retrain a generative artificial intelligence model using newly added blood test data to improve the accuracy of diabetes diagnosis.

[0945] 4. Integration with research paper databases

[0946] The server periodically retrieves the latest medical papers from domestic and international journal databases, machine-translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[0947] Specific example: Obtain the latest papers on heart disease from PubMed and translate them into Japanese using natural language processing technology. Then, add the analysis results to the model and retrain it.

[0948] Terminal program processing

[0949] 1. Providing a user interface

[0950] The terminal provides a user interface that allows medical staff to easily input data and review diagnostic results. This interface is designed to be intuitive to use.

[0951] Specific example: Provide an input field for doctors to enter patient blood pressure measurement results and a dashboard to display the diagnostic results.

[0952] 2. Display of diagnostic results

[0953] The terminal displays a diagnostic result calculated by a generative artificial intelligence model based on the entered medical data. Medical staff then decide on a treatment plan based on this result.

[0954] Specific example: When a doctor inputs a patient's symptoms, a generative artificial intelligence model determines that "influenza is highly likely" and displays a list of appropriate treatments.

[0955] User program processing

[0956] 1. Data entry

[0957] The medical staff, acting as users, input patient medical data into the terminal. The entered data is sent to the server, and processing begins.

[0958] Specific example: A nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form.

[0959] 2. Input of feedback

[0960] Medical staff, acting as users, input feedback on the treatments they administered after the diagnosis and their effectiveness, and send this feedback to the server. This submitted feedback data is then used again as training data for a generative artificial intelligence model.

[0961] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[0962] In this way, this system dramatically improves diagnostic accuracy in medical settings and enables the provision of optimal treatment plans for each patient.

[0963] The following describes the processing flow.

[0964] Program processing flow

[0965] Server program processing

[0966] Step 1: Data Collection and Anonymization

[0967] The server establishes a secure communication protocol (e.g., HTTPS) with each medical institution and periodically requests treatment and test data.

[0968] Example: A server sends a request to a hospital server via an API to retrieve patient test results and treatment records.

[0969] Step 2: Data reception and verification

[0970] The server analyzes the received data and automatically checks for format consistency and the presence of required fields.

[0971] Example: Verify that JSON data has the correct structure and log any errors in the error log.

[0972] Step 3: Data anonymization

[0973] The server removes personal information from incoming data, anonymizing it to protect patient privacy.

[0974] Example: Remove patient names, addresses, and contact information from the data, and randomly generate identifiers.

[0975] Step 4: Data classification and storage

[0976] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[0977] Example: Use an SQL statement to insert the test results into the test data table.

[0978] Step 5: Training a generative artificial intelligence model

[0979] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[0980] Example: Export data in CSV format and input it into a machine learning algorithm to train a neural network.

[0981] Step 6: Integration with the research paper database

[0982] The server periodically retrieves new papers from external academic databases and performs translation and text analysis.

[0983] Example: Set up a scheduled job to download papers from PubMed via API and then process them for machine translation.

[0984] Step 7: Retrain the model

[0985] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[0986] Example: Retrain the model using a dataset that includes the latest research paper data.

[0987] Terminal program processing

[0988] Step 1: Provide the user interface

[0989] The terminal provides input forms and dashboards that medical staff can easily operate.

[0990] Example: Develop a test result input form and a diagnostic results dashboard using the React framework.

[0991] Step 2: Enter diagnostic data

[0992] The medical staff, who are the users, enter the patient's medical data (symptoms, test results, etc.) into the input form on the terminal.

[0993] Example: A doctor enters the patient's temperature, blood pressure, and pulse rate into the input fields.

[0994] Step 3: Submit a diagnostic request

[0995] The terminal sends the entered data to the server and makes a diagnostic request.

[0996] Example: By pressing the submit button, the input data is sent to the server via the HTTPS protocol.

[0997] Step 4: Displaying the diagnostic results

[0998] The server uses a generative artificial intelligence model to calculate the diagnostic results and sends them back to the terminal.

[0999] Example: A generative artificial intelligence model calculates the "likelihood of influenza" and sends the result to the terminal in JSON format.

[1000] Step 5: Confirm and display the diagnostic results.

[1001] The terminal displays the diagnostic results returned from the server on the user interface.

[1002] Example: Display the diagnostic results on the dashboard so that doctors can review them.

[1003] User program processing

[1004] Step 1: Entering medical data

[1005] The medical staff, acting as users, enter the patient's medical data into the input fields on the terminal and then transmit it.

[1006] Example: A nurse copies test results obtained from the electronic medical record system into an input form.

[1007] Step 2: Confirmation of diagnostic results

[1008] The medical staff, who are the users, review the diagnostic results displayed on the terminal and decide on a treatment plan based on those results.

[1009] Example: The doctor checks the displayed diagnosis result, "Possible influenza," and prescribes antiviral medication.

[1010] Step 3: Entering Feedback

[1011] The medical staff, who are the users, input post-treatment feedback information into their terminals and send it to the server.

[1012] Example: As follow-up data after treatment, the patient's condition improvement and the presence or absence of side effects are entered and sent to the database.

[1013] This series of processing steps allows the system to improve diagnostic accuracy in medical settings and provide personalized treatment quickly.

[1014] (Example 1)

[1015] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1016] Conventional medical systems have been insufficient in anonymizing treatment and test data collected from each medical institution, and in improving diagnostic accuracy using generative artificial intelligence models. Furthermore, the lack of retraining utilizing the latest medical papers and feedback information has made it difficult to improve the accuracy of diagnostic results and provide prompt treatment.

[1017] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1018] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model using the treatment data and test data stored in the database; means for periodically acquiring the latest medical papers, machine translating and analyzing them; means for inputting the translated paper data into the generative artificial intelligence model for retraining; and means for medical staff to input feedback on treatment effectiveness and transmit it to the server. As a result, the generative artificial intelligence model continuously retrains using the collected data and the latest medical knowledge, enabling improved diagnostic accuracy and rapid treatment.

[1019] "Treatment data" refers to a series of pieces of information about the treatments provided to a patient at a medical institution, the medications prescribed, and the treatment outcomes.

[1020] "Test data" refers to information showing the results of various medical tests (blood tests, imaging tests, urine tests, etc.) performed on a patient.

[1021] "Anonymization" is the process of removing a patient's personal information (such as name, address, and contact information) so that a specific individual cannot be identified.

[1022] A "database" is a data aggregation system that efficiently stores collected treatment and examination data and manages it so that it can be quickly accessed as needed.

[1023] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates appropriate diagnostic results or predictions for new data.

[1024] A "user interface" is a software component that provides a visual screen layout and input methods for medical staff to operate a system.

[1025] "Diagnosis" refers to a professional evaluation and opinion regarding a patient's medical condition and symptoms, based on the analysis of collected data and generative artificial intelligence models.

[1026] "Feedback" refers to information that medical staff input regarding diagnostic results and treatment effectiveness, and is used to improve the accuracy and retrain generative artificial intelligence models.

[1027] "Paper data" refers to the latest medical information published in academic papers and research results, and is used as data for training generative artificial intelligence models.

[1028] "Machine translation" is a technology that automatically converts text written in one language into another language using a computer.

[1029] This invention relates to a system that collects anonymized treatment and examination data from various medical institutions, stores this data in a database, and then uses a generative artificial intelligence model to train the database and calculate a diagnosis for a patient's condition. This system enables medical staff to make quick and accurate diagnoses. Furthermore, the system aims to further improve diagnostic accuracy by retraining the model based on the latest medical papers and feedback information.

[1030] Server program processing

[1031] 1. Data collection and anonymization

[1032] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information is removed and the data is anonymized. For example, the server sends an API request at 2 AM every day to request new data, analyzes the received data, and removes personal information. A dedicated data analysis engine is used for this process.

[1033] 2. Storing in a database

[1034] The server categorizes the anonymized data and stores it in the corresponding tables. The database is designed to allow high-speed access to large amounts of data, and database types such as MySQL are commonly used.

[1035] 3. Training of generative artificial intelligence models

[1036] The server uses periodically collected data to train a generative artificial intelligence model. For example, it checks the training schedule every Sunday night and retrains the model with new data. Machine learning frameworks such as TensorFlow are used in this process.

[1037] 4. Integration with research paper databases

[1038] The server regularly retrieves the latest medical papers from domestic and international research paper databases, machine-translates them, and analyzes them. For example, once a month, it uses the PubMed API to retrieve the latest papers on heart disease, translates them using the Google Translate API, analyzes them with a natural language processing engine, and inputs the results into a model.

[1039] Terminal program processing

[1040] 1. Providing a user interface

[1041] The terminal provides a user interface for medical staff to operate. This interface is intuitive and includes input fields such as blood pressure measurement results and a dashboard that displays diagnostic results.

[1042] 2. Display of diagnostic results

[1043] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. For example, when a doctor enters a patient's symptoms, the terminal sends a request to the server and displays a diagnostic result such as "high probability of influenza."

[1044] User program processing

[1045] 1. Data entry

[1046] Medical staff, who are the users, input patient medical data through a terminal. For example, a nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form, and the terminal transmits this information to the server in real time.

[1047] 2. Input of feedback

[1048] Medical staff, acting as users, input feedback on the effectiveness of treatments based on diagnostic results and patient responses. For example, doctors input feedback on the efficacy of prescribed medications and send it from their terminals to the server. The collected feedback is then used to retrain generative artificial intelligence models.

[1049] Specific example

[1050] For example, the following prompt statement may be used:

[1051] "Please enter the patient's blood pressure measurement results."

[1052] "The generative artificial intelligence model has determined that there is a possibility of diabetes."

[1053] This system can support improved diagnostic accuracy and faster decision-making regarding treatment plans in medical settings.

[1054] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1055] Step 1: Data Collection and Anonymization

[1056] The server periodically collects treatment and test data from each medical institution. Specifically, the server sends an API request at 2:00 AM every day to request new data. The data received from the medical institution's server is sent in JSON format. The server analyzes this data, removes personal information such as names, addresses, and contact information, and anonymizes it. The input is treatment and test data in JSON format from the medical institution, and the output is anonymized data from which personal information has been removed. The anonymized data is temporarily stored in storage.

[1057] Step 2: Storing in the database

[1058] The server reads anonymized data from temporary storage and categorizes it. Specifically, test data such as blood test results and blood pressure measurements are stored in the "Test Data" table, and treatment data such as treatment history and prescription information is stored in the "Treatment Data" table. This database uses MySQL or similar, enabling high-speed access. The input is anonymized data, and the output is structured data stored in the database. The server logs that the data has been correctly stored in the database.

[1059] Step 3: Training a generative artificial intelligence model

[1060] The server periodically retrieves new data from the database and retrains the generative artificial intelligence model. Specifically, it checks the training schedule at 3 AM every Sunday and retrains the model using data collected over the past week. Machine learning frameworks such as TensorFlow are used for this process. The input is new treatment and examination data, and the output is the updated generative artificial intelligence model. The model's accuracy is evaluated using test data, and if the accuracy meets the criteria, it is saved as a new model.

[1061] Step 4: Integration with the paper database

[1062] The server regularly retrieves the latest medical papers from domestic and international journal databases. Specifically, on the 1st of each month, it uses the PubMed API to retrieve the latest papers on heart disease. The retrieved paper data is machine-translated into Japanese using the Google Translate API. Then, it is analyzed using natural language processing techniques and converted into a format that can be input into a generative artificial intelligence model. The input is the paper data, and the output is new data that is added to the model.

[1063] Step 5: Provide the user interface

[1064] The terminal provides a user interface that allows medical staff to input data and review diagnostic results. Specifically, it includes input fields for doctors to enter patient blood pressure measurements and symptoms, and a dashboard to display diagnostic results. Input consists of medical data entered by medical staff, while output consists of data sent to the server and diagnostic results displayed on the terminal.

[1065] Step 6: Displaying the diagnostic results

[1066] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. Specifically, when a doctor enters the patient's symptoms, the terminal sends a request to the server to retrieve the diagnostic results. As a result, the dashboard displays a diagnostic result such as "high probability of influenza" and recommended treatment methods. The input is medical data, and the output is the diagnostic result.

[1067] Step 7: Data Entry

[1068] The medical staff, acting as users, input patient medical data into the terminal. Specifically, nurses enter the temperature, blood pressure, and pulse rate of patients with fever into a dedicated form, and this data is sent to the server. The input is the patient's medical data, and the output is the data sent to the server.

[1069] Step 8: Entering Feedback

[1070] Medical staff, acting as users, input feedback on the effectiveness of treatment and patient responses based on diagnostic results. Specifically, they input information on the efficacy of medications prescribed by doctors into a dedicated form and send it from their terminal to the server. The input is treatment feedback data, and the output is the data sent to the server. This feedback data is used to retrain generative artificial intelligence models.

[1071] (Application Example 1)

[1072] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1073] Conventional diagnostic support systems often required medical staff to manually input patient symptoms and test results, resulting in lengthy diagnostic timeliness. Furthermore, diagnostic results were not displayed in real time, and diagnostic accuracy was still insufficient. Additionally, there was a lack of direct feedback from patients regarding the effectiveness of post-diagnosis treatment, hindering effective retraining. To address these issues, there was a need for real-time data sharing and rapid delivery of diagnostic results.

[1074] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1075] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model on the treatment data and test data stored in the database; means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model; means for displaying the diagnosis result corresponding to the medical condition on a terminal; means for creating a virtual avatar of the patient and enabling interaction with medical staff; means for inputting the patient's symptoms and test results in real time and sharing them with medical staff; means for displaying the diagnosis result and treatment method in real time; and means for the patient to input feedback on the effectiveness of treatment after diagnosis. This enables rapid data sharing and real-time display of diagnosis results, and patient feedback is also quickly reflected in the system, thereby improving diagnostic accuracy.

[1076] A "medical institution" refers to a facility that performs medical procedures, such as a hospital, clinic, or medical practice.

[1077] "Treatment data" refers to information about a patient's treatment that medical staff record during the course of their medical care. Examples include the details of drug treatment and information about prescribed medications.

[1078] "Test data" refers to data from tests performed to check a patient's health status. Examples include the results of blood tests and X-ray examinations.

[1079] "Anonymization" is the process of removing personally identifiable information (such as name, address, and contact information) from data so that individuals cannot be identified.

[1080] A "database" is a system used to store and manage collected treatment and examination data.

[1081] A "generative artificial intelligence model" is an AI model that learns from large amounts of data and has the ability to recognize patterns.

[1082] "Medical condition" refers to the symptoms a patient is exhibiting or their current health status.

[1083] "Diagnosis result" refers to the analysis results regarding the medical condition calculated by the generative artificial intelligence model, and represents the conclusion of the diagnosis.

[1084] A "terminal" is a device used by medical staff or patients, and examples include smartphones, smart glasses, and head-mounted displays.

[1085] A "virtual avatar" is a digital representation of a patient in a virtual space.

[1086] "Real-time" refers to a state where data collection, processing, and display occur instantly.

[1087] "Feedback" refers to the process of returning information about treatment effectiveness and patient response obtained after diagnosis back to the system.

[1088] "User interface" refers to the screens and methods of operation that users use to operate a system or input data.

[1089] The overall system configuration and each processing procedure for implementing this invention are described below. The system consists of three elements: a server, a terminal, and a user.

[1090] Server program processing

[1091] 1. Data collection and anonymization

[1092] The server periodically collects treatment and test data from each medical institution. During this process, it anonymizes the data by removing patients' personal information. For example, a Python script is used to remove names and addresses.

[1093] 2. Storing in a database

[1094] The collected anonymized data is categorized into diagnostic data, test data, treatment data, etc., and stored in databases such as Amazon RDS.

[1095] 3. Training of generative artificial intelligence models

[1096] The server periodically trains a generative artificial intelligence model (using TensorFlow) based on the information in the database. This improves the accuracy of diagnostics based on the latest data.

[1097] 4. Integration with research paper databases

[1098] The server periodically retrieves the latest medical papers from domestic and international medical journal databases, and then machine-translates and analyzes them. For example, it uses the Google Translation API to perform natural language processing, and then re-inputs this analyzed data into a generative artificial intelligence model for retraining.

[1099] Terminal program processing

[1100] 1. Providing a user interface

[1101] The devices (such as smart glasses, head-mounted displays, and smartphones) provide an intuitive user interface that allows medical staff to easily input data and quickly review diagnostic results. For example, this can be achieved through cross-platform applications developed using React Native.

[1102] 2. Creating a virtual avatar for the patient

[1103] The terminal application provides a function that allows patients to create virtual avatars and have those avatars interact with medical staff. The virtual avatars are implemented using Unity or Unreal Engine.

[1104] 3. Data entry and sharing

[1105] Patient symptoms and test results are entered in real time via a terminal and sent to the server. Real-time input and sharing allow medical staff to access the data immediately.

[1106] 4. Real-time display of diagnostic results

[1107] The diagnostic results and recommended treatments calculated by the generative artificial intelligence model are displayed on the device in real time. For example, the diagnostic results are projected onto smart glasses.

[1108] 5. Feedback function

[1109] After diagnosis, patients input feedback about the effectiveness of treatment via a terminal and send it to a server. This feedback is also used to retrain the generative artificial intelligence model.

[1110] User program processing

[1111] 1. Data entry

[1112] Medical staff and patients, who are the users, input medical data and test results into the terminal. The entered data is immediately sent to the server and processing begins.

[1113] 2. Input of feedback

[1114] Medical staff and patients, acting as users, input feedback on the treatments they received after a diagnosis and their effectiveness, and send this feedback to the server. This collected feedback data is then used to retrain the AI ​​model.

[1115] Examples of prompt statements

[1116] Patient's symptoms: fever of 37.8°C, sore throat, cough

[1117] Past medical history: Influenza, allergies

[1118] ---

[1119] Body temperature measurement result: 37.8°C

[1120] Blood pressure measurement result: 120 / 80

[1121] Pulse rate: 88 bpm

[1122] ---

[1123] Questions for generative AI models:

[1124] What disease might be suspected based on these symptoms?

[1125] What treatments and medications are recommended?

[1126] By inputting such prompts, the generative artificial intelligence model provides accurate diagnostic results and optimal treatment methods.

[1127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1128] Step 1:

[1129] The server collects treatment and test data from each medical institution. During collection, patient personal information is removed and anonymized. For example, a Python script is used to remove personally identifiable information such as names and addresses.

[1130] Input: Raw treatment and test data provided by each medical institution.

[1131] Output: Anonymized treatment and laboratory data.

[1132] Step 2:

[1133] The server stores anonymized treatment and test data in a database. Here, the data is categorized into diagnostic data, test data, treatment data, etc. Amazon RDS or a similar database is recommended.

[1134] Input: Anonymized treatment and laboratory data.

[1135] Output: Categorized data stored in the database.

[1136] Step 3:

[1137] The server periodically trains a generative artificial intelligence model using treatment and examination data stored in the database. The model is updated with the latest data using AI frameworks such as TensorFlow.

[1138] Input: Treatment data and test data stored in the database.

[1139] Output: A trained generative artificial intelligence model.

[1140] Step 4:

[1141] The server periodically retrieves the latest medical papers from domestic and international medical paper databases. This paper data is machine translated using the Google Translation API and other tools, and then analyzed using natural language processing techniques. The analysis results are then added to a generative artificial intelligence model for further training.

[1142] Input: Latest medical research paper data.

[1143] Output: A retrained generative artificial intelligence model.

[1144] Step 5:

[1145] The terminal provides medical staff with a user interface for data entry. Using this interface, medical staff can input patient symptoms and test results, and the data is sent to the server. For example, React Native can be used to enable intuitive operation.

[1146] Input: Patient symptoms and test results entered by medical staff.

[1147] Output: Medical data sent to the server.

[1148] Step 6:

[1149] The device creates a virtual avatar of the patient. Using Unity or Unreal Engine, it generates a patient avatar in a virtual space, enabling interaction with medical staff.

[1150] Input: Basic patient information and image data.

[1151] Output: A virtual avatar displayed in a virtual space.

[1152] Step 7:

[1153] The terminal shares the patient's symptoms and test results entered into it with the server in real time. This allows medical staff to access the data immediately.

[1154] Input: Patient's symptoms and test results.

[1155] Output: Data sent to the server in real time.

[1156] Step 8:

[1157] The terminal displays diagnostic results and recommended treatments calculated by a generative artificial intelligence model in real time. The diagnostic results are projected using devices such as smart glasses, and medical staff determine treatment plans based on this information.

[1158] Input: Diagnostic results calculated by a generative artificial intelligence model.

[1159] Output: Diagnostic results projected onto the terminal in real time.

[1160] Step 9:

[1161] After the diagnosis, the user inputs feedback on the treatment's effectiveness into the device and sends it to the server. This allows the generative artificial intelligence model to retrain, improving the accuracy of the diagnosis.

[1162] Input: User-submitted feedback regarding treatment effectiveness.

[1163] Output: Feedback data sent to the server.

[1164] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1165] Modes for carrying out the invention

[1166] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to train on this data, and provides accurate diagnostic results for each medical condition. By further incorporating an emotion engine that recognizes the user's emotions into this system, the method of providing diagnostic results can be adjusted based on the user's emotional state.

[1167] Server program processing

[1168] 1. Data collection and anonymization:

[1169] The server establishes secure communication protocols with each medical institution and regularly collects treatment and test data. During this process, patient personal information is removed and anonymized.

[1170] Specific example: Retrieve patient test results and treatment records from a hospital server, and anonymize them by removing personal information.

[1171] 2. Storing in the database:

[1172] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[1173] Specific example: Classify inspection data into appropriate tables and store them in a database.

[1174] 3. Training of generative artificial intelligence models:

[1175] The server uses data extracted from the database to train a generative artificial intelligence model. This improves the accuracy of diagnosing medical conditions.

[1176] Specific example: Retrain a generative artificial intelligence model using new treatment data to improve diagnostic accuracy.

[1177] 4. Integration with research paper databases:

[1178] The server periodically retrieves the latest research papers from an external database, translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[1179] Specific example: Obtain new papers on heart disease from PubMed, translate and analyze them using natural language processing techniques, and add them to the model.

[1180] Terminal program processing

[1181] 1. Providing a user interface:

[1182] The terminal provides a user interface that is easy for medical staff to use. This interface is designed to be intuitive to operate.

[1183] Specific example: Using the React framework, build an input form for entering patient medical data and a dashboard to display diagnostic results.

[1184] 2. How the emotion engine works:

[1185] While the user is reviewing the diagnostic results, the device activates an emotion engine to analyze the user's emotional state based on facial expression data, voice tone, and other factors.

[1186] Specific example: Using the device's camera and microphone, facial expression analysis software is run to determine in real time whether the user is surprised, relieved, or otherwise experiencing other emotions.

[1187] 3. Display of emotionally-based diagnostic results:

[1188] The device adjusts how the diagnostic results are displayed based on emotional data analyzed by the emotion engine. For example, if the user is feeling anxious, it will provide additional explanations or step-by-step information.

[1189] Specific example: If the system determines that the user is surprised when the diagnostic results are displayed, a detailed explanation of the diagnostic results will be displayed in a pop-up window.

[1190] User program processing

[1191] 1. Entering medical data:

[1192] The medical staff, acting as users, enter patient medical data into the input fields on the terminal and send it to the server.

[1193] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form.

[1194] 2. Confirmation of the diagnosis results:

[1195] The medical staff, as users, review the diagnostic results displayed on the terminal and consider the emotional analysis results from the emotion engine when deciding how to explain things to the patient.

[1196] Specific example: A doctor reviews a diagnosis result displayed by a generative artificial intelligence model, which indicates "possible pneumonia," and carefully explains it to a patient who appears anxious.

[1197] 3. Entering feedback:

[1198] Medical staff, acting as users, input feedback on the treatments performed after the diagnosis and their effectiveness into a terminal and send it to the server. The transmitted feedback data is then used again as training data for a generative artificial intelligence model.

[1199] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[1200] This series of processing steps not only improves diagnostic accuracy in medical settings but also enables flexible responses based on user emotions, providing a better patient experience.

[1201] The following describes the processing flow.

[1202] Modes for carrying out the invention

[1203] Server program processing

[1204] Step 1: Data Collection and Anonymization

[1205] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient identification information is removed and the data is anonymized.

[1206] Example: Retrieve patient test results and treatment records from the hospital server, and anonymize them by removing personal information.

[1207] Step 2: Data reception and verification

[1208] The server analyzes the received data and verifies whether the data format is correct and consistent.

[1209] Example: Check if JSON data has the correct structure and log any problems in the error log.

[1210] Step 3: Data anonymization

[1211] The server removes personal information from the received data and performs anonymization.

[1212] Specific example: Remove patient names and addresses from data and generate a unique anonymous identifier.

[1213] Step 4: Data classification and storage

[1214] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[1215] Specific example: Classify inspection data into appropriate tables and store them in a database.

[1216] Step 5: Training a generative artificial intelligence model

[1217] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[1218] Specific example: Export data in CSV format, input it into a machine learning algorithm, and train the model.

[1219] Step 6: Integration with the research paper database

[1220] The server periodically retrieves the latest papers from external research paper databases and performs translation and analysis.

[1221] Specific example: Obtain new articles on heart disease from PubMed, translate them into Japanese using machine translation, and then analyze them.

[1222] Step 7: Retrain the model

[1223] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[1224] Specific example: Retrain the model with a dataset that includes the latest research paper data to improve diagnostic accuracy.

[1225] Terminal program processing

[1226] Step 1: Provide the user interface

[1227] The terminal provides input forms and dashboards that medical staff can easily operate.

[1228] Specific example: Using the React framework, build a medical data input form and a diagnostic results display dashboard.

[1229] Step 2: Operating the Emotion Engine

[1230] While the user is reviewing the diagnostic results, the device operates an emotion engine, analyzing facial expression data and voice tone.

[1231] Specific example: Using the device's camera and microphone, facial expression analysis software is used to determine the user's emotional state in real time.

[1232] Step 3: Entering medical data

[1233] Medical staff, who are the users, input patient medical data into their terminals and send it to the server.

[1234] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form and sends a diagnostic request to the server.

[1235] Step 4: Processing diagnostic requests

[1236] The server uses a generative artificial intelligence model based on the input medical data to calculate the diagnosis.

[1237] Specific example: A generative artificial intelligence model outputs a diagnosis result stating "high suspicion of infectious disease," and sends that result to the terminal in JSON format.

[1238] Step 5: Displaying the diagnostic results

[1239] The terminal displays the diagnostic results received from the server. Additional analysis results from the emotion engine are also displayed simultaneously.

[1240] Specific example: Along with the diagnostic results, a pop-up message or additional explanation tailored to the user's emotional state will be displayed.

[1241] User program processing

[1242] Step 1: Confirming the emotionally-based diagnostic results

[1243] The medical staff, who are the users, review the diagnostic results and the analysis results of the emotion engine, and then explain them to the patient based on that information.

[1244] Specific example: The system carefully explains to the user the diagnosis that "there is a high possibility of pneumonia" and the emotional information that the user has identified as "anxiety."

[1245] Step 2: Entering emotionally-based feedback

[1246] After the diagnosis, the medical staff, acting as users, input the patient's response and treatment results as feedback into the terminal and send it to the server.

[1247] Specific example: The user inputs information about the patient's condition and post-treatment follow-up, and sends this information, along with the results of the emotion engine's analysis indicating "satisfied," to the server.

[1248] This series of processing steps not only improves diagnostic accuracy in medical settings, but also enables flexible responses based on user emotions, thereby providing a better patient experience.

[1249] (Example 2)

[1250] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1251] Traditional medical diagnostic systems required manual collection and anonymization of treatment and test data, which was time-consuming and labor-intensive. Furthermore, there was a lack of systems capable of efficiently learning from the collected data and providing rapid and accurate diagnostic results. Additionally, the presentation of diagnostic results was not tailored to the patient's emotional state, resulting in a poor patient experience.

[1252] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, and means for analyzing the user's emotions on the terminal and adjusting the method of presenting the diagnosis result based on the emotion result. This enables efficient collection and anonymization of treatment data and examination data, rapid and accurate diagnosis, and presentation of diagnosis results that respond to the patient's emotions.

[1253] "Treatment data" refers to information about treatment provided by a medical institution, including details such as the patient's treatment plan, treatment progress, and medications used.

[1254] "Test data" refers to information that includes the results of various tests performed on a patient, and includes data such as blood tests, urine tests, and imaging diagnostics.

[1255] "Anonymization" is the process of removing or concealing information that identifies an individual, and is a means of handling data securely while protecting patient privacy.

[1256] A "database" is a system for organizing and storing collected treatment and examination data, and is a collection of information that enables efficient data retrieval and management.

[1257] A "generative artificial intelligence model" is an artificial intelligence system that uses collected and learned data to perform tasks such as prediction and classification, equipped with algorithms and data structures.

[1258] "Diagnosis results" refer to information that shows judgments and conclusions about the patient's condition calculated by a generative artificial intelligence model based on collected treatment and examination data.

[1259] A "terminal" refers to a device such as a computer or tablet operated by medical staff, used for data input and display, sentiment analysis, and other similar tasks.

[1260] "Emotional analysis" is a technology that identifies a user's emotional state from their facial expressions, voice tone, and behavior, and is a means of understanding the user's psychological state.

[1261] A "user interface" refers to the interface, including screens and input forms, that medical staff use to access and operate a system through their terminals.

[1262] This invention relates to a system that provides diagnostic results by training a generative artificial intelligence model based on treatment and examination data collected from medical institutions. This system consists of three main components: a server, a terminal, and a user.

[1263] Server Processing

[1264] The server connects to each healthcare facility using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. This data is first anonymized by removing or concealing personal information. For example, personally identifiable information such as names and addresses in the collected data is replaced with "anonymous IDs."

[1265] Anonymized data is categorized into "diagnostic data," "test data," and "treatment data," and stored in a database. This database enables efficient data management and retrieval.

[1266] Next, a generative artificial intelligence model is trained using data extracted from this database. During this training process, new treatment data is added to the training set, updating the existing model. For example, retraining the model using a backpropagation algorithm can improve the accuracy of disease diagnosis.

[1267] Furthermore, the server periodically retrieves the latest research papers from external academic databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). As a result, the paper data is also incorporated into the generative artificial intelligence model for retraining.

[1268] Terminal processing

[1269] The terminal provides a user interface that is easy for medical staff to use. For example, patient information input forms and diagnostic result dashboards are built using the React framework. Users can easily input data and check diagnostic results by simply operating the system.

[1270] When displaying diagnostic results, the device captures the user's facial expressions and voice tone in real time and analyzes the user's psychological state using emotion analysis algorithms (e.g., OpenFace or IBM Watson Tone Analyzer). Once the user's emotional state, such as surprise or anxiety, is identified, the way the diagnostic results are presented is adjusted accordingly. For example, if the user is feeling anxious, a pop-up window with additional detailed explanations will be displayed.

[1271] User processing

[1272] Medical staff, as users, enter patient symptoms and medical data into input fields on their terminals and send them to the server. For example, they might enter a patient's fever or cough into an input form and submit it.

[1273] When the diagnostic results are displayed on the terminal, the emotional analysis results are also reviewed. Medical staff explain the results to the patient based on the diagnostic results presented by the generative artificial intelligence model. In doing so, the patient's emotional state is also taken into consideration to provide a more appropriate explanation.

[1274] Feedback regarding treatment results and their effectiveness is also entered into the terminal and sent to the server. The transmitted feedback data is used to retrain generative artificial intelligence models, contributing to improved diagnostic accuracy in the future.

[1275] Example of a prompt

[1276] Use the following prompts to input data into the generative AI model:

[1277] Please retrain the diagnostic accuracy for cardiac disease using recently collected treatment data. Please also incorporate the latest research papers.

[1278] This will not only improve the accuracy of diagnoses in medical settings, but also enable flexible responses based on user emotions, thereby providing a better patient experience.

[1279] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1280] Step 1:

[1281] Data collection and anonymization

[1282] The server connects with each healthcare institution using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. It receives treatment and test data sent from healthcare institutions as input. Because this data contains patient personal information, it is first anonymized by removing personal information (name, ID, address, etc.). Specifically, it scans each field of the collected data and replaces personal information with meaningless data such as an "anonymous ID." The output is the anonymized treatment and test data.

[1283] Step 2:

[1284] Storage in database

[1285] The server categorizes anonymized data into categories such as "diagnostic data," "test data," and "treatment data." It receives anonymized treatment and test data as input. This data is then organized by category for storage in the appropriate database tables. For example, if anonymized test data is collected, it is organized and stored in the "test data" table. The output is an efficiently organized database.

[1286] Step 3:

[1287] Learning of generative artificial intelligence models

[1288] The server extracts relevant data from the database and trains a generative artificial intelligence model. It extracts "diagnostic data," "test data," and "treatment data" from the database as input. Based on this data, it uses the generative AI model to train and improve the accuracy of disease diagnosis. Specifically, it adds new treatment data to the model's training set and uses a backpropagation algorithm to update the existing model. The output is the trained generative AI model.

[1289] Step 4:

[1290] Integration with research paper databases

[1291] The server periodically retrieves the latest research papers from external research paper databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). It receives paper data retrieved from external research paper databases as input. The retrieved paper text is passed through translation and analysis tools to convert it into a format suitable for input into a generative artificial intelligence model. Specifically, it uses the PubMed API to perform keyword searches, downloads new papers, and performs translation and content analysis using natural language processing techniques. The output is the analyzed paper data used for retraining.

[1292] Step 5:

[1293] Providing a user interface

[1294] The terminal provides an interface that is easy for medical staff to use. It receives data for medical staff to input patient information as input. Using the React framework, a patient information input form and a dashboard displaying diagnostic results are built. Specifically, the input form and dashboard are designed and implemented to allow users to operate them intuitively. The output is a user-friendly interface.

[1295] Step 6:

[1296] How the emotion engine works

[1297] The device operates an emotion engine while the user is reviewing the diagnostic results, analyzing the user's facial expressions and voice tone. It receives user facial and voice data acquired using the device's camera and microphone as input. This data is analyzed using an emotion analysis algorithm (e.g., OpenFace or IBM Watson Tone Analyzer) to identify the user's emotional state. Specifically, the camera captures facial expressions, which are then analyzed in real time by the algorithm. The output provides data regarding the user's emotional state.

[1298] Step 7:

[1299] Display of emotion-based diagnostic results

[1300] The device adjusts how the diagnostic results are presented based on data analyzed by the emotion engine. It receives the analysis results from the emotion engine as input. It selects an appropriate method for displaying the diagnostic results based on the analysis results, such as when the user is feeling anxious. Specifically, if the user is determined to be anxious, a pop-up window containing a detailed explanation is displayed. The output is a display of the diagnostic results adapted to the user.

[1301] Step 8:

[1302] Entering medical data

[1303] The medical staff, acting as users, input patient symptoms and medical data into the terminal's input fields. For input, they enter medical data such as the patient's fever or cough into the input form. The terminal then sends this data to the server. Specifically, when the medical staff enters data into the input form and presses the submit button, the input data is sent to the server. The output is the medical data sent to the server.

[1304] Step 9:

[1305] Checking the diagnosis results

[1306] The medical staff, acting as users, review the diagnostic results displayed on the terminal and provide explanations to the patient, taking into account the emotional analysis results from the emotion engine. The input is the diagnostic results and emotional analysis results displayed on the terminal. Based on this data, the medical staff provides the most appropriate explanation to the patient. Specifically, they review the displayed diagnostic results and emotional analysis results, paying particular attention to explanations if the patient appears anxious. The output is the most appropriate explanation for the patient.

[1307] Step 10:

[1308] Feedback Input

[1309] Medical staff, acting as users, input feedback on treatment and its effectiveness after a diagnosis into a terminal and send it to the server. The input consists of data on treatment effectiveness and patient response entered into an input form. The terminal then sends this feedback data to the server. Specifically, when medical staff enter data into the feedback form and press the submit button, the input data is sent to the server. The output is the feedback data sent to the server. This data is then used to retrain generative artificial intelligence models, improving diagnostic accuracy.

[1310] (Application Example 2)

[1311] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1312] In modern factories, accurately diagnosing machine malfunctions and maintenance needs is difficult. Furthermore, when communicating fault diagnosis results to workers, the information provided is often difficult to understand, leading to anxiety and misunderstandings. A system is needed to solve these problems and enable efficient and safe work processes.

[1313] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, means for analyzing the user's emotions using the terminal's camera and microphone, and means for adjusting the method of presenting the diagnosis result based on the analyzed emotion data. As a result, it is possible to calculate a diagnosis result using an AI model based on collected machine operation data and failure data, and to perform emotion analysis of workers using an emotion engine, thereby providing a diagnosis result in an easy-to-understand format.

[1314] "Treatment data" refers to data from a medical institution regarding the process and methods of treatment for a patient's medical condition.

[1315] "Test data" refers to data related to the results of various tests conducted at a medical institution to diagnose a patient's medical condition.

[1316] "Anonymization" is the process of removing information that can identify an individual, so that the individual cannot be identified.

[1317] A "generative artificial intelligence model" is a type of artificial intelligence that learns from data, generates new information, and provides predictions and diagnostic results.

[1318] "Learning" is the process by which a generative artificial intelligence model uses provided data to improve its prediction accuracy and performance.

[1319] A "diagnosis result" is a judgment or conclusion about a medical condition calculated by a generative artificial intelligence model based on the provided data.

[1320] A "terminal" is a device that provides a user interface, allowing medical staff to input medical data and review diagnostic results.

[1321] "Camera and microphone" refers to devices installed in a terminal for capturing video and audio.

[1322] Analyzing "emotions" involves determining the user's emotional state based on their facial expressions and voice, using data acquired from the camera and microphone.

[1323] Adjusting the "presentation method" means changing the way the diagnostic results are displayed and the content of those results to suit the user, based on the analyzed emotional data.

[1324] "Machine operation data" refers to data regarding the operating status of various machines in a factory.

[1325] "Failure data" refers to information such as the state of a machine and error logs when it malfunctions in a factory.

[1326] System Overview

[1327] This invention is a system that collects treatment and examination data anonymized from various medical institutions, trains a generative artificial intelligence model on this data, and provides accurate diagnostic results. Furthermore, it has a function that analyzes the user's emotions using the terminal's camera and microphone, and adjusts the method of presenting the diagnostic results based on the analysis results.

[1328] Hardware and software used

[1329] The following hardware and software are used for the server and terminal:

[1330] 1. Server: Performs data collection, anonymization, storage in a database, training of generative artificial intelligence models, and calculation of diagnostic results.

[1331] 2. Camera and microphone: Capture the user's facial expressions and voice in real time and use them for emotion analysis.

[1332] 3. Database: Stores treatment data and test data.

[1333] 4. Generative artificial intelligence models: These models calculate diagnostic results based on data.

[1334] 5. Emotion Analysis Engine: Analyzes the user's emotions using data acquired from the camera and microphone.

[1335] 6. User Interface (UI): This interface handles the input of medical data, the display of diagnostic results, and the presentation of sentiment analysis results.

[1336] Specific system configuration

[1337] Data collection and anonymization

[1338] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. For example, patient test results and treatment records are retrieved from the hospital server, personal information is removed, and then the data is stored in the database.

[1339] Storage in database

[1340] The collected treatment and examination data are stored in a database on the server. This database is categorized into categories such as "diagnostic data," "examination data," and "treatment data" for efficient management. For example, vibration data and temperature data are classified into appropriate tables and stored in the database.

[1341] Training and diagnostics of generative artificial intelligence models

[1342] The server uses data extracted from the database to train a generative artificial intelligence model. This improves the accuracy of diagnosing medical conditions. Furthermore, it periodically retrieves the latest research paper data to retrain the model. For example, it translates the latest papers from PubMed using natural language processing techniques and adds the analysis results to the training.

[1343] Emotional analysis and presentation of diagnostic results

[1344] The device uses its built-in camera and microphone to analyze the user's emotional state in real time. The emotion analysis engine recognizes emotions from the user's facial expressions and voice, and adjusts how the diagnostic results are displayed. For example, if the system determines that the user is surprised when the diagnostic results are displayed, a detailed explanation will be shown in a pop-up window.

[1345] Specific examples and prompt statements

[1346] Specific example:

[1347] Machine vibration and temperature data are collected, and an AI model predicts the "possibility of bearing wear." If the emotional engine detects that the worker is startled, a pop-up will display photos of bearing wear symptoms and a video explaining the replacement procedure.

[1348] Example of a prompt:

[1349] Collect failure data, analyze it using an AI model, and display the judgment results. Also, analyze the emotions of the workers using cameras. Based on the obtained emotion data, adjust the presentation method of the judgment results.

[1350] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1351] Step 1:

[1352] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. The input consists of treatment and test data received from medical institutions, and the output is anonymized data. Specific operations include data format conversion and removal of personal information fields.

[1353] Step 2:

[1354] The server stores the collected anonymized data in a database. The input consists of anonymized treatment and examination data, and the output is data categorized and stored within the database. Specifically, the server performs insertion operations into the database tables.

[1355] Step 3:

[1356] The server trains a generative artificial intelligence model using treatment and examination data stored in a database. The input is training data extracted from the database, and the output is the trained generative AI model. Specifically, the server performs tasks such as updating model parameters and applying the learning algorithm.

[1357] Step 4:

[1358] The server uses a generative artificial intelligence model to calculate a diagnosis for a medical condition. New treatment and test data are used as input, and the output is a diagnosis. Specifically, the server inputs data into a pre-trained model and outputs the diagnosis.

[1359] Step 5:

[1360] The server displays the diagnostic results on the terminal. The input is the calculated diagnostic results, and the output is the diagnostic results displayed on the terminal screen. Specific actions include updating UI components and visualizing the data.

[1361] Step 6:

[1362] The device analyzes the user's emotions in real time using a camera and microphone. Inputs include video and audio data acquired from the camera and microphone, while output is user emotion data. Specifically, it applies facial recognition algorithms and voice analysis algorithms.

[1363] Step 7:

[1364] The device adjusts how the diagnostic results are presented based on the analyzed emotional data. Inputs include the user's emotional data and diagnostic results, and output is a display of the adjusted diagnostic results. Specific actions include changing the display method of the diagnostic results and displaying pop-ups with additional explanations.

[1365] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1366] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1367] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1368] [Fourth Embodiment]

[1369] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1370] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1371] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1372] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1373] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1374] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1375] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1376] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1377] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1378] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1379] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1380] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1381] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1382] Modes for carrying out the invention

[1383] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to learn from this data, and provides accurate diagnostic results for each medical condition. This system allows medical staff to quickly and accurately obtain personalized diagnostic information for each patient, enabling appropriate treatment and drug administration.

[1384] Server program processing

[1385] 1. Data collection and anonymization

[1386] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information (name, address, contact information, etc.) is removed and anonymized.

[1387] Specific example: Obtain "Patient X's blood test results" and "blood pressure measurement results" from Hospital A's server, and anonymize this data by removing names and addresses.

[1388] 2. Storing in a database

[1389] The server categorizes the anonymized data (diagnostic data, test data, treatment data, etc.) and stores it in a database. This database is designed to allow high-speed access to large amounts of data.

[1390] Specific example: Store blood test results and blood pressure measurement results in the "Test Data" table of the database.

[1391] 3. Training of generative artificial intelligence models

[1392] The server periodically uses database information to train generative artificial intelligence models. This improves the accuracy of disease diagnosis based on the latest treatment and examination data.

[1393] Specific example: Retrain a generative artificial intelligence model using newly added blood test data to improve the accuracy of diabetes diagnosis.

[1394] 4. Integration with research paper databases

[1395] The server periodically retrieves the latest medical papers from domestic and international journal databases, machine-translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[1396] Specific example: Obtain the latest papers on heart disease from PubMed and translate them into Japanese using natural language processing technology. Then, add the analysis results to the model and retrain it.

[1397] Terminal program processing

[1398] 1. Providing a user interface

[1399] The terminal provides a user interface that allows medical staff to easily input data and review diagnostic results. This interface is designed to be intuitive to use.

[1400] Specific example: Provide an input field for doctors to enter patient blood pressure measurement results and a dashboard to display the diagnostic results.

[1401] 2. Display of diagnostic results

[1402] The terminal displays a diagnostic result calculated by a generative artificial intelligence model based on the entered medical data. Medical staff then decide on a treatment plan based on this result.

[1403] Specific example: When a doctor inputs a patient's symptoms, a generative artificial intelligence model determines that "influenza is highly likely" and displays a list of appropriate treatments.

[1404] User program processing

[1405] 1. Data entry

[1406] The medical staff, acting as users, input patient medical data into the terminal. The entered data is sent to the server, and processing begins.

[1407] Specific example: A nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form.

[1408] 2. Input of feedback

[1409] Medical staff, acting as users, input feedback on the treatments they administered after the diagnosis and their effectiveness, and send this feedback to the server. This submitted feedback data is then used again as training data for a generative artificial intelligence model.

[1410] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[1411] In this way, this system dramatically improves diagnostic accuracy in medical settings and enables the provision of optimal treatment plans for each patient.

[1412] The following describes the processing flow.

[1413] Program processing flow

[1414] Server program processing

[1415] Step 1: Data Collection and Anonymization

[1416] The server establishes a secure communication protocol (e.g., HTTPS) with each medical institution and periodically requests treatment and test data.

[1417] Example: A server sends a request to a hospital server via an API to retrieve patient test results and treatment records.

[1418] Step 2: Data reception and verification

[1419] The server analyzes the received data and automatically checks for format consistency and the presence of required fields.

[1420] Example: Verify that JSON data has the correct structure and log any errors in the error log.

[1421] Step 3: Data anonymization

[1422] The server removes personal information from incoming data, anonymizing it to protect patient privacy.

[1423] Example: Remove patient names, addresses, and contact information from the data, and randomly generate identifiers.

[1424] Step 4: Data classification and storage

[1425] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[1426] Example: Use an SQL statement to insert the test results into the test data table.

[1427] Step 5: Training a generative artificial intelligence model

[1428] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[1429] Example: Export data in CSV format and input it into a machine learning algorithm to train a neural network.

[1430] Step 6: Integration with the research paper database

[1431] The server periodically retrieves new papers from external academic databases and performs translation and text analysis.

[1432] Example: Set up a scheduled job to download papers from PubMed via API and then process them for machine translation.

[1433] Step 7: Retrain the model

[1434] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[1435] Example: Retrain the model using a dataset that includes the latest research paper data.

[1436] Terminal program processing

[1437] Step 1: Provide the user interface

[1438] The terminal provides input forms and dashboards that medical staff can easily operate.

[1439] Example: Develop a test result input form and a diagnostic results dashboard using the React framework.

[1440] Step 2: Enter diagnostic data

[1441] The medical staff, who are the users, enter the patient's medical data (symptoms, test results, etc.) into the input form on the terminal.

[1442] Example: A doctor enters the patient's temperature, blood pressure, and pulse rate into the input fields.

[1443] Step 3: Submit a diagnostic request

[1444] The terminal sends the entered data to the server and makes a diagnostic request.

[1445] Example: By pressing the submit button, the input data is sent to the server via the HTTPS protocol.

[1446] Step 4: Displaying the diagnostic results

[1447] The server uses a generative artificial intelligence model to calculate the diagnostic results and sends them back to the terminal.

[1448] Example: A generative artificial intelligence model calculates the "likelihood of influenza" and sends the result to the terminal in JSON format.

[1449] Step 5: Confirm and display the diagnostic results.

[1450] The terminal displays the diagnostic results returned from the server on the user interface.

[1451] Example: Display the diagnostic results on the dashboard so that doctors can review them.

[1452] User program processing

[1453] Step 1: Entering medical data

[1454] The medical staff, acting as users, enter the patient's medical data into the input fields on the terminal and then transmit it.

[1455] Example: A nurse copies test results obtained from the electronic medical record system into an input form.

[1456] Step 2: Confirmation of diagnostic results

[1457] The medical staff, who are the users, review the diagnostic results displayed on the terminal and decide on a treatment plan based on those results.

[1458] Example: The doctor checks the displayed diagnosis result, "Possible influenza," and prescribes antiviral medication.

[1459] Step 3: Entering Feedback

[1460] The medical staff, who are the users, input post-treatment feedback information into their terminals and send it to the server.

[1461] Example: As follow-up data after treatment, the patient's condition improvement and the presence or absence of side effects are entered and sent to the database.

[1462] This series of processing steps allows the system to improve diagnostic accuracy in medical settings and provide personalized treatment quickly.

[1463] (Example 1)

[1464] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1465] Conventional medical systems have been insufficient in anonymizing treatment and test data collected from each medical institution, and in improving diagnostic accuracy using generative artificial intelligence models. Furthermore, the lack of retraining utilizing the latest medical papers and feedback information has made it difficult to improve the accuracy of diagnostic results and provide prompt treatment.

[1466] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1467] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model using the treatment data and test data stored in the database; means for periodically acquiring the latest medical papers, machine translating and analyzing them; means for inputting the translated paper data into the generative artificial intelligence model for retraining; and means for medical staff to input feedback on treatment effectiveness and transmit it to the server. As a result, the generative artificial intelligence model continuously retrains using the collected data and the latest medical knowledge, enabling improved diagnostic accuracy and rapid treatment.

[1468] "Treatment data" refers to a series of pieces of information about the treatments provided to a patient at a medical institution, the medications prescribed, and the treatment outcomes.

[1469] "Test data" refers to information showing the results of various medical tests (blood tests, imaging tests, urine tests, etc.) performed on a patient.

[1470] "Anonymization" is the process of removing a patient's personal information (such as name, address, and contact information) so that a specific individual cannot be identified.

[1471] A "database" is a data aggregation system that efficiently stores collected treatment and examination data and manages it so that it can be quickly accessed as needed.

[1472] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates appropriate diagnostic results or predictions for new data.

[1473] A "user interface" is a software component that provides a visual screen layout and input methods for medical staff to operate a system.

[1474] "Diagnosis" refers to a professional evaluation and opinion regarding a patient's medical condition and symptoms, based on the analysis of collected data and generative artificial intelligence models.

[1475] "Feedback" refers to information that medical staff input regarding diagnostic results and treatment effectiveness, and is used to improve the accuracy and retrain generative artificial intelligence models.

[1476] "Paper data" refers to the latest medical information published in academic papers and research results, and is used as data for training generative artificial intelligence models.

[1477] "Machine translation" is a technology that automatically converts text written in one language into another language using a computer.

[1478] This invention relates to a system that collects anonymized treatment and examination data from various medical institutions, stores this data in a database, and then uses a generative artificial intelligence model to train the database and calculate a diagnosis for a patient's condition. This system enables medical staff to make quick and accurate diagnoses. Furthermore, the system aims to further improve diagnostic accuracy by retraining the model based on the latest medical papers and feedback information.

[1479] Server program processing

[1480] 1. Data collection and anonymization

[1481] The server periodically collects treatment and test data from each medical institution. During this process, patient personal information is removed and the data is anonymized. For example, the server sends an API request at 2 AM every day to request new data, analyzes the received data, and removes personal information. A dedicated data analysis engine is used for this process.

[1482] 2. Storing in a database

[1483] The server categorizes the anonymized data and stores it in the corresponding tables. The database is designed to allow high-speed access to large amounts of data, and database types such as MySQL are commonly used.

[1484] 3. Training of generative artificial intelligence models

[1485] The server uses periodically collected data to train a generative artificial intelligence model. For example, it checks the training schedule every Sunday night and retrains the model with new data. Machine learning frameworks such as TensorFlow are used in this process.

[1486] 4. Integration with research paper databases

[1487] The server regularly retrieves the latest medical papers from domestic and international research paper databases, machine-translates them, and analyzes them. For example, once a month, it uses the PubMed API to retrieve the latest papers on heart disease, translates them using the Google Translate API, analyzes them with a natural language processing engine, and inputs the results into a model.

[1488] Terminal program processing

[1489] 1. Providing a user interface

[1490] The terminal provides a user interface for medical staff to operate. This interface is intuitive and includes input fields such as blood pressure measurement results and a dashboard that displays diagnostic results.

[1491] 2. Display of diagnostic results

[1492] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. For example, when a doctor enters a patient's symptoms, the terminal sends a request to the server and displays a diagnostic result such as "high probability of influenza."

[1493] User program processing

[1494] 1. Data entry

[1495] Medical staff, who are the users, input patient medical data through a terminal. For example, a nurse enters the temperature, blood pressure, and pulse rate of a patient with a fever into a dedicated form, and the terminal transmits this information to the server in real time.

[1496] 2. Input of feedback

[1497] Medical staff, acting as users, input feedback on the effectiveness of treatments based on diagnostic results and patient responses. For example, doctors input feedback on the efficacy of prescribed medications and send it from their terminals to the server. The collected feedback is then used to retrain generative artificial intelligence models.

[1498] Specific example

[1499] For example, the following prompt statement may be used:

[1500] "Please enter the patient's blood pressure measurement results."

[1501] "The generative artificial intelligence model has determined that there is a possibility of diabetes."

[1502] This system can support improved diagnostic accuracy and faster decision-making regarding treatment plans in medical settings.

[1503] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1504] Step 1: Data Collection and Anonymization

[1505] The server periodically collects treatment and test data from each medical institution. Specifically, the server sends an API request at 2:00 AM every day to request new data. The data received from the medical institution's server is sent in JSON format. The server analyzes this data, removes personal information such as names, addresses, and contact information, and anonymizes it. The input is treatment and test data in JSON format from the medical institution, and the output is anonymized data from which personal information has been removed. The anonymized data is temporarily stored in storage.

[1506] Step 2: Storing in the database

[1507] The server reads anonymized data from temporary storage and categorizes it. Specifically, test data such as blood test results and blood pressure measurements are stored in the "Test Data" table, and treatment data such as treatment history and prescription information is stored in the "Treatment Data" table. This database uses MySQL or similar, enabling high-speed access. The input is anonymized data, and the output is structured data stored in the database. The server logs that the data has been correctly stored in the database.

[1508] Step 3: Training a generative artificial intelligence model

[1509] The server periodically retrieves new data from the database and retrains the generative artificial intelligence model. Specifically, it checks the training schedule at 3 AM every Sunday and retrains the model using data collected over the past week. Machine learning frameworks such as TensorFlow are used for this process. The input is new treatment and examination data, and the output is the updated generative artificial intelligence model. The model's accuracy is evaluated using test data, and if the accuracy meets the criteria, it is saved as a new model.

[1510] Step 4: Integration with the paper database

[1511] The server regularly retrieves the latest medical papers from domestic and international journal databases. Specifically, on the 1st of each month, it uses the PubMed API to retrieve the latest papers on heart disease. The retrieved paper data is machine-translated into Japanese using the Google Translate API. Then, it is analyzed using natural language processing techniques and converted into a format that can be input into a generative artificial intelligence model. The input is the paper data, and the output is new data that is added to the model.

[1512] Step 5: Provide the user interface

[1513] The terminal provides a user interface that allows medical staff to input data and review diagnostic results. Specifically, it includes input fields for doctors to enter patient blood pressure measurements and symptoms, and a dashboard to display diagnostic results. Input consists of medical data entered by medical staff, while output consists of data sent to the server and diagnostic results displayed on the terminal.

[1514] Step 6: Displaying the diagnostic results

[1515] The terminal displays diagnostic results calculated by a generative artificial intelligence model based on the entered medical data. Specifically, when a doctor enters the patient's symptoms, the terminal sends a request to the server to retrieve the diagnostic results. As a result, the dashboard displays a diagnostic result such as "high probability of influenza" and recommended treatment methods. The input is medical data, and the output is the diagnostic result.

[1516] Step 7: Data Entry

[1517] The medical staff, acting as users, input patient medical data into the terminal. Specifically, nurses enter the temperature, blood pressure, and pulse rate of patients with fever into a dedicated form, and this data is sent to the server. The input is the patient's medical data, and the output is the data sent to the server.

[1518] Step 8: Entering Feedback

[1519] Medical staff, acting as users, input feedback on the effectiveness of treatment and patient responses based on diagnostic results. Specifically, they input information on the efficacy of medications prescribed by doctors into a dedicated form and send it from their terminal to the server. The input is treatment feedback data, and the output is the data sent to the server. This feedback data is used to retrain generative artificial intelligence models.

[1520] (Application Example 1)

[1521] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1522] Conventional diagnostic support systems often required medical staff to manually input patient symptoms and test results, resulting in lengthy diagnostic timeliness. Furthermore, diagnostic results were not displayed in real time, and diagnostic accuracy was still insufficient. Additionally, there was a lack of direct feedback from patients regarding the effectiveness of post-diagnosis treatment, hindering effective retraining. To address these issues, there was a need for real-time data sharing and rapid delivery of diagnostic results.

[1523] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1524] In this invention, the server includes means for anonymizing and collecting treatment data and test data from each medical institution; means for storing the collected treatment data and test data in a database; means for training a generative artificial intelligence model on the treatment data and test data stored in the database; means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model; means for displaying the diagnosis result corresponding to the medical condition on a terminal; means for creating a virtual avatar of the patient and enabling interaction with medical staff; means for inputting the patient's symptoms and test results in real time and sharing them with medical staff; means for displaying the diagnosis result and treatment method in real time; and means for the patient to input feedback on the effectiveness of treatment after diagnosis. This enables rapid data sharing and real-time display of diagnosis results, and patient feedback is also quickly reflected in the system, thereby improving diagnostic accuracy.

[1525] A "medical institution" refers to a facility that performs medical procedures, such as a hospital, clinic, or medical practice.

[1526] "Treatment data" refers to information about a patient's treatment that medical staff record during the course of their medical care. Examples include the details of drug treatment and information about prescribed medications.

[1527] "Test data" refers to data from tests performed to check a patient's health status. Examples include the results of blood tests and X-ray examinations.

[1528] "Anonymization" is the process of removing personally identifiable information (such as name, address, and contact information) from data so that individuals cannot be identified.

[1529] A "database" is a system used to store and manage collected treatment and examination data.

[1530] A "generative artificial intelligence model" is an AI model that learns from large amounts of data and has the ability to recognize patterns.

[1531] "Medical condition" refers to the symptoms a patient is exhibiting or their current health status.

[1532] "Diagnosis result" refers to the analysis results regarding the medical condition calculated by the generative artificial intelligence model, and represents the conclusion of the diagnosis.

[1533] A "terminal" is a device used by medical staff or patients, and examples include smartphones, smart glasses, and head-mounted displays.

[1534] A "virtual avatar" is a digital representation of a patient in a virtual space.

[1535] "Real-time" refers to a state where data collection, processing, and display occur instantly.

[1536] "Feedback" refers to the process of returning information about treatment effectiveness and patient response obtained after diagnosis back to the system.

[1537] "User interface" refers to the screens and methods of operation that users use to operate a system or input data.

[1538] The overall system configuration and each processing procedure for implementing this invention are described below. The system consists of three elements: a server, a terminal, and a user.

[1539] Server program processing

[1540] 1. Data collection and anonymization

[1541] The server periodically collects treatment and test data from each medical institution. During this process, it anonymizes the data by removing patients' personal information. For example, a Python script is used to remove names and addresses.

[1542] 2. Storing in a database

[1543] The collected anonymized data is categorized into diagnostic data, test data, treatment data, etc., and stored in databases such as Amazon RDS.

[1544] 3. Training of generative artificial intelligence models

[1545] The server periodically trains a generative artificial intelligence model (using TensorFlow) based on the information in the database. This improves the accuracy of diagnostics based on the latest data.

[1546] 4. Integration with research paper databases

[1547] The server periodically retrieves the latest medical papers from domestic and international medical journal databases, and then machine-translates and analyzes them. For example, it uses the Google Translation API to perform natural language processing, and then re-inputs this analyzed data into a generative artificial intelligence model for retraining.

[1548] Terminal program processing

[1549] 1. Providing a user interface

[1550] The devices (such as smart glasses, head-mounted displays, and smartphones) provide an intuitive user interface that allows medical staff to easily input data and quickly review diagnostic results. For example, this can be achieved through cross-platform applications developed using React Native.

[1551] 2. Creating a virtual avatar for the patient

[1552] The terminal application provides a function that allows patients to create virtual avatars and have those avatars interact with medical staff. The virtual avatars are implemented using Unity or Unreal Engine.

[1553] 3. Data entry and sharing

[1554] Patient symptoms and test results are entered in real time via a terminal and sent to the server. Real-time input and sharing allow medical staff to access the data immediately.

[1555] 4. Real-time display of diagnostic results

[1556] The diagnostic results and recommended treatments calculated by the generative artificial intelligence model are displayed on the device in real time. For example, the diagnostic results are projected onto smart glasses.

[1557] 5. Feedback function

[1558] After diagnosis, patients input feedback about the effectiveness of treatment via a terminal and send it to a server. This feedback is also used to retrain the generative artificial intelligence model.

[1559] User program processing

[1560] 1. Data entry

[1561] Medical staff and patients, who are the users, input medical data and test results into the terminal. The entered data is immediately sent to the server and processing begins.

[1562] 2. Input of feedback

[1563] Medical staff and patients, acting as users, input feedback on the treatments they received after a diagnosis and their effectiveness, and send this feedback to the server. This collected feedback data is then used to retrain the AI ​​model.

[1564] Examples of prompt statements

[1565] Patient's symptoms: fever of 37.8°C, sore throat, cough

[1566] Past medical history: Influenza, allergies

[1567] ---

[1568] Body temperature measurement result: 37.8°C

[1569] Blood pressure measurement result: 120 / 80

[1570] Pulse rate: 88 bpm

[1571] ---

[1572] Questions for generative AI models:

[1573] What disease might be suspected based on these symptoms?

[1574] What treatments and medications are recommended?

[1575] By inputting such prompts, the generative artificial intelligence model provides accurate diagnostic results and optimal treatment methods.

[1576] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1577] Step 1:

[1578] The server collects treatment and test data from each medical institution. During collection, patient personal information is removed and anonymized. For example, a Python script is used to remove personally identifiable information such as names and addresses.

[1579] Input: Raw treatment and test data provided by each medical institution.

[1580] Output: Anonymized treatment and laboratory data.

[1581] Step 2:

[1582] The server stores anonymized treatment and test data in a database. Here, the data is categorized into diagnostic data, test data, treatment data, etc. Amazon RDS or a similar database is recommended.

[1583] Input: Anonymized treatment and laboratory data.

[1584] Output: Categorized data stored in the database.

[1585] Step 3:

[1586] The server periodically trains a generative artificial intelligence model using treatment and examination data stored in the database. The model is updated with the latest data using AI frameworks such as TensorFlow.

[1587] Input: Treatment data and test data stored in the database.

[1588] Output: A trained generative artificial intelligence model.

[1589] Step 4:

[1590] The server periodically retrieves the latest medical papers from domestic and international medical paper databases. This paper data is machine translated using the Google Translation API and other tools, and then analyzed using natural language processing techniques. The analysis results are then added to a generative artificial intelligence model for further training.

[1591] Input: Latest medical research paper data.

[1592] Output: A retrained generative artificial intelligence model.

[1593] Step 5:

[1594] The terminal provides medical staff with a user interface for data entry. Using this interface, medical staff can input patient symptoms and test results, and the data is sent to the server. For example, React Native can be used to enable intuitive operation.

[1595] Input: Patient symptoms and test results entered by medical staff.

[1596] Output: Medical data sent to the server.

[1597] Step 6:

[1598] The device creates a virtual avatar of the patient. Using Unity or Unreal Engine, it generates a patient avatar in a virtual space, enabling interaction with medical staff.

[1599] Input: Basic patient information and image data.

[1600] Output: A virtual avatar displayed in a virtual space.

[1601] Step 7:

[1602] The terminal shares the patient's symptoms and test results entered into it with the server in real time. This allows medical staff to access the data immediately.

[1603] Input: Patient's symptoms and test results.

[1604] Output: Data sent to the server in real time.

[1605] Step 8:

[1606] The terminal displays diagnostic results and recommended treatments calculated by a generative artificial intelligence model in real time. The diagnostic results are projected using devices such as smart glasses, and medical staff determine treatment plans based on this information.

[1607] Input: Diagnostic results calculated by a generative artificial intelligence model.

[1608] Output: Diagnostic results projected onto the terminal in real time.

[1609] Step 9:

[1610] After the diagnosis, the user inputs feedback on the treatment's effectiveness into the device and sends it to the server. This allows the generative artificial intelligence model to retrain, improving the accuracy of the diagnosis.

[1611] Input: User-submitted feedback regarding treatment effectiveness.

[1612] Output: Feedback data sent to the server.

[1613] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1614] Modes for carrying out the invention

[1615] This invention relates to a system that collects anonymized treatment and examination data held by medical institutions, uses a generative artificial intelligence model to train on this data, and provides accurate diagnostic results for each medical condition. By further incorporating an emotion engine that recognizes the user's emotions into this system, the method of providing diagnostic results can be adjusted based on the user's emotional state.

[1616] Server program processing

[1617] 1. Data collection and anonymization:

[1618] The server establishes secure communication protocols with each medical institution and regularly collects treatment and test data. During this process, patient personal information is removed and anonymized.

[1619] Specific example: Retrieve patient test results and treatment records from a hospital server, and anonymize them by removing personal information.

[1620] 2. Storing in the database:

[1621] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[1622] Specific example: Classify inspection data into appropriate tables and store them in a database.

[1623] 3. Training of generative artificial intelligence models:

[1624] The server uses data extracted from the database to train a generative artificial intelligence model. This improves the accuracy of diagnosing medical conditions.

[1625] Specific example: Retrain a generative artificial intelligence model using new treatment data to improve diagnostic accuracy.

[1626] 4. Integration with research paper databases:

[1627] The server periodically retrieves the latest research papers from an external database, translates them, and analyzes them. The retrieved data is then input into a generative artificial intelligence model for retraining.

[1628] Specific example: Obtain new papers on heart disease from PubMed, translate and analyze them using natural language processing techniques, and add them to the model.

[1629] Terminal program processing

[1630] 1. Providing a user interface:

[1631] The terminal provides a user interface that is easy for medical staff to use. This interface is designed to be intuitive to operate.

[1632] Specific example: Using the React framework, build an input form for entering patient medical data and a dashboard to display diagnostic results.

[1633] 2. How the emotion engine works:

[1634] While the user is reviewing the diagnostic results, the device activates an emotion engine to analyze the user's emotional state based on facial expression data, voice tone, and other factors.

[1635] Specific example: Using the device's camera and microphone, facial expression analysis software is run to determine in real time whether the user is surprised, relieved, or otherwise experiencing other emotions.

[1636] 3. Display of emotionally-based diagnostic results:

[1637] The device adjusts how the diagnostic results are displayed based on emotional data analyzed by the emotion engine. For example, if the user is feeling anxious, it will provide additional explanations or step-by-step information.

[1638] Specific example: If the system determines that the user is surprised when the diagnostic results are displayed, a detailed explanation of the diagnostic results will be displayed in a pop-up window.

[1639] User program processing

[1640] 1. Entering medical data:

[1641] The medical staff, acting as users, enter patient medical data into the input fields on the terminal and send it to the server.

[1642] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form.

[1643] 2. Confirmation of the diagnosis results:

[1644] The medical staff, as users, review the diagnostic results displayed on the terminal and consider the emotional analysis results from the emotion engine when deciding how to explain things to the patient.

[1645] Specific example: A doctor reviews a diagnosis result displayed by a generative artificial intelligence model, which indicates "possible pneumonia," and carefully explains it to a patient who appears anxious.

[1646] 3. Entering feedback:

[1647] Medical staff, acting as users, input feedback on the treatments performed after the diagnosis and their effectiveness into a terminal and send it to the server. The transmitted feedback data is then used again as training data for a generative artificial intelligence model.

[1648] Specific example: The effectiveness of the medication prescribed by the doctor and the patient's response are input as feedback and sent to the server.

[1649] This series of processing steps not only improves diagnostic accuracy in medical settings but also enables flexible responses based on user emotions, providing a better patient experience.

[1650] The following describes the processing flow.

[1651] Modes for carrying out the invention

[1652] Server program processing

[1653] Step 1: Data Collection and Anonymization

[1654] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient identification information is removed and the data is anonymized.

[1655] Example: Retrieve patient test results and treatment records from the hospital server, and anonymize them by removing personal information.

[1656] Step 2: Data reception and verification

[1657] The server analyzes the received data and verifies whether the data format is correct and consistent.

[1658] Example: Check if JSON data has the correct structure and log any problems in the error log.

[1659] Step 3: Data anonymization

[1660] The server removes personal information from the received data and performs anonymization.

[1661] Specific example: Remove patient names and addresses from data and generate a unique anonymous identifier.

[1662] Step 4: Data classification and storage

[1663] The server classifies the anonymized data into categories such as "diagnostic data," "test data," and "treatment data," and stores it in a database.

[1664] Specific example: Classify inspection data into appropriate tables and store them in a database.

[1665] Step 5: Training a generative artificial intelligence model

[1666] The server periodically uses data extracted from the database to train a generative artificial intelligence model.

[1667] Specific example: Export data in CSV format, input it into a machine learning algorithm, and train the model.

[1668] Step 6: Integration with the research paper database

[1669] The server periodically retrieves the latest papers from external research paper databases and performs translation and analysis.

[1670] Specific example: Obtain new articles on heart disease from PubMed, translate them into Japanese using machine translation, and then analyze them.

[1671] Step 7: Retrain the model

[1672] The server retrains its generative artificial intelligence model based on newly acquired research paper data.

[1673] Specific example: Retrain the model with a dataset that includes the latest research paper data to improve diagnostic accuracy.

[1674] Terminal program processing

[1675] Step 1: Provide the user interface

[1676] The terminal provides input forms and dashboards that medical staff can easily operate.

[1677] Specific example: Using the React framework, build a medical data input form and a diagnostic results display dashboard.

[1678] Step 2: Operating the Emotion Engine

[1679] While the user is reviewing the diagnostic results, the device operates an emotion engine, analyzing facial expression data and voice tone.

[1680] Specific example: Using the device's camera and microphone, facial expression analysis software is used to determine the user's emotional state in real time.

[1681] Step 3: Entering medical data

[1682] Medical staff, who are the users, input patient medical data into their terminals and send it to the server.

[1683] Specific example: A doctor enters the patient's symptoms, such as fever and cough, into an input form and sends a diagnostic request to the server.

[1684] Step 4: Processing diagnostic requests

[1685] The server uses a generative artificial intelligence model based on the input medical data to calculate the diagnosis.

[1686] Specific example: A generative artificial intelligence model outputs a diagnosis result stating "high suspicion of infectious disease," and sends that result to the terminal in JSON format.

[1687] Step 5: Displaying the diagnostic results

[1688] The terminal displays the diagnostic results received from the server. Additional analysis results from the emotion engine are also displayed simultaneously.

[1689] Specific example: Along with the diagnostic results, a pop-up message or additional explanation tailored to the user's emotional state will be displayed.

[1690] User program processing

[1691] Step 1: Confirming the emotionally-based diagnostic results

[1692] The medical staff, who are the users, review the diagnostic results and the analysis results of the emotion engine, and then explain them to the patient based on that information.

[1693] Specific example: The system carefully explains to the user the diagnosis that "there is a high possibility of pneumonia" and the emotional information that the user has identified as "anxiety."

[1694] Step 2: Entering emotionally-based feedback

[1695] After the diagnosis, the medical staff, acting as users, input the patient's response and treatment results as feedback into the terminal and send it to the server.

[1696] Specific example: The user inputs information about the patient's condition and post-treatment follow-up, and sends this information, along with the results of the emotion engine's analysis indicating "satisfied," to the server.

[1697] This series of processing steps not only improves diagnostic accuracy in medical settings, but also enables flexible responses based on user emotions, thereby providing a better patient experience.

[1698] (Example 2)

[1699] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1700] Traditional medical diagnostic systems required manual collection and anonymization of treatment and test data, which was time-consuming and labor-intensive. Furthermore, there was a lack of systems capable of efficiently learning from the collected data and providing rapid and accurate diagnostic results. Additionally, the presentation of diagnostic results was not tailored to the patient's emotional state, resulting in a poor patient experience.

[1701] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, and means for analyzing the user's emotions on the terminal and adjusting the method of presenting the diagnosis result based on the emotion result. This enables efficient collection and anonymization of treatment data and examination data, rapid and accurate diagnosis, and presentation of diagnosis results that respond to the patient's emotions.

[1702] "Treatment data" refers to information about treatment provided by a medical institution, including details such as the patient's treatment plan, treatment progress, and medications used.

[1703] "Test data" refers to information that includes the results of various tests performed on a patient, and includes data such as blood tests, urine tests, and imaging diagnostics.

[1704] "Anonymization" is the process of removing or concealing information that identifies an individual, and is a means of handling data securely while protecting patient privacy.

[1705] A "database" is a system for organizing and storing collected treatment and examination data, and is a collection of information that enables efficient data retrieval and management.

[1706] A "generative artificial intelligence model" is an artificial intelligence system that uses collected and learned data to perform tasks such as prediction and classification, equipped with algorithms and data structures.

[1707] "Diagnosis results" refer to information that shows judgments and conclusions about the patient's condition calculated by a generative artificial intelligence model based on collected treatment and examination data.

[1708] A "terminal" refers to a device such as a computer or tablet operated by medical staff, used for data input and display, sentiment analysis, and other similar tasks.

[1709] "Emotional analysis" is a technology that identifies a user's emotional state from their facial expressions, voice tone, and behavior, and is a means of understanding the user's psychological state.

[1710] A "user interface" refers to the interface, including screens and input forms, that medical staff use to access and operate a system through their terminals.

[1711] This invention relates to a system that provides diagnostic results by training a generative artificial intelligence model based on treatment and examination data collected from medical institutions. This system consists of three main components: a server, a terminal, and a user.

[1712] Server Processing

[1713] The server connects to each healthcare facility using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. This data is first anonymized by removing or concealing personal information. For example, personally identifiable information such as names and addresses in the collected data is replaced with "anonymous IDs."

[1714] Anonymized data is categorized into "diagnostic data," "test data," and "treatment data," and stored in a database. This database enables efficient data management and retrieval.

[1715] Next, a generative artificial intelligence model is trained using data extracted from this database. During this training process, new treatment data is added to the training set, updating the existing model. For example, retraining the model using a backpropagation algorithm can improve the accuracy of disease diagnosis.

[1716] Furthermore, the server periodically retrieves the latest research papers from external academic databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). As a result, the paper data is also incorporated into the generative artificial intelligence model for retraining.

[1717] Terminal processing

[1718] The terminal provides a user interface that is easy for medical staff to use. For example, patient information input forms and diagnostic result dashboards are built using the React framework. Users can easily input data and check diagnostic results by simply operating the system.

[1719] When displaying diagnostic results, the device captures the user's facial expressions and voice tone in real time and analyzes the user's psychological state using emotion analysis algorithms (e.g., OpenFace or IBM Watson Tone Analyzer). Once the user's emotional state, such as surprise or anxiety, is identified, the way the diagnostic results are presented is adjusted accordingly. For example, if the user is feeling anxious, a pop-up window with additional detailed explanations will be displayed.

[1720] User processing

[1721] Medical staff, as users, enter patient symptoms and medical data into input fields on their terminals and send them to the server. For example, they might enter a patient's fever or cough into an input form and submit it.

[1722] When the diagnostic results are displayed on the terminal, the emotional analysis results are also reviewed. Medical staff explain the results to the patient based on the diagnostic results presented by the generative artificial intelligence model. In doing so, the patient's emotional state is also taken into consideration to provide a more appropriate explanation.

[1723] Feedback regarding treatment results and their effectiveness is also entered into the terminal and sent to the server. The transmitted feedback data is used to retrain generative artificial intelligence models, contributing to improved diagnostic accuracy in the future.

[1724] Example of a prompt

[1725] Use the following prompts to input data into the generative AI model:

[1726] Please retrain the diagnostic accuracy for cardiac disease using recently collected treatment data. Please also incorporate the latest research papers.

[1727] This will not only improve the accuracy of diagnoses in medical settings, but also enable flexible responses based on user emotions, thereby providing a better patient experience.

[1728] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1729] Step 1:

[1730] Data collection and anonymization

[1731] The server connects with each healthcare institution using a secure communication protocol (e.g., TLS) and periodically collects treatment and test data. It receives treatment and test data sent from healthcare institutions as input. Because this data contains patient personal information, it is first anonymized by removing personal information (name, ID, address, etc.). Specifically, it scans each field of the collected data and replaces personal information with meaningless data such as an "anonymous ID." The output is the anonymized treatment and test data.

[1732] Step 2:

[1733] Storage in database

[1734] The server categorizes anonymized data into categories such as "diagnostic data," "test data," and "treatment data." It receives anonymized treatment and test data as input. This data is then organized by category for storage in the appropriate database tables. For example, if anonymized test data is collected, it is organized and stored in the "test data" table. The output is an efficiently organized database.

[1735] Step 3:

[1736] Learning of generative artificial intelligence models

[1737] The server extracts relevant data from the database and trains a generative artificial intelligence model. It extracts "diagnostic data," "test data," and "treatment data" from the database as input. Based on this data, it uses the generative AI model to train and improve the accuracy of disease diagnosis. Specifically, it adds new treatment data to the model's training set and uses a backpropagation algorithm to update the existing model. The output is the trained generative AI model.

[1738] Step 4:

[1739] Integration with research paper databases

[1740] The server periodically retrieves the latest research papers from external research paper databases (e.g., PubMed) and translates and analyzes them using natural language processing techniques (e.g., GPT-4). It receives paper data retrieved from external research paper databases as input. The retrieved paper text is passed through translation and analysis tools to convert it into a format suitable for input into a generative artificial intelligence model. Specifically, it uses the PubMed API to perform keyword searches, downloads new papers, and performs translation and content analysis using natural language processing techniques. The output is the analyzed paper data used for retraining.

[1741] Step 5:

[1742] Providing a user interface

[1743] The terminal provides an interface that is easy for medical staff to use. It receives data for medical staff to input patient information as input. Using the React framework, a patient information input form and a dashboard displaying diagnostic results are built. Specifically, the input form and dashboard are designed and implemented to allow users to operate them intuitively. The output is a user-friendly interface.

[1744] Step 6:

[1745] How the emotion engine works

[1746] The device operates an emotion engine while the user is reviewing the diagnostic results, analyzing the user's facial expressions and voice tone. It receives user facial and voice data acquired using the device's camera and microphone as input. This data is analyzed using an emotion analysis algorithm (e.g., OpenFace or IBM Watson Tone Analyzer) to identify the user's emotional state. Specifically, the camera captures facial expressions, which are then analyzed in real time by the algorithm. The output provides data regarding the user's emotional state.

[1747] Step 7:

[1748] Display of emotion-based diagnostic results

[1749] The device adjusts how the diagnostic results are presented based on data analyzed by the emotion engine. It receives the analysis results from the emotion engine as input. It selects an appropriate method for displaying the diagnostic results based on the analysis results, such as when the user is feeling anxious. Specifically, if the user is determined to be anxious, a pop-up window containing a detailed explanation is displayed. The output is a display of the diagnostic results adapted to the user.

[1750] Step 8:

[1751] Entering medical data

[1752] The medical staff, acting as users, input patient symptoms and medical data into the terminal's input fields. For input, they enter medical data such as the patient's fever or cough into the input form. The terminal then sends this data to the server. Specifically, when the medical staff enters data into the input form and presses the submit button, the input data is sent to the server. The output is the medical data sent to the server.

[1753] Step 9:

[1754] Checking the diagnosis results

[1755] The medical staff, acting as users, review the diagnostic results displayed on the terminal and provide explanations to the patient, taking into account the emotional analysis results from the emotion engine. The input is the diagnostic results and emotional analysis results displayed on the terminal. Based on this data, the medical staff provides the most appropriate explanation to the patient. Specifically, they review the displayed diagnostic results and emotional analysis results, paying particular attention to explanations if the patient appears anxious. The output is the most appropriate explanation for the patient.

[1756] Step 10:

[1757] Feedback Input

[1758] Medical staff, acting as users, input feedback on treatment and its effectiveness after a diagnosis into a terminal and send it to the server. The input consists of data on treatment effectiveness and patient response entered into an input form. The terminal then sends this feedback data to the server. Specifically, when medical staff enter data into the feedback form and press the submit button, the input data is sent to the server. The output is the feedback data sent to the server. This data is then used to retrain generative artificial intelligence models, improving diagnostic accuracy.

[1759] (Application Example 2)

[1760] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1761] In modern factories, accurately diagnosing machine malfunctions and maintenance needs is difficult. Furthermore, when communicating fault diagnosis results to workers, the information provided is often difficult to understand, leading to anxiety and misunderstandings. A system is needed to solve these problems and enable efficient and safe work processes.

[1762] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for anonymizing and collecting treatment data and examination data from each medical institution, means for storing the collected treatment data and examination data in a database, means for training the treatment data and examination data stored in the database using a generative artificial intelligence model, means for calculating a diagnosis result for a medical condition using the generative artificial intelligence model, means for displaying the diagnosis result corresponding to the medical condition on a terminal, means for analyzing the user's emotions using the terminal's camera and microphone, and means for adjusting the method of presenting the diagnosis result based on the analyzed emotion data. As a result, it is possible to calculate a diagnosis result using an AI model based on collected machine operation data and failure data, and to perform emotion analysis of workers using an emotion engine, thereby providing a diagnosis result in an easy-to-understand format.

[1763] "Treatment data" refers to data from a medical institution regarding the process and methods of treatment for a patient's medical condition.

[1764] "Test data" refers to data related to the results of various tests conducted at a medical institution to diagnose a patient's medical condition.

[1765] "Anonymization" is the process of removing information that can identify an individual, so that the individual cannot be identified.

[1766] A "generative artificial intelligence model" is a type of artificial intelligence that learns from data, generates new information, and provides predictions and diagnostic results.

[1767] "Learning" is the process by which a generative artificial intelligence model uses provided data to improve its prediction accuracy and performance.

[1768] A "diagnosis result" is a judgment or conclusion about a medical condition calculated by a generative artificial intelligence model based on the provided data.

[1769] A "terminal" is a device that provides a user interface, allowing medical staff to input medical data and review diagnostic results.

[1770] "Camera and microphone" refers to devices installed in a terminal for capturing video and audio.

[1771] Analyzing "emotions" involves determining the user's emotional state based on their facial expressions and voice, using data acquired from the camera and microphone.

[1772] Adjusting the "presentation method" means changing the way the diagnostic results are displayed and the content of those results to suit the user, based on the analyzed emotional data.

[1773] "Machine operation data" refers to data regarding the operating status of various machines in a factory.

[1774] "Failure data" refers to information such as the state of a machine and error logs when it malfunctions in a factory.

[1775] System Overview

[1776] This invention is a system that collects treatment and examination data anonymized from various medical institutions, trains a generative artificial intelligence model on this data, and provides accurate diagnostic results. Furthermore, it has a function that analyzes the user's emotions using the terminal's camera and microphone, and adjusts the method of presenting the diagnostic results based on the analysis results.

[1777] Hardware and software used

[1778] The following hardware and software are used for the server and terminal:

[1779] 1. Server: Performs data collection, anonymization, storage in a database, training of generative artificial intelligence models, and calculation of diagnostic results.

[1780] 2. Camera and microphone: Capture the user's facial expressions and voice in real time and use them for emotion analysis.

[1781] 3. Database: Stores treatment data and test data.

[1782] 4. Generative artificial intelligence models: These models calculate diagnostic results based on data.

[1783] 5. Emotion Analysis Engine: Analyzes the user's emotions using data acquired from the camera and microphone.

[1784] 6. User Interface (UI): This interface handles the input of medical data, the display of diagnostic results, and the presentation of sentiment analysis results.

[1785] Specific system configuration

[1786] Data collection and anonymization

[1787] The server establishes secure communication protocols with each medical institution and periodically collects treatment and test data. During this process, patient personal information is removed to anonymize the data. For example, patient test results and treatment records are retrieved from the hospital server, personal information is re...

Claims

1. A means of collecting treatment data and test data from each medical institution in an anonymized form, A means for storing collected treatment data and test data in a database, A means for training a generative artificial intelligence model using treatment data and examination data stored in a database, A means for calculating a diagnostic result for a disease condition using a generative artificial intelligence model, A system that includes means for displaying diagnostic results corresponding to the patient's condition on a terminal.

2. The system according to claim 1, comprising means for periodically adding new research paper data to a generative artificial intelligence model and retraining it.

3. The system according to claim 1, which provides a user interface and means for medical staff to input medical data and quickly and accurately confirm diagnostic results.

Images