System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting medical records from a plurality of medical institutions; means for anonymizing personal data of patients from the collected medical records; means for analyzing the anonymized medical records using a generative AI to extract disease features; means for compiling the analyzed results into a report; and means for distributing the report to the medical institutions, research institution, pharmaceutical companies, and medical information providers.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, a vast amount of information is stored in the medical records created at each medical institution during consultations, but this information is not being fully utilized. The main reason for this is that data is managed individually at each medical institution and not consolidated. Furthermore, because medical record data is in text format, it is difficult to create a database, and analyzing and utilizing it requires a great deal of effort. As a result, important information such as disease characteristics and undiscovered symptoms is buried, preventing it from being utilized in medical examinations and research institutions. It is necessary to solve this problem and effectively utilize medical record information to improve the quality of medical care. [Means for solving the problem]

[0005] This invention provides a means for collecting medical record data from multiple medical institutions and storing it in a secure database. Medical record data collected from relevant parties is first anonymized and converted into structured data. The structured data is then analyzed using generative AI to extract disease characteristics and undiscovered symptoms. The analysis results are compiled into a report and distributed to medical institutions, research institutions, pharmaceutical companies, and medical information providers. The system of this invention also includes a means for quality checks and automatic correction of medical record data, as well as a means for automatically detecting outliers and overlooked symptoms using generative AI. This makes it easier to analyze and utilize medical record information, contributing to improving the quality of medical care.

[0006] "Medical record data" refers to a patient's medical records and information on their health status created during consultations at medical institutions.

[0007] "Anonymization" refers to processing personal information in data so that it cannot be identified by third parties.

[0008] "Structured data" refers to data that is organized according to a specific format and is in a form that is easy for computers to process.

[0009] "Generative AI" refers to algorithms that use artificial intelligence technology to generate and extract new information and features from data.

[0010] "Analysis" refers to the use of collected data to analyze patterns and trends using statistical and machine learning techniques.

[0011] "Report" refers to a document or electronic file summarizing the results of an analysis, used to communicate information to relevant parties.

[0012] "Medical institution" refers to a facility that provides medical services to patients, such as a hospital or clinic.

[0013] "Research institution" refers to a facility that conducts scientific research, such as a university or research institute.

[0014] "Pharmaceutical companies" refer to companies engaged in research and development, manufacturing and sales of pharmaceuticals.

[0015] A "medical information provider" refers to a company that collects and analyzes medical-related information and provides it to medical institutions, pharmaceutical companies, etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that collects medical record data from multiple medical institutions, stores it in a secure database, and analyzes it. The following components are required to implement this system:

[0038] System Configuration

[0039] 1. Medical record data collection method

[0040] The server is designed to receive medical record data sent from each medical institution. At the medical institution, the user (IT staff at the medical institution) converts the medical record data into a specific format and sends it to the server via a secure communication path.

[0041] 2. Anonymization measures

[0042] The server has a processing function that automatically removes or anonymizes personal information from the received medical record data, while appropriately protecting personally identifiable information such as the patient's name, address, and telephone number.

[0043] 3. Structured Data Conversion Methods

[0044] The server uses natural language processing (NLP) algorithms to convert textual medical record data into structured data that can be easily searched and analyzed in a database.

[0045] 4. Analysis Methods Using Generative AI

[0046] The server analyzes the structured data using generative AI (e.g., BERT or GPT models). The purpose of the analysis is to extract disease characteristics and hidden symptoms, which will contribute to improving diagnostic accuracy in medical settings and discovering new treatments.

[0047] 5. Report Generation Methods

[0048] The server generates a report based on the analysis results, including statistics and graphs from the analysis, and a list of discovered patterns and symptoms.

[0049] 6. Report Delivery Method

[0050] The server distributes the generated reports in an appropriate format to medical institutions, research institutes, pharmaceutical companies, and medical information providers. Terminals (e.g., doctors' computers or tablets) receive the distributed reports and make them viewable.

[0051] Specific examples

[0052] Examples of data collection

[0053] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, and past prescription drug information.

[0054] Examples of data anonymization

[0055] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects personal information.

[0056] Examples of data structuring

[0057] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[0058] Specific examples of data analysis

[0059] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[0060] Example of report generation and delivery

[0061] The server creates a report summarizing the analysis results in a chart and distributes it to related medical institutions (e.g., medical institutions A and B), research institutions, and pharmaceutical companies.

[0062] The terminal (medical institution B's personal computer) displays the received report to the doctor, who uses it as reference information for diagnosis and treatment planning.

[0063] In this way, the system of the present invention consistently performs the process from collecting medical record data to analyzing it and providing the information, thereby improving the quality of medical care and providing useful information to all parties involved in medical care.

[0064] The processing flow will be explained below.

[0065] Step 1: Data collection

[0066] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0067] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[0068] Step 2: Anonymize the data

[0069] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[0070] Step 3: Preprocessing the data

[0071] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[0072] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[0073] Step 4: Normalize the data

[0074] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[0075] Step 5: Data analysis with generative AI

[0076] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[0077] Step 6: Generate reports

[0078] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[0079] Step 7: Report Distribution

[0080] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0081] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0082] Step 8: Diagnostic support

[0083] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[0084] Through these steps, this system will consistently collect, analyze, and provide information on medical records, contributing to improved diagnostic accuracy and research progress throughout the medical industry.

[0085] Example 1

[0086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0087] Conventional medical data management systems have problems with the efficiency of data management and analysis due to the lack of consistent anonymization, structuring, analysis, and feedback of results for medical record data collected from multiple medical institutions. Furthermore, the lack of a function to instruct the generative AI model on individual analysis content makes advanced medical analysis difficult. To solve these issues, a comprehensive and efficient system is needed.

[0088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0089] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing patient personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI model and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for instructing the generative AI model on the analysis content using prompt statements. This makes it possible to consistently perform data anonymization, structuring, analysis, and feedback, thereby achieving efficient and advanced medical analysis.

[0090] The "collection means" refers to a device or program for receiving medical record data from multiple medical institutions.

[0091] An "anonymization means" is a device or program that removes personal information from collected medical record data or converts it into an unidentifiable form.

[0092] "Means for converting into structured data" refers to a device or program that formats text-based medical record data into a format that is easy to search and analyze in a database.

[0093] "Means for analyzing and extracting disease characteristics" refers to a device or program that uses a generative AI model to analyze structured data and identify disease characteristics and trends.

[0094] A "means for compiling a report" is a device or program that creates a report containing statistical data, graphs, patterns, etc. based on the analysis results.

[0095] The "distribution means" refers to a device or program that electronically transmits the generated report to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0096] "Means for instructing a generative AI model on analysis content using a prompt sentence" refers to a device or program that uses input in the form of a sentence to instruct a generative AI model on specific analysis content.

[0097] MODE FOR CARRYING OUT THE INVENTION

[0098] The present invention provides a consistent system for collecting medical record data from multiple medical institutions, anonymizing the data, structuring it, analyzing it, and providing feedback. Specific embodiments of this system are described below.

[0099] (Collection of medical record data)

[0100] The server receives medical record data sent from each medical institution. To do this, the user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system and sends them to the server using a secure communication channel (e.g., HTTPS). The data sent includes medical history, details of symptoms, and past prescription drug information.

[0101] (Data anonymization)

[0102] The server automatically removes or anonymizes the patient's personal information (e.g., name, address, phone number) from the received medical record data, and in the process, utilizes NLP algorithms to accurately detect personal information within the data.

[0103] (Data structuring)

[0104] The server then uses natural language processing (NLP) to convert the anonymized medical record data into structured data. This structure makes it easier to search and analyze within the database. For example, a statement such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into the format "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[0105] (Data analysis using generative AI)

[0106] The server analyzes the structured data using a generative AI model (e.g., BERT or GPT). During this process, a prompt is used to instruct the generative AI model on specific analysis content. For example, by entering the prompt "Analyze the frequency of heart attacks in patients with high blood pressure," the AI ​​model will perform an analysis on patients with high blood pressure. The resulting statistical data is that "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[0107] (Report generation and distribution)

[0108] The server creates a report based on the analysis results of the generative AI model. This report includes analysis results, statistical data, graphs, medical treatment patterns, etc. The completed report is distributed in an appropriate format (e.g., PDF format) to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0109] (Receiving reports)

[0110] The device (for example, the doctor's computer or tablet) receives the delivered report and makes it viewable, allowing the doctor to use the report as reference information for diagnosis and treatment planning.

[0111] This system will improve the quality of medical care and provide useful information to related parties by consistently collecting, analyzing, and providing feedback on medical record data.

[0112] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0113] Step 1: Collect medical record data

[0114] The user (IT staff at the medical institution) exports patient medical records from the electronic medical record system. Then, this exported data is formatted into a specific format (e.g., CSV format). The formatted data is sent to the server using a security protocol (e.g., HTTPS). The input is raw medical record data, and the output is formatted medical record data. Specifically, the export function of the electronic medical record system is used to execute the data formatting script.

[0115] Step 2: Receiving the data

[0116] The server receives data sent via the HTTPS protocol and temporarily stores it. The input is formatted medical record data, and the output is medical record data stored on the server. Specifically, the receiving server program is executed to process the data.

[0117] Step 3: Anonymize data

[0118] The server automatically identifies personal information (e.g., name, address, phone number) from the received medical record data and deletes or anonymizes this information. The input is the received medical record data, and the output is the anonymized data. Specifically, it runs a personal information identification program using an NLP algorithm and replaces the identified personal information with, for example, "Patient X."

[0119] Step 4: Structuring your data

[0120] The server converts the anonymized medical record data into structured data using a natural language processing (NLP) algorithm. The input is the anonymized medical record data, and the output is structured data. Specifically, an NLP analysis program is run to convert the unstructured data into a format such as "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[0121] Step 5: Analyze the data

[0122] The server analyzes the data using a generative AI model (e.g., BERT or GPT) based on the structured data. The input is the structured data and a prompt, and the output is the analysis result. Specifically, the prompt "Please analyze the frequency of heart attacks in patients with high blood pressure" is entered into the generative AI model, and the analysis begins when the analysis execution button is pressed. As a result, data such as "50% of patients with high blood pressure have experienced a heart attack within the past five years" is obtained.

[0123] Step 6: Generate reports

[0124] The server creates a report containing information such as statistical data, graphs, and medical treatment patterns based on the analysis results of the generative AI model. The input is the analysis results, and the output is the report. Specifically, the analysis results are input into a report generation program and formatted into a format (e.g., PDF).

[0125] Step 7: Report Distribution

[0126] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers via a secure communication protocol. The input is the generated report, and the output is the receiving system of the distribution destination. Specifically, it runs a program that uploads the report to email or a dedicated portal.

[0127] Step 8: Receive and view the report

[0128] The terminal (doctor's PC or tablet) receives the delivered report and makes it available for viewing by the user. The input is the delivered report, and the output is the displayed report. Specifically, a PDF reader or dedicated application is launched to display the received report.

[0129] (Application example 1)

[0130] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0131] Conventional medical data collection and analysis systems do not fully integrate the process of collecting data from multiple medical institutions, anonymizing it, converting it into structured data, and analyzing it using generative AI. This makes it difficult to display information in real time, especially when used on-site at medical facilities. This can lead to delays in timely diagnostic support and treatment planning.

[0132] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0133] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing personal patient information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for collecting patient data and displaying the report in real time using a smartphone application running in medical facilities. This allows for the collection, analysis, and provision of medical record data all in one process, enabling real-time diagnostic support, particularly in medical facilities.

[0134] "Multiple medical institutions" refers to facilities that provide medical services, such as multiple different hospitals, clinics, and other medical facilities.

[0135] "Medical record data" refers to electronic or paper medical data that includes information such as a patient's medical records, medical history, prescription history, and test results.

[0136] "Anonymization" is the process of protecting personal information by removing or transforming information that can identify a specific individual (such as name, address, or phone number).

[0137] "Structured data" is data that is organized according to a specific format and can be easily searched and analyzed within a database.

[0138] "Generative AI" is artificial intelligence that uses natural language processing algorithms and deep learning models (e.g., BERT and GPT) to analyze large datasets and generate new information and patterns.

[0139] A "report" is a document created based on the results of analysis by generative AI, containing information such as statistics, graphs, and a list of discovered patterns and symptoms.

[0140] "Medical facilities" are facilities that provide medical services, such as hospitals, clinics, pharmacies, etc.

[0141] A "smartphone application" is software that runs on a smartphone and is designed to perform a specific task.

[0142] "Real-time" refers to data processing and information delivery occurring almost immediately.

[0143] This system collects medical record data from multiple medical institutions, anonymizes the data, converts it into structured data, analyzes it using generative AI, and generates and distributes reports. It also has the ability to collect patient data in real time and display the analysis results using a smartphone application running in medical facilities.

[0144] System Components

[0145] 1. Methods for collecting medical record data

[0146] The server receives the medical record data sent from each medical institution. This reception uses a secure communication protocol (e.g., HTTPS). At the medical institution, IT personnel export patient data from the electronic medical record system and send it to the server in a specified format (e.g., XML, JSON).

[0147] 2. Anonymization measures

[0148] The server automatically deletes or anonymizes personal information (such as names, addresses, and phone numbers) from the received medical record data. This process can be performed using a Python data processing library (e.g., Pandas).

[0149] 3. Structured Data Conversion Methods

[0150] The server uses natural language processing (NLP) algorithms to convert the text data from the medical records into structured data, using Hugging Face's Transformers library, which can then be used in a Named Entity Recognition (NER) pipeline to extract key medical information.

[0151] 4. Analysis Methods Using Generative AI

[0152] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[0153] 5. Report Generation Methods

[0154] The server generates a report based on the analysis results, which includes statistical data, graphs, and a list of symptoms and patterns discovered. Report generation can use libraries such as Pandas and Matplotlib.

[0155] 6. Report Delivery Method

[0156] The server distributes the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers. For this purpose, an email distribution service (e.g., SMTP server) or cloud storage (e.g., AWS S3) can be used.

[0157] 7. Smartphone Applications

[0158] The device (smartphone) will be used by doctors and staff in healthcare facilities to collect patient data and display reports in real time. This application can be built using mobile development frameworks such as React Native.

[0159] Specific examples

[0160] Examples of data collection

[0161] A user (IT staff at a medical institution) exports a patient's medical records from the electronic medical record system and sends them to the server, including medical history, details of symptoms, and past prescription drug information.

[0162] Examples of data anonymization

[0163] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects privacy.

[0164] Examples of data structuring

[0165] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[0166] Specific examples of data analysis

[0167] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[0168] Example of report generation and delivery

[0169] The server creates a report summarizing the analysis results in charts and graphs and distributes it to relevant medical institutions, research institutes, and pharmaceutical companies. The received report is displayed on the terminal (a PC or tablet at the medical institution) to doctors, who use it as reference information for diagnosis and treatment plans.

[0170] Example prompts for generative AI models

[0171] Please describe the symptoms of high blood pressure in detail.

[0172] Please also include the number of heart attacks you have had in the past year.

[0173] The above is the "Mode for Carrying Out the Invention."

[0174] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0175] Step 1:

[0176] A user exports a patient's medical record from an electronic medical record system and sends it to a server in a specified format (e.g., XML, JSON). The data exported by the user includes the patient's medical history, symptom details, and past prescription drug information. The input is the exported medical record data, and the output is the medical record data sent to the server.

[0177] Step 2:

[0178] The server automatically removes or anonymizes the patient's personal information (such as name, address, and phone number) from the received medical record data. This process uses the Python Pandas library. The input is the received medical record data, and the output is the data from which personal information has been removed or anonymized.

[0179] Step 3:

[0180] The server converts the anonymized medical record data into structured data using natural language processing (NLP) algorithms. To do this, it uses Hugging Face's Transformers library and extracts important medical information using a Named Entity Recognition (NER) pipeline. The input is anonymized medical record data, and the output is structured data.

[0181] Step 4:

[0182] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis can improve diagnostic accuracy and discover new treatments. The input is structured data, and the output is the analysis results.

[0183] Step 5:

[0184] The server generates a report based on the analysis results. The report includes statistical data, graphs, and a list of detected symptoms and patterns. Libraries such as Pandas and Matplotlib are used to generate the report. The input is the analysis results, and the output is the generated report.

[0185] Step 6:

[0186] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers using email distribution services and cloud storage. The input is the generated report, and the output is the distributed report.

[0187] Step 7:

[0188] The device (smartphone) is used by doctors and staff at medical facilities to collect patient data and display reports in real time. It is built using mobile development frameworks such as React Native. This application allows users to input prompts to the generative AI model and quickly obtain analysis results. The input is the prompts to the generative AI model, and the output is the analysis results displayed in real time.

[0189] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0190] The present invention is a system that securely analyzes medical record data collected from medical institutions and provides diagnostic support information, and further improves the accuracy of diagnostic information and advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the present invention are described in detail below.

[0191] System Configuration

[0192] 1. Medical record data collection method

[0193] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0194] The server receives the medical record data sent from the medical institution and checks the consistency and format of the data.

[0195] 2. Anonymization measures

[0196] The server then processes the received medical record data to remove or anonymize the patient's personal information, thereby ensuring patient privacy.

[0197] 3. Data preprocessing methods

[0198] The server checks the quality of the medical record data and automatically or semi-automatically corrects missing information and typos, using natural language processing (NLP) technology to convert text-based data into structured data.

[0199] 4. Data normalization methods

[0200] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[0201] 5. Analysis Methods Using Generative AI

[0202] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[0203] 6. Report Generation Methods

[0204] The server then generates a report based on the analysis results, which includes disease characteristics, newly discovered symptoms, graphs, and charts.

[0205] 7. Report Delivery Method

[0206] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0207] The terminal (a medical institution's computer or tablet) reads the report received from the server and displays it so that the doctor can easily view it during the examination.

[0208] 8. Emotion Engine

[0209] The server incorporates an emotion engine that recognizes the user's emotions and collects emotion data. The emotion engine uses sensors such as cameras and microphones to analyze emotions from the user's facial expressions and voice.

[0210] 9. Utilizing Emotional Data

[0211] The server uses the emotional data collected by the emotion engine to customize diagnostic information and advice, for example providing additional reassuring information if the user is feeling anxious.

[0212] The user (doctor) will use the emotion data to adjust communication with the patient and care plan. This information will be fed back to medical institutions and research institutes to help them further improve their services.

[0213] Specific examples

[0214] Examples of data collection and anonymization

[0215] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, prescription drug information, etc.

[0216] The server analyzes the received medical record data and anonymizes the patient's personal information, for example, removing the patient's name and address.

[0217] Examples of data preprocessing and normalization

[0218] The server corrects grammatical errors and missing values ​​from the medical record data and uses natural language processing technology to structure sentences such as "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[0219] The data is converted into a unified format, and data from different medical institutions is centralized in the same format.

[0220] Specific examples of analysis and report generation using generative AI

[0221] The server uses generative AI to extract analysis results such as "50% of hypertension patients have experienced a heart attack within the past five years."

[0222] The analysis results are generated as a report including graphs and charts.

[0223] Examples of emotion engines

[0224] The terminal (a device equipped with a camera and microphone used in medical institutions) captures the patient's facial expressions and voice during the examination and analyzes them using an emotion engine.

[0225] The server uses the data obtained from the emotion engine to customize diagnostic information, such as adding relaxation information to patients who show anxious expressions.

[0226] Specific examples of report distribution and diagnostic support

[0227] The server sends the generated report to Medical Institution B, a research institute, or a pharmaceutical company.

[0228] The terminal (a personal computer at Medical Institution B) presents the report to the doctor in a viewable format, and is used to assist in diagnosis and treatment planning.

[0229] This system will improve the quality of medical care and provide useful information to those involved by consistently collecting and analyzing medical records and utilizing emotional data.

[0230] The processing flow will be explained below.

[0231] Step 1: Data collection

[0232] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0233] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[0234] Step 2: Anonymize the data

[0235] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[0236] Step 3: Preprocessing the data

[0237] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[0238] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[0239] Step 4: Normalize the data

[0240] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[0241] Step 5: Data analysis with generative AI

[0242] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[0243] Step 6: Generate reports

[0244] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[0245] Step 7: Report Distribution

[0246] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0247] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0248] Step 8: Diagnostic support

[0249] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[0250] Step 9: Collect emotion data

[0251] The terminal (a device equipped with a camera or microphone at the medical institution) captures the patient's facial expressions and voice during the examination and sends them to the server.

[0252] Step 10: Analyze the sentiment data

[0253] The server analyzes the received emotional data using an emotion engine. For example, it uses facial expression analysis and voice analysis technology to determine the patient's emotional state (e.g., anxiety, relief, anger, etc.).

[0254] Step 11: Customize diagnostic information

[0255] The server then customizes diagnostic information and advice based on the emotional data obtained from the emotion engine. For example, if the patient is anxious, it may provide additional relaxation information or reference materials.

[0256] Step 12: Feedback of Emotional Data

[0257] The server will then provide feedback to medical and research institutions on emotion data and customized diagnostic information based on it, which will be used for case studies and to improve treatment.

[0258] In this way, the system of the present invention performs consistent processing from collecting medical record data to analyzing it and utilizing emotional data, thereby improving the quality of medical care and providing useful information to those involved.

[0259] Example 2

[0260] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0261] In modern medical settings, a large amount of data is managed as electronic medical records, and there is a need to utilize these records to improve diagnostic accuracy and develop treatment plans tailored to individual patients. It is also becoming increasingly important to provide medical care that takes into account the patient's emotions and psychological state. However, the format and quality of data collected from multiple medical institutions is not uniform, and unifying and analyzing this data requires a great deal of effort. Furthermore, there is a lack of technology to incorporate patient emotional data into diagnoses. Therefore, there is a need for a system that can efficiently collect, anonymize, preprocess, normalize, and analyze medical data, as well as utilize emotional data in a consistent manner.

[0262] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0263] In this invention, the server includes means for collecting medical data from multiple medical institutions, means for anonymizing personal information from the collected medical data, means for converting the anonymized medical data into structured data, means for analyzing the structured data using a generative AI model and extracting health status characteristics, means for compiling the analysis results into a report, means for distributing the report to various medical institutions, means for collecting emotional data, and means for customizing diagnostic information and advice using the emotional data.This enables efficient organization and analysis of medical data and realizes the provision of diagnostic information that takes patient emotions into consideration.

[0264] "Medical data" refers to information managed by medical institutions, including patient medical records, details of symptoms, prescription drug information, etc.

[0265] "Anonymization" is the process of removing or replacing a patient's personal information from medical data so that individuals cannot be identified.

[0266] "Structured data" is data that is organized and formatted according to certain rules, and includes database formats and JSON formats.

[0267] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques such as BERT and GPT, and is a technology for performing data analysis and pattern extraction.

[0268] "Analysis" is the process of extracting features and patterns from collected and anonymized medical data using a generative AI model.

[0269] A "report" is a document that summarizes the results of an analysis and provides specific data and its interpretation using graphs and charts.

[0270] "Various medical institutions" are organizations that require medical data, including medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0271] "Emotional data" refers to data that indicates the emotional state of a patient, obtained from facial expressions and voice, and is collected using a camera and microphone.

[0272] "Diagnostic Information" means information about a patient's health condition that is based on collected and analyzed medical data.

[0273] "Advice" is treatment or care advice provided based on diagnostic information and emotional data.

[0274] "Preprocessing" is the process of checking the quality of medical data, filling in missing information, and correcting typos.

[0275] "Normalization" is the process of converting and organizing different forms of medical data into a unified format.

[0276] The present invention is a system for analyzing medical data collected from medical institutions and providing diagnostic support information. This system includes a function that improves the accuracy of diagnostic information and advice by taking into account patient emotional data. Specific embodiments of the system are described in detail below.

[0277] System Configuration Overview

[0278] Medical record data collection method

[0279] The user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system to CSV or XML format and sends them to the server using a secure communication protocol (such as HTTPS).

[0280] The server receives this data and performs a data consistency check, for example, to check whether the number of columns and data format of the CSV file match.

[0281] Medical record data anonymization method

[0282] The server anonymizes personal information from the received medical data by replacing the patient's name, address, etc. with a random ID, and also uses a personal information filtering algorithm to prevent personal information from being accidentally left behind.

[0283] Data preprocessing measures

[0284] The server automatically checks the quality of the medical data, for example by completing missing values ​​and correcting typos. It also uses natural language processing (NLP) technology to convert text data into structured data. Specifically, it uses open-source NLP libraries (e.g., spaCy and NLTK).

[0285] Data normalization measures

[0286] The server converts the structured data into a unified format using schema mapping techniques, for example, to convert data from different medical institutions into a common database format (SQL or NoSQL).

[0287] Analysis methods using generative AI

[0288] The server analyzes the data using generative AI (e.g., BERT or GPT models). This AI model extracts health features and hidden patterns from patient data. For example, it finds patterns such as "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[0289] Report Generation Method

[0290] The server generates a report based on the analysis results, including the diagnosis, details of any symptoms detected, and graphs and charts, using data visualization tools (e.g., Matplotlib or Tableau).

[0291] Report delivery method

[0292] The server distributes the generated reports to various medical institutions via secure email or a dedicated data distribution platform.

[0293] The terminal (computer or tablet at the medical institution) receives the report from the server and makes it available for the doctor to view during the consultation. The corresponding application is installed on the device.

[0294] Emotion engine emotion data collection method

[0295] The device (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to a server in real time. This data is then analyzed using emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[0296] How to use emotion data

[0297] The server then customizes diagnostic information and advice based on the emotional data, for example adding information that promotes relaxation if the patient expresses anxiety.

[0298] The user (doctor) uses the emotional data to adjust communication with the patient and provide appropriate treatment.

[0299] Examples and prompts

[0300] Examples of data collection and anonymization

[0301] The user (IT staff at Medical Institution A) exports "Patient A's medical records, symptom details, and prescription drug information" from the electronic medical record system and sends them to the server.

[0302] The server receives "Patient A's medical records" and anonymizes them by replacing personal information (name, address) with a random ID.

[0303] Specific examples of analysis and report generation using generative AI

[0304] The server uses an AI model to analyze a "dataset of hypertension patients" and calculate the "percentage of patients who have experienced a heart attack within the past five years."

[0305] Based on the results of this analysis, a "heart attack risk report for hypertensive patients" is generated and sent to medical institution B in PDF format.

[0306] Specific operation of the emotion engine

[0307] The terminal (a device with a camera) captures the patient's facial expression during the examination and transmits data indicating "anxiety" to the server in real time.

[0308] The server analyzes this emotional data and adds relaxation information (for example, deep breathing techniques) to the diagnostic information for patients who show "anxiety."

[0309] Prompt Sentence Examples

[0310] "Please tell me how to extract patient medical history, symptom details, and prescription drug information from medical institution medical record data, anonymize them, and then analyze them."

[0311] "Please explain the specific steps to extract disease characteristics and hidden patterns from medical record data using generative AI."

[0312] "If a patient is feeling anxious during a consultation, can you tell me specifically how we can use the emotion engine to customize the diagnostic information?"

[0313] This system enables consistent collection, analysis, and utilization of emotional data in medical data, improving the accuracy and efficiency of diagnostic support.

[0314] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0315] Step 1:

[0316] Medical record data collection

[0317] The user (IT staff at a medical institution) exports patient medical records from the electronic medical record system to CSV or XML format and sends them to the server. The input is the patient's medical data, and the output is the raw data received by the server.

[0318] The server receives this data and performs a data integrity check, specifically checking the file format and basic data validation.

[0319] Step 2:

[0320] Medical record data anonymization

[0321] The server processes the received medical data to remove or anonymize personal information. The input is the received medical record data, and the output is the anonymized data.

[0322] The server replaces personally identifiable information, such as the patient's name and address, with a random ID, and then uses an algorithm to remove personal information.

[0323] Step 3:

[0324] Data Preprocessing

[0325] The server checks the quality of the medical record data and automatically corrects missing information and typos. The input is anonymized medical data, and the output is quality-improved data.

[0326] Natural language processing (NLP) techniques are used to convert text into structured data. Specifically, open-source NLP libraries (e.g., spaCy and NLTK) are used to convert sentences such as "The patient has high blood pressure" into "{'Medical history': 'High blood pressure'}".

[0327] Step 4:

[0328] Data normalization

[0329] The server converts the structured data into a unified format. The input is the preprocessed structured data, and the output is the unified format data.

[0330] Perform schema mapping to convert data from different healthcare institutions into a common format (e.g., SQL or NoSQL database format).

[0331] Step 5:

[0332] Data analysis with generative AI

[0333] The server performs data analysis using generative AI (e.g., BERT or GPT models). The input is normalized structured data, and the output is the analysis results.

[0334] AI models extract health features and hidden patterns from input data, yielding insights such as "50% of people with high blood pressure have experienced a heart attack within the past five years."

[0335] Step 6:

[0336] Report Generation

[0337] The server creates a report based on the analysis results. The input is the analysis results from the AI ​​model, and the output is a report (e.g., in PDF format).

[0338] The report includes diagnostic findings, details of any symptoms found, and graphs and charts, and is generated using a data visualization tool (e.g., Matplotlib or Tableau).

[0339] Step 7:

[0340] Report Distribution

[0341] The server distributes the generated reports to various medical institutions. The input is the generated report and the output is the transmitted report.

[0342] Delivery will be via secure email or a dedicated data delivery platform.

[0343] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0344] Step 8:

[0345] Emotion engine emotion data collection

[0346] The terminal (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to the server in real time. The input is the patient's facial expressions and voice, and the output is the captured emotional data.

[0347] The collected emotional data is analyzed by emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[0348] Step 9:

[0349] Utilizing Emotional Data

[0350] The server customizes diagnostic information and advice based on the emotional data. The input is the analyzed emotional data, and the output is customized diagnostic information.

[0351] For example, if the patient is showing signs of anxiety, add information that has a relaxing effect.

[0352] Based on this emotional data, the user (doctor) can adjust communication with the patient and provide appropriate treatment.

[0353] This system enables efficient collection and analysis of medical data and utilization of emotional data, improving the accuracy and efficiency of diagnostic support.

[0354] (Application example 2)

[0355] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] Conventional health management systems are limited to collecting and analyzing medical record data, making it difficult to provide flexible health support based on users' emotions and individual health conditions. Furthermore, healthcare information provided is often standardized, lacking specific advice tailored to each user's condition. This makes it difficult for users to properly manage their health, creating a demand for more personalized health support.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0358] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing the user's personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to related institutions, means for collecting user health condition data and providing health information based on the analysis results, emotion analysis means for recognizing the user's emotions, and means for customizing the health information using the emotion data. This makes it possible to provide health support information that is individually customized based on the user's health condition and emotions.

[0359] "Medical record data" refers to medical information such as medical records and prescription information collected by medical institutions during the course of treating patients.

[0360] "Anonymization" is the process of deleting or processing a patient's personal information from collected medical record data so that a specific individual cannot be identified.

[0361] "Structured data" is data that has been converted from vast amounts of medical information into a unified format or database format, making it easier to analyze.

[0362] "Generative AI" is a machine learning model that learns from large amounts of data and generates and analyzes new information and patterns. It is a technology used in natural language processing and image recognition.

[0363] A "report" is a document or data file that summarizes the analysis results of generative AI and the diagnostic information based on them, and is provided in a format that can be used by users and related institutions.

[0364] "Related institutions" include medical institutions, research institutions, pharmaceutical companies and medical information providers.

[0365] "Health status data" refers to information about a user's physical and mental health status, including vital signs and medical records.

[0366] "Emotion analysis" is a technology that detects and analyzes emotions from a user's facial expressions and voice, and is used to understand the user's psychological state.

[0367] "Customization" refers to individually adjusting and providing information and services that are tailored to the individual circumstances and needs of each user.

[0368] This invention is a system for the healthcare and food delivery fields that analyzes a user's health condition and emotional information and provides individually customized health information and meal plans. This system collects and analyzes medical record data from medical institutions and combines it with emotional analysis of the user to provide more personalized health support.

[0369] System Configuration

[0370] 1. Medical record data collection method

[0371] The user (IT staff at the medical institution) exports medical information from the electronic medical record system and sends it to the server. The medical record data includes medical history, symptoms, and prescription information.

[0372] The server receives the medical record data and checks the data for consistency and format.

[0373] 2. Anonymization measures

[0374] The server deletes or anonymizes personal information from the received medical record data, ensuring privacy and preventing third parties from identifying specific individuals.

[0375] 3. Data preprocessing methods

[0376] The server checks the quality of the medical record data, automatically corrects missing information and typos, and uses natural language processing (NLP) technology to convert text data into structured data.

[0377] 4. Data normalization methods

[0378] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[0379] 5. Analysis Methods Using Generative AI

[0380] The server uses generative AI (e.g., BERT or GPT models) to analyze health status and hidden patterns, and predicts the user's health status and risk.

[0381] 6. Report Generation Methods

[0382] The server compiles the analysis results from the generative AI into a report, which includes the characteristics of the disease and the analysis results along with graphs and charts.

[0383] 7. Report Delivery Method

[0384] The server transmits the generated report to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0385] The terminal (a computer or tablet at the medical institution) reads the report and displays it so that the doctor can view it during the examination.

[0386] 8. Emotion analysis method

[0387] The server incorporates an emotion engine that recognizes the user's emotions, and collects and analyzes emotion data using sensors such as cameras and microphones. The emotion engine determines the user's emotions from their facial expressions and voice.

[0388] 9. Means of customizing health information

[0389] The server then uses the data obtained through emotion analysis to customize health information and advice for the user. For example, if the user is feeling stressed, it will suggest relaxing meals and exercise.

[0390] Specific examples

[0391] Health analysis: Based on the user's health data, a low-salt meal plan is suggested for a user diagnosed with high blood pressure. Generative AI is also used to include analysis results in the report, such as "50% of high blood pressure patients have experienced a heart attack within the past five years."

[0392] Emotion analysis: If the user's facial expressions and voice data are captured during the consultation and they are feeling anxious or stressed, the system will suggest relaxing herbal tea or soothing music.

[0393] Prompt Sentence Examples

[0394] Suggest recipes that are best for users with high blood pressure.

[0395] If the user is feeling anxious, suggest a relaxing meal.

[0396] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0397] Step 1:

[0398] Medical record data collection

[0399] Input: Medical record data provided by medical institutions (CSV or XML format)

[0400] Operation: The user (IT staff at the medical institution) exports the target data from the electronic medical record system and sends it to the server.

[0401] Output: Medical record data saved on the server

[0402] Step 2:

[0403] Data anonymization

[0404] Input: Collected medical record data

[0405] How it works: The server removes or anonymizes personal information such as names and addresses from the patient records by replacing personal information fields in the database with random values.

[0406] Output: De-identified medical record data

[0407] Step 3:

[0408] Data Preprocessing

[0409] Input: De-identified medical record data

[0410] How it works: The server checks the quality of the data, corrects missing information and typos, and uses natural language processing (NLP) to structure an example like "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[0411] Output: Preprocessed structured data

[0412] Step 4:

[0413] Data normalization

[0414] Input: Preprocessed structured data

[0415] How it works: The server converts data provided in different formats into a unified format, making it easier to analyze and integrate diverse data sets.

[0416] Output: Normalized structured data

[0417] Step 5:

[0418] Analysis by generative AI

[0419] Input: Normalized structured data

[0420] How it works: The server analyzes data using a generative AI model (e.g., BERT or GPT) to detect disease signatures, commonly overlooked symptoms, and outliers. The generative AI leverages patterns and knowledge learned from the dataset to form a predictive model.

[0421] Output: Analysis results

[0422] Step 6:

[0423] Report Generation

[0424] Input: Analysis results of the generating AI

[0425] How it works: The server compiles the analysis results into a report, which includes discovered disease characteristics, proposed new treatments, and graphs and charts.

[0426] Output: Generated report

[0427] Step 7:

[0428] Report Distribution

[0429] Input: Generated report

[0430] Operation: The server distributes the generated report to related institutions (medical institutions, research institutions, pharmaceutical companies, medical information providers). The terminal (a PC or tablet at the medical institution) receives the report and displays it in a viewable format.

[0431] Output: Reports delivered to medical institutions and related organizations

[0432] Step 8:

[0433] Emotion analysis

[0434] Input: User's facial expressions and voice data

[0435] How it works: The device (a device with a camera and microphone) captures the user's facial expressions and voice during the consultation, and the server performs emotion analysis. Specifically, it recognizes faces from camera footage and analyzes emotional tones from voice data.

[0436] Output: Emotion analysis results

[0437] Step 9:

[0438] Customized health information

[0439] Input: Analysis results of the generation AI, emotion analysis results

[0440] How it works: The server customizes health information and advice based on your health and emotional state. For example, if a user has high blood pressure and is feeling anxious, it might suggest a relaxing meal plan and specific recipes.

[0441] Output: Customized health information and advice

[0442] Example prompt:

[0443] Suggest recipes that are best for users with high blood pressure.

[0444] If the user is feeling anxious, suggest a relaxing meal.

[0445] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0446] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0447] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0448] [Second embodiment]

[0449] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0450] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0451] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0452] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0453] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0454] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0455] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0456] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0457] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0458] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0459] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0460] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0461] The present invention is a system that collects medical record data from multiple medical institutions, stores it in a secure database, and analyzes it. The following components are required to implement this system:

[0462] System Configuration

[0463] 1. Medical record data collection method

[0464] The server is designed to receive medical record data sent from each medical institution. At the medical institution, the user (IT staff at the medical institution) converts the medical record data into a specific format and sends it to the server via a secure communication path.

[0465] 2. Anonymization measures

[0466] The server has a processing function that automatically removes or anonymizes personal information from the received medical record data, while appropriately protecting personally identifiable information such as the patient's name, address, and telephone number.

[0467] 3. Structured Data Conversion Methods

[0468] The server uses natural language processing (NLP) algorithms to convert textual medical record data into structured data that can be easily searched and analyzed in a database.

[0469] 4. Analysis Methods Using Generative AI

[0470] The server analyzes the structured data using generative AI (e.g., BERT or GPT models). The purpose of the analysis is to extract disease characteristics and hidden symptoms, which will contribute to improving diagnostic accuracy in medical settings and discovering new treatments.

[0471] 5. Report Generation Methods

[0472] The server generates a report based on the analysis results, including statistics and graphs from the analysis, and a list of discovered patterns and symptoms.

[0473] 6. Report Delivery Method

[0474] The server distributes the generated reports in an appropriate format to medical institutions, research institutes, pharmaceutical companies, and medical information providers. Terminals (e.g., doctors' computers or tablets) receive the distributed reports and make them viewable.

[0475] Specific examples

[0476] Examples of data collection

[0477] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, and past prescription drug information.

[0478] Examples of data anonymization

[0479] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects personal information.

[0480] Examples of data structuring

[0481] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[0482] Specific examples of data analysis

[0483] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[0484] Example of report generation and delivery

[0485] The server creates a report summarizing the analysis results in a chart and distributes it to related medical institutions (e.g., medical institutions A and B), research institutions, and pharmaceutical companies.

[0486] The terminal (medical institution B's personal computer) displays the received report to the doctor, who uses it as reference information for diagnosis and treatment planning.

[0487] In this way, the system of the present invention consistently performs the process from collecting medical record data to analyzing it and providing the information, thereby improving the quality of medical care and providing useful information to all parties involved in medical care.

[0488] The processing flow will be explained below.

[0489] Step 1: Data collection

[0490] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0491] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[0492] Step 2: Anonymize the data

[0493] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[0494] Step 3: Preprocessing the data

[0495] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[0496] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[0497] Step 4: Normalize the data

[0498] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[0499] Step 5: Data analysis with generative AI

[0500] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[0501] Step 6: Generate reports

[0502] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[0503] Step 7: Report Distribution

[0504] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0505] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0506] Step 8: Diagnostic support

[0507] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[0508] Through these steps, this system will consistently collect, analyze, and provide information on medical records, contributing to improved diagnostic accuracy and research progress throughout the medical industry.

[0509] Example 1

[0510] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0511] Conventional medical data management systems have problems with the efficiency of data management and analysis due to the lack of consistent anonymization, structuring, analysis, and feedback of results for medical record data collected from multiple medical institutions. Furthermore, the lack of a function to instruct the generative AI model on individual analysis content makes advanced medical analysis difficult. To solve these issues, a comprehensive and efficient system is needed.

[0512] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0513] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing patient personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI model and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for instructing the generative AI model on the analysis content using prompt statements. This makes it possible to consistently perform data anonymization, structuring, analysis, and feedback, thereby achieving efficient and advanced medical analysis.

[0514] The "collection means" refers to a device or program for receiving medical record data from multiple medical institutions.

[0515] An "anonymization means" is a device or program that removes personal information from collected medical record data or converts it into an unidentifiable form.

[0516] "Means for converting into structured data" refers to a device or program that formats text-based medical record data into a format that is easy to search and analyze in a database.

[0517] "Means for analyzing and extracting disease characteristics" refers to a device or program that uses a generative AI model to analyze structured data and identify disease characteristics and trends.

[0518] A "means for compiling a report" is a device or program that creates a report containing statistical data, graphs, patterns, etc. based on the analysis results.

[0519] The "distribution means" refers to a device or program that electronically transmits the generated report to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0520] "Means for instructing a generative AI model on analysis content using a prompt sentence" refers to a device or program that uses input in the form of a sentence to instruct a generative AI model on specific analysis content.

[0521] MODE FOR CARRYING OUT THE INVENTION

[0522] The present invention provides a consistent system for collecting medical record data from multiple medical institutions, anonymizing the data, structuring it, analyzing it, and providing feedback. Specific embodiments of this system are described below.

[0523] (Collection of medical record data)

[0524] The server receives medical record data sent from each medical institution. To do this, the user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system and sends them to the server using a secure communication channel (e.g., HTTPS). The data sent includes medical history, details of symptoms, and past prescription drug information.

[0525] (Data anonymization)

[0526] The server automatically removes or anonymizes the patient's personal information (e.g., name, address, phone number) from the received medical record data, and in the process, utilizes NLP algorithms to accurately detect personal information within the data.

[0527] (Data structuring)

[0528] The server then uses natural language processing (NLP) to convert the anonymized medical record data into structured data. This structure makes it easier to search and analyze within the database. For example, a statement such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into the format "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[0529] (Data analysis using generative AI)

[0530] The server analyzes the structured data using a generative AI model (e.g., BERT or GPT). During this process, a prompt is used to instruct the generative AI model on specific analysis content. For example, by entering the prompt "Analyze the frequency of heart attacks in patients with high blood pressure," the AI ​​model will perform an analysis on patients with high blood pressure. The resulting statistical data is that "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[0531] (Report generation and distribution)

[0532] The server creates a report based on the analysis results of the generative AI model. This report includes analysis results, statistical data, graphs, medical treatment patterns, etc. The completed report is distributed in an appropriate format (e.g., PDF format) to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0533] (Receiving reports)

[0534] The device (for example, the doctor's computer or tablet) receives the delivered report and makes it viewable, allowing the doctor to use the report as reference information for diagnosis and treatment planning.

[0535] This system will improve the quality of medical care and provide useful information to related parties by consistently collecting, analyzing, and providing feedback on medical record data.

[0536] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0537] Step 1: Collect medical record data

[0538] The user (IT staff at the medical institution) exports patient medical records from the electronic medical record system. Then, this exported data is formatted into a specific format (e.g., CSV format). The formatted data is sent to the server using a security protocol (e.g., HTTPS). The input is raw medical record data, and the output is formatted medical record data. Specifically, the export function of the electronic medical record system is used to execute the data formatting script.

[0539] Step 2: Receiving the data

[0540] The server receives data sent via the HTTPS protocol and temporarily stores it. The input is formatted medical record data, and the output is medical record data stored on the server. Specifically, the receiving server program is executed to process the data.

[0541] Step 3: Anonymize data

[0542] The server automatically identifies personal information (e.g., name, address, phone number) from the received medical record data and deletes or anonymizes this information. The input is the received medical record data, and the output is the anonymized data. Specifically, it runs a personal information identification program using an NLP algorithm and replaces the identified personal information with, for example, "Patient X."

[0543] Step 4: Structuring your data

[0544] The server converts the anonymized medical record data into structured data using a natural language processing (NLP) algorithm. The input is the anonymized medical record data, and the output is structured data. Specifically, an NLP analysis program is run to convert the unstructured data into a format such as "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[0545] Step 5: Analyze the data

[0546] The server analyzes the data using a generative AI model (e.g., BERT or GPT) based on the structured data. The input is the structured data and a prompt, and the output is the analysis result. Specifically, the prompt "Please analyze the frequency of heart attacks in patients with high blood pressure" is entered into the generative AI model, and the analysis begins when the analysis execution button is pressed. As a result, data such as "50% of patients with high blood pressure have experienced a heart attack within the past five years" is obtained.

[0547] Step 6: Generate reports

[0548] The server creates a report containing information such as statistical data, graphs, and medical treatment patterns based on the analysis results of the generative AI model. The input is the analysis results, and the output is the report. Specifically, the analysis results are input into a report generation program and formatted into a format (e.g., PDF).

[0549] Step 7: Report Distribution

[0550] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers via a secure communication protocol. The input is the generated report, and the output is the receiving system of the distribution destination. Specifically, it runs a program that uploads the report to email or a dedicated portal.

[0551] Step 8: Receive and view the report

[0552] The terminal (doctor's PC or tablet) receives the delivered report and makes it available for viewing by the user. The input is the delivered report, and the output is the displayed report. Specifically, a PDF reader or dedicated application is launched to display the received report.

[0553] (Application example 1)

[0554] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0555] Conventional medical data collection and analysis systems do not fully integrate the process of collecting data from multiple medical institutions, anonymizing it, converting it into structured data, and analyzing it using generative AI. This makes it difficult to display information in real time, especially when used on-site at medical facilities. This can lead to delays in timely diagnostic support and treatment planning.

[0556] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0557] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing personal patient information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for collecting patient data and displaying the report in real time using a smartphone application running in medical facilities. This allows for the collection, analysis, and provision of medical record data all in one process, enabling real-time diagnostic support, particularly in medical facilities.

[0558] "Multiple medical institutions" refers to facilities that provide medical services, such as multiple different hospitals, clinics, and other medical facilities.

[0559] "Medical record data" refers to electronic or paper medical data that includes information such as a patient's medical records, medical history, prescription history, and test results.

[0560] "Anonymization" is the process of protecting personal information by removing or transforming information that can identify a specific individual (such as name, address, or phone number).

[0561] "Structured data" is data that is organized according to a specific format and can be easily searched and analyzed within a database.

[0562] "Generative AI" is artificial intelligence that uses natural language processing algorithms and deep learning models (e.g., BERT and GPT) to analyze large datasets and generate new information and patterns.

[0563] A "report" is a document created based on the results of analysis by generative AI, containing information such as statistics, graphs, and a list of discovered patterns and symptoms.

[0564] "Medical facilities" are facilities that provide medical services, such as hospitals, clinics, pharmacies, etc.

[0565] A "smartphone application" is software that runs on a smartphone and is designed to perform a specific task.

[0566] "Real-time" refers to data processing and information delivery occurring almost immediately.

[0567] This system collects medical record data from multiple medical institutions, anonymizes the data, converts it into structured data, analyzes it using generative AI, and generates and distributes reports. It also has the ability to collect patient data in real time and display the analysis results using a smartphone application running in medical facilities.

[0568] System Components

[0569] 1. Methods for collecting medical record data

[0570] The server receives the medical record data sent from each medical institution. This reception uses a secure communication protocol (e.g., HTTPS). At the medical institution, IT personnel export patient data from the electronic medical record system and send it to the server in a specified format (e.g., XML, JSON).

[0571] 2. Anonymization measures

[0572] The server automatically deletes or anonymizes personal information (such as names, addresses, and phone numbers) from the received medical record data. This process can be performed using a Python data processing library (e.g., Pandas).

[0573] 3. Structured Data Conversion Methods

[0574] The server uses natural language processing (NLP) algorithms to convert the text data from the medical records into structured data, using Hugging Face's Transformers library, which can then be used in a Named Entity Recognition (NER) pipeline to extract key medical information.

[0575] 4. Analysis Methods Using Generative AI

[0576] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[0577] 5. Report Generation Methods

[0578] The server generates a report based on the analysis results, which includes statistical data, graphs, and a list of symptoms and patterns discovered. Report generation can use libraries such as Pandas and Matplotlib.

[0579] 6. Report Delivery Method

[0580] The server distributes the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers. For this purpose, an email distribution service (e.g., SMTP server) or cloud storage (e.g., AWS S3) can be used.

[0581] 7. Smartphone Applications

[0582] The device (smartphone) will be used by doctors and staff in healthcare facilities to collect patient data and display reports in real time. This application can be built using mobile development frameworks such as React Native.

[0583] Specific examples

[0584] Examples of data collection

[0585] A user (IT staff at a medical institution) exports a patient's medical records from the electronic medical record system and sends them to the server, including medical history, details of symptoms, and past prescription drug information.

[0586] Examples of data anonymization

[0587] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects privacy.

[0588] Examples of data structuring

[0589] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[0590] Specific examples of data analysis

[0591] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[0592] Example of report generation and delivery

[0593] The server creates a report summarizing the analysis results in charts and graphs and distributes it to relevant medical institutions, research institutes, and pharmaceutical companies. The received report is displayed on the terminal (a PC or tablet at the medical institution) to doctors, who use it as reference information for diagnosis and treatment plans.

[0594] Example prompts for generative AI models

[0595] Please describe the symptoms of high blood pressure in detail.

[0596] Please also include the number of heart attacks you have had in the past year.

[0597] The above is the "Mode for Carrying Out the Invention."

[0598] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0599] Step 1:

[0600] A user exports a patient's medical record from an electronic medical record system and sends it to a server in a specified format (e.g., XML, JSON). The data exported by the user includes the patient's medical history, symptom details, and past prescription drug information. The input is the exported medical record data, and the output is the medical record data sent to the server.

[0601] Step 2:

[0602] The server automatically removes or anonymizes the patient's personal information (such as name, address, and phone number) from the received medical record data. This process uses the Python Pandas library. The input is the received medical record data, and the output is the data from which personal information has been removed or anonymized.

[0603] Step 3:

[0604] The server converts the anonymized medical record data into structured data using natural language processing (NLP) algorithms. To do this, it uses Hugging Face's Transformers library and extracts important medical information using a Named Entity Recognition (NER) pipeline. The input is anonymized medical record data, and the output is structured data.

[0605] Step 4:

[0606] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis can improve diagnostic accuracy and discover new treatments. The input is structured data, and the output is the analysis results.

[0607] Step 5:

[0608] The server generates a report based on the analysis results. The report includes statistical data, graphs, and a list of detected symptoms and patterns. Libraries such as Pandas and Matplotlib are used to generate the report. The input is the analysis results, and the output is the generated report.

[0609] Step 6:

[0610] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers using email distribution services and cloud storage. The input is the generated report, and the output is the distributed report.

[0611] Step 7:

[0612] The device (smartphone) is used by doctors and staff at medical facilities to collect patient data and display reports in real time. It is built using mobile development frameworks such as React Native. This application allows users to input prompts to the generative AI model and quickly obtain analysis results. The input is the prompts to the generative AI model, and the output is the analysis results displayed in real time.

[0613] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0614] The present invention is a system that securely analyzes medical record data collected from medical institutions and provides diagnostic support information, and further improves the accuracy of diagnostic information and advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the present invention are described in detail below.

[0615] System Configuration

[0616] 1. Medical record data collection method

[0617] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0618] The server receives the medical record data sent from the medical institution and checks the consistency and format of the data.

[0619] 2. Anonymization measures

[0620] The server then processes the received medical record data to remove or anonymize the patient's personal information, thereby ensuring patient privacy.

[0621] 3. Data preprocessing methods

[0622] The server checks the quality of the medical record data and automatically or semi-automatically corrects missing information and typos, using natural language processing (NLP) technology to convert text-based data into structured data.

[0623] 4. Data normalization methods

[0624] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[0625] 5. Analysis Methods Using Generative AI

[0626] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[0627] 6. Report Generation Methods

[0628] The server then generates a report based on the analysis results, which includes disease characteristics, newly discovered symptoms, graphs, and charts.

[0629] 7. Report Delivery Method

[0630] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0631] The terminal (a medical institution's computer or tablet) reads the report received from the server and displays it so that the doctor can easily view it during the examination.

[0632] 8. Emotion Engine

[0633] The server incorporates an emotion engine that recognizes the user's emotions and collects emotion data. The emotion engine uses sensors such as cameras and microphones to analyze emotions from the user's facial expressions and voice.

[0634] 9. Utilizing Emotional Data

[0635] The server uses the emotional data collected by the emotion engine to customize diagnostic information and advice, for example providing additional reassuring information if the user is feeling anxious.

[0636] The user (doctor) will use the emotion data to adjust communication with the patient and care plan. This information will be fed back to medical institutions and research institutes to help them further improve their services.

[0637] Specific examples

[0638] Examples of data collection and anonymization

[0639] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, prescription drug information, etc.

[0640] The server analyzes the received medical record data and anonymizes the patient's personal information, for example, removing the patient's name and address.

[0641] Examples of data preprocessing and normalization

[0642] The server corrects grammatical errors and missing values ​​from the medical record data and uses natural language processing technology to structure sentences such as "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[0643] The data is converted into a unified format, and data from different medical institutions is centralized in the same format.

[0644] Specific examples of analysis and report generation using generative AI

[0645] The server uses generative AI to extract analysis results such as "50% of hypertension patients have experienced a heart attack within the past five years."

[0646] The analysis results are generated as a report including graphs and charts.

[0647] Examples of emotion engines

[0648] The terminal (a device equipped with a camera and microphone used in medical institutions) captures the patient's facial expressions and voice during the examination and analyzes them using an emotion engine.

[0649] The server uses the data obtained from the emotion engine to customize diagnostic information, such as adding relaxation information to patients who show anxious expressions.

[0650] Specific examples of report distribution and diagnostic support

[0651] The server sends the generated report to Medical Institution B, a research institute, or a pharmaceutical company.

[0652] The terminal (a personal computer at Medical Institution B) presents the report to the doctor in a viewable format, and is used to assist in diagnosis and treatment planning.

[0653] This system will improve the quality of medical care and provide useful information to those involved by consistently collecting and analyzing medical records and utilizing emotional data.

[0654] The processing flow will be explained below.

[0655] Step 1: Data collection

[0656] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0657] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[0658] Step 2: Anonymize the data

[0659] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[0660] Step 3: Preprocessing the data

[0661] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[0662] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[0663] Step 4: Normalize the data

[0664] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[0665] Step 5: Data analysis with generative AI

[0666] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[0667] Step 6: Generate reports

[0668] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[0669] Step 7: Report Distribution

[0670] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0671] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0672] Step 8: Diagnostic support

[0673] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[0674] Step 9: Collect emotion data

[0675] The terminal (a device equipped with a camera or microphone at the medical institution) captures the patient's facial expressions and voice during the examination and sends them to the server.

[0676] Step 10: Analyze the sentiment data

[0677] The server analyzes the received emotional data using an emotion engine. For example, it uses facial expression analysis and voice analysis technology to determine the patient's emotional state (e.g., anxiety, relief, anger, etc.).

[0678] Step 11: Customize diagnostic information

[0679] The server then customizes diagnostic information and advice based on the emotional data obtained from the emotion engine. For example, if the patient is anxious, it may provide additional relaxation information or reference materials.

[0680] Step 12: Feedback of Emotional Data

[0681] The server will then provide feedback to medical and research institutions on emotion data and customized diagnostic information based on it, which will be used for case studies and to improve treatment.

[0682] In this way, the system of the present invention performs consistent processing from collecting medical record data to analyzing it and utilizing emotional data, thereby improving the quality of medical care and providing useful information to those involved.

[0683] Example 2

[0684] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] In modern medical settings, a large amount of data is managed as electronic medical records, and there is a need to utilize these records to improve diagnostic accuracy and develop treatment plans tailored to individual patients. It is also becoming increasingly important to provide medical care that takes into account the patient's emotions and psychological state. However, the format and quality of data collected from multiple medical institutions is not uniform, and unifying and analyzing this data requires a great deal of effort. Furthermore, there is a lack of technology to incorporate patient emotional data into diagnoses. Therefore, there is a need for a system that can efficiently collect, anonymize, preprocess, normalize, and analyze medical data, as well as utilize emotional data in a consistent manner.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0687] In this invention, the server includes means for collecting medical data from multiple medical institutions, means for anonymizing personal information from the collected medical data, means for converting the anonymized medical data into structured data, means for analyzing the structured data using a generative AI model and extracting health status characteristics, means for compiling the analysis results into a report, means for distributing the report to various medical institutions, means for collecting emotional data, and means for customizing diagnostic information and advice using the emotional data.This enables efficient organization and analysis of medical data and realizes the provision of diagnostic information that takes patient emotions into consideration.

[0688] "Medical data" refers to information managed by medical institutions, including patient medical records, details of symptoms, prescription drug information, etc.

[0689] "Anonymization" is the process of removing or replacing a patient's personal information from medical data so that individuals cannot be identified.

[0690] "Structured data" is data that is organized and formatted according to certain rules, and includes database formats and JSON formats.

[0691] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques such as BERT and GPT, and is a technology for performing data analysis and pattern extraction.

[0692] "Analysis" is the process of extracting features and patterns from collected and anonymized medical data using a generative AI model.

[0693] A "report" is a document that summarizes the results of an analysis and provides specific data and its interpretation using graphs and charts.

[0694] "Various medical institutions" are organizations that require medical data, including medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0695] "Emotional data" refers to data that indicates the emotional state of a patient, obtained from facial expressions and voice, and is collected using a camera and microphone.

[0696] "Diagnostic Information" means information about a patient's health condition that is based on collected and analyzed medical data.

[0697] "Advice" is treatment or care advice provided based on diagnostic information and emotional data.

[0698] "Preprocessing" is the process of checking the quality of medical data, filling in missing information, and correcting typos.

[0699] "Normalization" is the process of converting and organizing different forms of medical data into a unified format.

[0700] The present invention is a system for analyzing medical data collected from medical institutions and providing diagnostic support information. This system includes a function that improves the accuracy of diagnostic information and advice by taking into account patient emotional data. Specific embodiments of the system are described in detail below.

[0701] System Configuration Overview

[0702] Medical record data collection method

[0703] The user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system to CSV or XML format and sends them to the server using a secure communication protocol (such as HTTPS).

[0704] The server receives this data and performs a data consistency check, for example, to check whether the number of columns and data format of the CSV file match.

[0705] Medical record data anonymization method

[0706] The server anonymizes personal information from the received medical data by replacing the patient's name, address, etc. with a random ID, and also uses a personal information filtering algorithm to prevent personal information from being accidentally left behind.

[0707] Data preprocessing measures

[0708] The server automatically checks the quality of the medical data, for example by completing missing values ​​and correcting typos. It also uses natural language processing (NLP) technology to convert text data into structured data. Specifically, it uses open-source NLP libraries (e.g., spaCy and NLTK).

[0709] Data normalization measures

[0710] The server converts the structured data into a unified format using schema mapping techniques, for example, to convert data from different medical institutions into a common database format (SQL or NoSQL).

[0711] Analysis methods using generative AI

[0712] The server analyzes the data using generative AI (e.g., BERT or GPT models). This AI model extracts health features and hidden patterns from patient data. For example, it finds patterns such as "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[0713] Report Generation Method

[0714] The server generates a report based on the analysis results, including the diagnosis, details of any symptoms detected, and graphs and charts, using data visualization tools (e.g., Matplotlib or Tableau).

[0715] Report delivery method

[0716] The server distributes the generated reports to various medical institutions via secure email or a dedicated data distribution platform.

[0717] The terminal (computer or tablet at the medical institution) receives the report from the server and makes it available for the doctor to view during the consultation. The corresponding application is installed on the device.

[0718] Emotion engine emotion data collection method

[0719] The device (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to a server in real time. This data is then analyzed using emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[0720] How to use emotion data

[0721] The server then customizes diagnostic information and advice based on the emotional data, for example adding information that promotes relaxation if the patient expresses anxiety.

[0722] The user (doctor) uses the emotional data to adjust communication with the patient and provide appropriate treatment.

[0723] Examples and prompts

[0724] Examples of data collection and anonymization

[0725] The user (IT staff at Medical Institution A) exports "Patient A's medical records, symptom details, and prescription drug information" from the electronic medical record system and sends them to the server.

[0726] The server receives "Patient A's medical records" and anonymizes them by replacing personal information (name, address) with a random ID.

[0727] Specific examples of analysis and report generation using generative AI

[0728] The server uses an AI model to analyze a "dataset of hypertension patients" and calculate the "percentage of patients who have experienced a heart attack within the past five years."

[0729] Based on the results of this analysis, a "heart attack risk report for hypertensive patients" is generated and sent to medical institution B in PDF format.

[0730] Specific operation of the emotion engine

[0731] The terminal (a device with a camera) captures the patient's facial expression during the examination and transmits data indicating "anxiety" to the server in real time.

[0732] The server analyzes this emotional data and adds relaxation information (for example, deep breathing techniques) to the diagnostic information for patients who show "anxiety."

[0733] Prompt Sentence Examples

[0734] "Please tell me how to extract patient medical history, symptom details, and prescription drug information from medical institution medical record data, anonymize them, and then analyze them."

[0735] "Please explain the specific steps to extract disease characteristics and hidden patterns from medical record data using generative AI."

[0736] "If a patient is feeling anxious during a consultation, can you tell me specifically how we can use the emotion engine to customize the diagnostic information?"

[0737] This system enables consistent collection, analysis, and utilization of emotional data in medical data, improving the accuracy and efficiency of diagnostic support.

[0738] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0739] Step 1:

[0740] Medical record data collection

[0741] The user (IT staff at a medical institution) exports patient medical records from the electronic medical record system to CSV or XML format and sends them to the server. The input is the patient's medical data, and the output is the raw data received by the server.

[0742] The server receives this data and performs a data integrity check, specifically checking the file format and basic data validation.

[0743] Step 2:

[0744] Medical record data anonymization

[0745] The server processes the received medical data to remove or anonymize personal information. The input is the received medical record data, and the output is the anonymized data.

[0746] The server replaces personally identifiable information, such as the patient's name and address, with a random ID, and then uses an algorithm to remove personal information.

[0747] Step 3:

[0748] Data Preprocessing

[0749] The server checks the quality of the medical record data and automatically corrects missing information and typos. The input is anonymized medical data, and the output is quality-improved data.

[0750] Natural language processing (NLP) techniques are used to convert text into structured data. Specifically, open-source NLP libraries (e.g., spaCy and NLTK) are used to convert sentences such as "The patient has high blood pressure" into "{'Medical history': 'High blood pressure'}".

[0751] Step 4:

[0752] Data normalization

[0753] The server converts the structured data into a unified format. The input is the preprocessed structured data, and the output is the unified format data.

[0754] Perform schema mapping to convert data from different healthcare institutions into a common format (e.g., SQL or NoSQL database format).

[0755] Step 5:

[0756] Data analysis with generative AI

[0757] The server performs data analysis using generative AI (e.g., BERT or GPT models). The input is normalized structured data, and the output is the analysis results.

[0758] AI models extract health features and hidden patterns from input data, yielding insights such as "50% of people with high blood pressure have experienced a heart attack within the past five years."

[0759] Step 6:

[0760] Report Generation

[0761] The server creates a report based on the analysis results. The input is the analysis results from the AI ​​model, and the output is a report (e.g., in PDF format).

[0762] The report includes diagnostic findings, details of any symptoms found, and graphs and charts, and is generated using a data visualization tool (e.g., Matplotlib or Tableau).

[0763] Step 7:

[0764] Report Distribution

[0765] The server distributes the generated reports to various medical institutions. The input is the generated report and the output is the transmitted report.

[0766] Delivery will be via secure email or a dedicated data delivery platform.

[0767] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0768] Step 8:

[0769] Emotion engine emotion data collection

[0770] The terminal (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to the server in real time. The input is the patient's facial expressions and voice, and the output is the captured emotional data.

[0771] The collected emotional data is analyzed by emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[0772] Step 9:

[0773] Utilizing Emotional Data

[0774] The server customizes diagnostic information and advice based on the emotional data. The input is the analyzed emotional data, and the output is customized diagnostic information.

[0775] For example, if the patient is showing signs of anxiety, add information that has a relaxing effect.

[0776] Based on this emotional data, the user (doctor) can adjust communication with the patient and provide appropriate treatment.

[0777] This system enables efficient collection and analysis of medical data and utilization of emotional data, improving the accuracy and efficiency of diagnostic support.

[0778] (Application example 2)

[0779] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0780] Conventional health management systems are limited to collecting and analyzing medical record data, making it difficult to provide flexible health support based on users' emotions and individual health conditions. Furthermore, healthcare information provided is often standardized, lacking specific advice tailored to each user's condition. This makes it difficult for users to properly manage their health, creating a demand for more personalized health support.

[0781] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0782] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing the user's personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to related institutions, means for collecting user health condition data and providing health information based on the analysis results, emotion analysis means for recognizing the user's emotions, and means for customizing the health information using the emotion data. This makes it possible to provide health support information that is individually customized based on the user's health condition and emotions.

[0783] "Medical record data" refers to medical information such as medical records and prescription information collected by medical institutions during the course of treating patients.

[0784] "Anonymization" is the process of deleting or processing a patient's personal information from collected medical record data so that a specific individual cannot be identified.

[0785] "Structured data" is data that has been converted from vast amounts of medical information into a unified format or database format, making it easier to analyze.

[0786] "Generative AI" is a machine learning model that learns from large amounts of data and generates and analyzes new information and patterns. It is a technology used in natural language processing and image recognition.

[0787] A "report" is a document or data file that summarizes the analysis results of generative AI and the diagnostic information based on them, and is provided in a format that can be used by users and related institutions.

[0788] "Related institutions" include medical institutions, research institutions, pharmaceutical companies and medical information providers.

[0789] "Health status data" refers to information about a user's physical and mental health status, including vital signs and medical records.

[0790] "Emotion analysis" is a technology that detects and analyzes emotions from a user's facial expressions and voice, and is used to understand the user's psychological state.

[0791] "Customization" refers to individually adjusting and providing information and services that are tailored to the individual circumstances and needs of each user.

[0792] This invention is a system for the healthcare and food delivery fields that analyzes a user's health condition and emotional information and provides individually customized health information and meal plans. This system collects and analyzes medical record data from medical institutions and combines it with emotional analysis of the user to provide more personalized health support.

[0793] System Configuration

[0794] 1. Medical record data collection method

[0795] The user (IT staff at the medical institution) exports medical information from the electronic medical record system and sends it to the server. The medical record data includes medical history, symptoms, and prescription information.

[0796] The server receives the medical record data and checks the data for consistency and format.

[0797] 2. Anonymization measures

[0798] The server deletes or anonymizes personal information from the received medical record data, ensuring privacy and preventing third parties from identifying specific individuals.

[0799] 3. Data preprocessing methods

[0800] The server checks the quality of the medical record data, automatically corrects missing information and typos, and uses natural language processing (NLP) technology to convert text data into structured data.

[0801] 4. Data normalization methods

[0802] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[0803] 5. Analysis Methods Using Generative AI

[0804] The server uses generative AI (e.g., BERT or GPT models) to analyze health status and hidden patterns, and predicts the user's health status and risk.

[0805] 6. Report Generation Methods

[0806] The server compiles the analysis results from the generative AI into a report, which includes the characteristics of the disease and the analysis results along with graphs and charts.

[0807] 7. Report Delivery Method

[0808] The server transmits the generated report to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0809] The terminal (a computer or tablet at the medical institution) reads the report and displays it so that the doctor can view it during the examination.

[0810] 8. Emotion analysis method

[0811] The server incorporates an emotion engine that recognizes the user's emotions, and collects and analyzes emotion data using sensors such as cameras and microphones. The emotion engine determines the user's emotions from their facial expressions and voice.

[0812] 9. Means of customizing health information

[0813] The server then uses the data obtained through emotion analysis to customize health information and advice for the user. For example, if the user is feeling stressed, it will suggest relaxing meals and exercise.

[0814] Specific examples

[0815] Health analysis: Based on the user's health data, a low-salt meal plan is suggested for a user diagnosed with high blood pressure. Generative AI is also used to include analysis results in the report, such as "50% of high blood pressure patients have experienced a heart attack within the past five years."

[0816] Emotion analysis: If the user's facial expressions and voice data are captured during the consultation and they are feeling anxious or stressed, the system will suggest relaxing herbal tea or soothing music.

[0817] Prompt Sentence Examples

[0818] Suggest recipes that are best for users with high blood pressure.

[0819] If the user is feeling anxious, suggest a relaxing meal.

[0820] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0821] Step 1:

[0822] Medical record data collection

[0823] Input: Medical record data provided by medical institutions (CSV or XML format)

[0824] Operation: The user (IT staff at the medical institution) exports the target data from the electronic medical record system and sends it to the server.

[0825] Output: Medical record data saved on the server

[0826] Step 2:

[0827] Data anonymization

[0828] Input: Collected medical record data

[0829] How it works: The server removes or anonymizes personal information such as names and addresses from the patient records by replacing personal information fields in the database with random values.

[0830] Output: De-identified medical record data

[0831] Step 3:

[0832] Data Preprocessing

[0833] Input: De-identified medical record data

[0834] How it works: The server checks the quality of the data, corrects missing information and typos, and uses natural language processing (NLP) to structure an example like "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[0835] Output: Preprocessed structured data

[0836] Step 4:

[0837] Data normalization

[0838] Input: Preprocessed structured data

[0839] How it works: The server converts data provided in different formats into a unified format, making it easier to analyze and integrate diverse data sets.

[0840] Output: Normalized structured data

[0841] Step 5:

[0842] Analysis by generative AI

[0843] Input: Normalized structured data

[0844] How it works: The server analyzes data using a generative AI model (e.g., BERT or GPT) to detect disease signatures, commonly overlooked symptoms, and outliers. The generative AI leverages patterns and knowledge learned from the dataset to form a predictive model.

[0845] Output: Analysis results

[0846] Step 6:

[0847] Report Generation

[0848] Input: Analysis results of the generating AI

[0849] How it works: The server compiles the analysis results into a report, which includes discovered disease characteristics, proposed new treatments, and graphs and charts.

[0850] Output: Generated report

[0851] Step 7:

[0852] Report Distribution

[0853] Input: Generated report

[0854] Operation: The server distributes the generated report to related institutions (medical institutions, research institutions, pharmaceutical companies, medical information providers). The terminal (a PC or tablet at the medical institution) receives the report and displays it in a viewable format.

[0855] Output: Reports delivered to medical institutions and related organizations

[0856] Step 8:

[0857] Emotion analysis

[0858] Input: User's facial expressions and voice data

[0859] How it works: The device (a device with a camera and microphone) captures the user's facial expressions and voice during the consultation, and the server performs emotion analysis. Specifically, it recognizes faces from camera footage and analyzes emotional tones from voice data.

[0860] Output: Emotion analysis results

[0861] Step 9:

[0862] Customized health information

[0863] Input: Analysis results of the generation AI, emotion analysis results

[0864] How it works: The server customizes health information and advice based on your health and emotional state. For example, if a user has high blood pressure and is feeling anxious, it might suggest a relaxing meal plan and specific recipes.

[0865] Output: Customized health information and advice

[0866] Example prompt:

[0867] Suggest recipes that are best for users with high blood pressure.

[0868] If the user is feeling anxious, suggest a relaxing meal.

[0869] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0870] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0871] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0872] [Third embodiment]

[0873] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0874] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0875] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0876] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0877] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0878] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0879] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0880] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0881] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0882] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0883] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0884] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0885] The present invention is a system that collects medical record data from multiple medical institutions, stores it in a secure database, and analyzes it. The following components are required to implement this system:

[0886] System Configuration

[0887] 1. Medical record data collection method

[0888] The server is designed to receive medical record data sent from each medical institution. At the medical institution, the user (IT staff at the medical institution) converts the medical record data into a specific format and sends it to the server via a secure communication path.

[0889] 2. Anonymization measures

[0890] The server has a processing function that automatically removes or anonymizes personal information from the received medical record data, while appropriately protecting personally identifiable information such as the patient's name, address, and telephone number.

[0891] 3. Structured Data Conversion Methods

[0892] The server uses natural language processing (NLP) algorithms to convert textual medical record data into structured data that can be easily searched and analyzed in a database.

[0893] 4. Analysis Methods Using Generative AI

[0894] The server analyzes the structured data using generative AI (e.g., BERT or GPT models). The purpose of the analysis is to extract disease characteristics and hidden symptoms, which will contribute to improving diagnostic accuracy in medical settings and discovering new treatments.

[0895] 5. Report Generation Methods

[0896] The server generates a report based on the analysis results, including statistics and graphs from the analysis, and a list of discovered patterns and symptoms.

[0897] 6. Report Delivery Method

[0898] The server distributes the generated reports in an appropriate format to medical institutions, research institutes, pharmaceutical companies, and medical information providers. Terminals (e.g., doctors' computers or tablets) receive the distributed reports and make them viewable.

[0899] Specific examples

[0900] Examples of data collection

[0901] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, and past prescription drug information.

[0902] Examples of data anonymization

[0903] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects personal information.

[0904] Examples of data structuring

[0905] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[0906] Specific examples of data analysis

[0907] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[0908] Example of report generation and delivery

[0909] The server creates a report summarizing the analysis results in a chart and distributes it to related medical institutions (e.g., medical institutions A and B), research institutions, and pharmaceutical companies.

[0910] The terminal (medical institution B's personal computer) displays the received report to the doctor, who uses it as reference information for diagnosis and treatment planning.

[0911] In this way, the system of the present invention consistently performs the process from collecting medical record data to analyzing it and providing the information, thereby improving the quality of medical care and providing useful information to all parties involved in medical care.

[0912] The processing flow will be explained below.

[0913] Step 1: Data collection

[0914] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[0915] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[0916] Step 2: Anonymize the data

[0917] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[0918] Step 3: Preprocessing the data

[0919] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[0920] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[0921] Step 4: Normalize the data

[0922] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[0923] Step 5: Data analysis with generative AI

[0924] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[0925] Step 6: Generate reports

[0926] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[0927] Step 7: Report Distribution

[0928] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[0929] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[0930] Step 8: Diagnostic support

[0931] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[0932] Through these steps, this system will consistently collect, analyze, and provide information on medical records, contributing to improved diagnostic accuracy and research progress throughout the medical industry.

[0933] Example 1

[0934] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0935] Conventional medical data management systems have problems with the efficiency of data management and analysis due to the lack of consistent anonymization, structuring, analysis, and feedback of results for medical record data collected from multiple medical institutions. Furthermore, the lack of a function to instruct the generative AI model on individual analysis content makes advanced medical analysis difficult. To solve these issues, a comprehensive and efficient system is needed.

[0936] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0937] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing patient personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI model and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for instructing the generative AI model on the analysis content using prompt statements. This makes it possible to consistently perform data anonymization, structuring, analysis, and feedback, thereby achieving efficient and advanced medical analysis.

[0938] The "collection means" refers to a device or program for receiving medical record data from multiple medical institutions.

[0939] An "anonymization means" is a device or program that removes personal information from collected medical record data or converts it into an unidentifiable form.

[0940] "Means for converting into structured data" refers to a device or program that formats text-based medical record data into a format that is easy to search and analyze in a database.

[0941] "Means for analyzing and extracting disease characteristics" refers to a device or program that uses a generative AI model to analyze structured data and identify disease characteristics and trends.

[0942] A "means for compiling a report" is a device or program that creates a report containing statistical data, graphs, patterns, etc. based on the analysis results.

[0943] The "distribution means" refers to a device or program that electronically transmits the generated report to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0944] "Means for instructing a generative AI model on analysis content using a prompt sentence" refers to a device or program that uses input in the form of a sentence to instruct a generative AI model on specific analysis content.

[0945] MODE FOR CARRYING OUT THE INVENTION

[0946] The present invention provides a consistent system for collecting medical record data from multiple medical institutions, anonymizing the data, structuring it, analyzing it, and providing feedback. Specific embodiments of this system are described below.

[0947] (Collection of medical record data)

[0948] The server receives medical record data sent from each medical institution. To do this, the user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system and sends them to the server using a secure communication channel (e.g., HTTPS). The data sent includes medical history, details of symptoms, and past prescription drug information.

[0949] (Data anonymization)

[0950] The server automatically removes or anonymizes the patient's personal information (e.g., name, address, phone number) from the received medical record data, and in the process, utilizes NLP algorithms to accurately detect personal information within the data.

[0951] (Data structuring)

[0952] The server then uses natural language processing (NLP) to convert the anonymized medical record data into structured data. This structure makes it easier to search and analyze within the database. For example, a statement such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into the format "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[0953] (Data analysis using generative AI)

[0954] The server analyzes the structured data using a generative AI model (e.g., BERT or GPT). During this process, a prompt is used to instruct the generative AI model on specific analysis content. For example, by entering the prompt "Analyze the frequency of heart attacks in patients with high blood pressure," the AI ​​model will perform an analysis on patients with high blood pressure. The resulting statistical data is that "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[0955] (Report generation and distribution)

[0956] The server creates a report based on the analysis results of the generative AI model. This report includes analysis results, statistical data, graphs, medical treatment patterns, etc. The completed report is distributed in an appropriate format (e.g., PDF format) to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[0957] (Receiving reports)

[0958] The device (for example, the doctor's computer or tablet) receives the delivered report and makes it viewable, allowing the doctor to use the report as reference information for diagnosis and treatment planning.

[0959] This system will improve the quality of medical care and provide useful information to related parties by consistently collecting, analyzing, and providing feedback on medical record data.

[0960] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0961] Step 1: Collect medical record data

[0962] The user (IT staff at the medical institution) exports patient medical records from the electronic medical record system. Then, this exported data is formatted into a specific format (e.g., CSV format). The formatted data is sent to the server using a security protocol (e.g., HTTPS). The input is raw medical record data, and the output is formatted medical record data. Specifically, the export function of the electronic medical record system is used to execute the data formatting script.

[0963] Step 2: Receiving the data

[0964] The server receives data sent via the HTTPS protocol and temporarily stores it. The input is formatted medical record data, and the output is medical record data stored on the server. Specifically, the receiving server program is executed to process the data.

[0965] Step 3: Anonymize data

[0966] The server automatically identifies personal information (e.g., name, address, phone number) from the received medical record data and deletes or anonymizes this information. The input is the received medical record data, and the output is the anonymized data. Specifically, it runs a personal information identification program using an NLP algorithm and replaces the identified personal information with, for example, "Patient X."

[0967] Step 4: Structuring your data

[0968] The server converts the anonymized medical record data into structured data using a natural language processing (NLP) algorithm. The input is the anonymized medical record data, and the output is structured data. Specifically, an NLP analysis program is run to convert the unstructured data into a format such as "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[0969] Step 5: Analyze the data

[0970] The server analyzes the data using a generative AI model (e.g., BERT or GPT) based on the structured data. The input is the structured data and a prompt, and the output is the analysis result. Specifically, the prompt "Please analyze the frequency of heart attacks in patients with high blood pressure" is entered into the generative AI model, and the analysis begins when the analysis execution button is pressed. As a result, data such as "50% of patients with high blood pressure have experienced a heart attack within the past five years" is obtained.

[0971] Step 6: Generate reports

[0972] The server creates a report containing information such as statistical data, graphs, and medical treatment patterns based on the analysis results of the generative AI model. The input is the analysis results, and the output is the report. Specifically, the analysis results are input into a report generation program and formatted into a format (e.g., PDF).

[0973] Step 7: Report Distribution

[0974] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers via a secure communication protocol. The input is the generated report, and the output is the receiving system of the distribution destination. Specifically, it runs a program that uploads the report to email or a dedicated portal.

[0975] Step 8: Receive and view the report

[0976] The terminal (doctor's PC or tablet) receives the delivered report and makes it available for viewing by the user. The input is the delivered report, and the output is the displayed report. Specifically, a PDF reader or dedicated application is launched to display the received report.

[0977] (Application example 1)

[0978] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0979] Conventional medical data collection and analysis systems do not fully integrate the process of collecting data from multiple medical institutions, anonymizing it, converting it into structured data, and analyzing it using generative AI. This makes it difficult to display information in real time, especially when used on-site at medical facilities. This can lead to delays in timely diagnostic support and treatment planning.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0981] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing personal patient information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for collecting patient data and displaying the report in real time using a smartphone application running in medical facilities. This allows for the collection, analysis, and provision of medical record data all in one process, enabling real-time diagnostic support, particularly in medical facilities.

[0982] "Multiple medical institutions" refers to facilities that provide medical services, such as multiple different hospitals, clinics, and other medical facilities.

[0983] "Medical record data" refers to electronic or paper medical data that includes information such as a patient's medical records, medical history, prescription history, and test results.

[0984] "Anonymization" is the process of protecting personal information by removing or transforming information that can identify a specific individual (such as name, address, or phone number).

[0985] "Structured data" is data that is organized according to a specific format and can be easily searched and analyzed within a database.

[0986] "Generative AI" is artificial intelligence that uses natural language processing algorithms and deep learning models (e.g., BERT and GPT) to analyze large datasets and generate new information and patterns.

[0987] A "report" is a document created based on the results of analysis by generative AI, containing information such as statistics, graphs, and a list of discovered patterns and symptoms.

[0988] "Medical facilities" are facilities that provide medical services, such as hospitals, clinics, pharmacies, etc.

[0989] A "smartphone application" is software that runs on a smartphone and is designed to perform a specific task.

[0990] "Real-time" refers to data processing and information delivery occurring almost immediately.

[0991] This system collects medical record data from multiple medical institutions, anonymizes the data, converts it into structured data, analyzes it using generative AI, and generates and distributes reports. It also has the ability to collect patient data in real time and display the analysis results using a smartphone application running in medical facilities.

[0992] System Components

[0993] 1. Methods for collecting medical record data

[0994] The server receives the medical record data sent from each medical institution. This reception uses a secure communication protocol (e.g., HTTPS). At the medical institution, IT personnel export patient data from the electronic medical record system and send it to the server in a specified format (e.g., XML, JSON).

[0995] 2. Anonymization measures

[0996] The server automatically deletes or anonymizes personal information (such as names, addresses, and phone numbers) from the received medical record data. This process can be performed using a Python data processing library (e.g., Pandas).

[0997] 3. Structured Data Conversion Methods

[0998] The server uses natural language processing (NLP) algorithms to convert the text data from the medical records into structured data, using Hugging Face's Transformers library, which can then be used in a Named Entity Recognition (NER) pipeline to extract key medical information.

[0999] 4. Analysis Methods Using Generative AI

[1000] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[1001] 5. Report Generation Methods

[1002] The server generates a report based on the analysis results, which includes statistical data, graphs, and a list of symptoms and patterns discovered. Report generation can use libraries such as Pandas and Matplotlib.

[1003] 6. Report Delivery Method

[1004] The server distributes the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers. For this purpose, an email distribution service (e.g., SMTP server) or cloud storage (e.g., AWS S3) can be used.

[1005] 7. Smartphone Applications

[1006] The device (smartphone) will be used by doctors and staff in healthcare facilities to collect patient data and display reports in real time. This application can be built using mobile development frameworks such as React Native.

[1007] Specific examples

[1008] Examples of data collection

[1009] A user (IT staff at a medical institution) exports a patient's medical records from the electronic medical record system and sends them to the server, including medical history, details of symptoms, and past prescription drug information.

[1010] Examples of data anonymization

[1011] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects privacy.

[1012] Examples of data structuring

[1013] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[1014] Specific examples of data analysis

[1015] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[1016] Example of report generation and delivery

[1017] The server creates a report summarizing the analysis results in charts and graphs and distributes it to relevant medical institutions, research institutes, and pharmaceutical companies. The received report is displayed on the terminal (a PC or tablet at the medical institution) to doctors, who use it as reference information for diagnosis and treatment plans.

[1018] Example prompts for generative AI models

[1019] Please describe the symptoms of high blood pressure in detail.

[1020] Please also include the number of heart attacks you have had in the past year.

[1021] The above is the "Mode for Carrying Out the Invention."

[1022] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1023] Step 1:

[1024] A user exports a patient's medical record from an electronic medical record system and sends it to a server in a specified format (e.g., XML, JSON). The data exported by the user includes the patient's medical history, symptom details, and past prescription drug information. The input is the exported medical record data, and the output is the medical record data sent to the server.

[1025] Step 2:

[1026] The server automatically removes or anonymizes the patient's personal information (such as name, address, and phone number) from the received medical record data. This process uses the Python Pandas library. The input is the received medical record data, and the output is the data from which personal information has been removed or anonymized.

[1027] Step 3:

[1028] The server converts the anonymized medical record data into structured data using natural language processing (NLP) algorithms. To do this, it uses Hugging Face's Transformers library and extracts important medical information using a Named Entity Recognition (NER) pipeline. The input is anonymized medical record data, and the output is structured data.

[1029] Step 4:

[1030] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis can improve diagnostic accuracy and discover new treatments. The input is structured data, and the output is the analysis results.

[1031] Step 5:

[1032] The server generates a report based on the analysis results. The report includes statistical data, graphs, and a list of detected symptoms and patterns. Libraries such as Pandas and Matplotlib are used to generate the report. The input is the analysis results, and the output is the generated report.

[1033] Step 6:

[1034] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers using email distribution services and cloud storage. The input is the generated report, and the output is the distributed report.

[1035] Step 7:

[1036] The device (smartphone) is used by doctors and staff at medical facilities to collect patient data and display reports in real time. It is built using mobile development frameworks such as React Native. This application allows users to input prompts to the generative AI model and quickly obtain analysis results. The input is the prompts to the generative AI model, and the output is the analysis results displayed in real time.

[1037] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1038] The present invention is a system that securely analyzes medical record data collected from medical institutions and provides diagnostic support information, and further improves the accuracy of diagnostic information and advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the present invention are described in detail below.

[1039] System Configuration

[1040] 1. Medical record data collection method

[1041] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[1042] The server receives the medical record data sent from the medical institution and checks the consistency and format of the data.

[1043] 2. Anonymization measures

[1044] The server then processes the received medical record data to remove or anonymize the patient's personal information, thereby ensuring patient privacy.

[1045] 3. Data preprocessing methods

[1046] The server checks the quality of the medical record data and automatically or semi-automatically corrects missing information and typos, using natural language processing (NLP) technology to convert text-based data into structured data.

[1047] 4. Data normalization methods

[1048] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[1049] 5. Analysis Methods Using Generative AI

[1050] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[1051] 6. Report Generation Methods

[1052] The server then generates a report based on the analysis results, which includes disease characteristics, newly discovered symptoms, graphs, and charts.

[1053] 7. Report Delivery Method

[1054] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1055] The terminal (a medical institution's computer or tablet) reads the report received from the server and displays it so that the doctor can easily view it during the examination.

[1056] 8. Emotion Engine

[1057] The server incorporates an emotion engine that recognizes the user's emotions and collects emotion data. The emotion engine uses sensors such as cameras and microphones to analyze emotions from the user's facial expressions and voice.

[1058] 9. Utilizing Emotional Data

[1059] The server uses the emotional data collected by the emotion engine to customize diagnostic information and advice, for example providing additional reassuring information if the user is feeling anxious.

[1060] The user (doctor) will use the emotion data to adjust communication with the patient and care plan. This information will be fed back to medical institutions and research institutes to help them further improve their services.

[1061] Specific examples

[1062] Examples of data collection and anonymization

[1063] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, prescription drug information, etc.

[1064] The server analyzes the received medical record data and anonymizes the patient's personal information, for example, removing the patient's name and address.

[1065] Examples of data preprocessing and normalization

[1066] The server corrects grammatical errors and missing values ​​from the medical record data and uses natural language processing technology to structure sentences such as "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[1067] The data is converted into a unified format, and data from different medical institutions is centralized in the same format.

[1068] Specific examples of analysis and report generation using generative AI

[1069] The server uses generative AI to extract analysis results such as "50% of hypertension patients have experienced a heart attack within the past five years."

[1070] The analysis results are generated as a report including graphs and charts.

[1071] Examples of emotion engines

[1072] The terminal (a device equipped with a camera and microphone used in medical institutions) captures the patient's facial expressions and voice during the examination and analyzes them using an emotion engine.

[1073] The server uses the data obtained from the emotion engine to customize diagnostic information, such as adding relaxation information to patients who show anxious expressions.

[1074] Specific examples of report distribution and diagnostic support

[1075] The server sends the generated report to Medical Institution B, a research institute, or a pharmaceutical company.

[1076] The terminal (a personal computer at Medical Institution B) presents the report to the doctor in a viewable format, and is used to assist in diagnosis and treatment planning.

[1077] This system will improve the quality of medical care and provide useful information to those involved by consistently collecting and analyzing medical records and utilizing emotional data.

[1078] The processing flow will be explained below.

[1079] Step 1: Data collection

[1080] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[1081] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[1082] Step 2: Anonymize the data

[1083] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[1084] Step 3: Preprocessing the data

[1085] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[1086] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[1087] Step 4: Normalize the data

[1088] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[1089] Step 5: Data analysis with generative AI

[1090] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[1091] Step 6: Generate reports

[1092] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[1093] Step 7: Report Distribution

[1094] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1095] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[1096] Step 8: Diagnostic support

[1097] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[1098] Step 9: Collect emotion data

[1099] The terminal (a device equipped with a camera or microphone at the medical institution) captures the patient's facial expressions and voice during the examination and sends them to the server.

[1100] Step 10: Analyze the sentiment data

[1101] The server analyzes the received emotional data using an emotion engine. For example, it uses facial expression analysis and voice analysis technology to determine the patient's emotional state (e.g., anxiety, relief, anger, etc.).

[1102] Step 11: Customize diagnostic information

[1103] The server then customizes diagnostic information and advice based on the emotional data obtained from the emotion engine. For example, if the patient is anxious, it may provide additional relaxation information or reference materials.

[1104] Step 12: Feedback of Emotional Data

[1105] The server will then provide feedback to medical and research institutions on emotion data and customized diagnostic information based on it, which will be used for case studies and to improve treatment.

[1106] In this way, the system of the present invention performs consistent processing from collecting medical record data to analyzing it and utilizing emotional data, thereby improving the quality of medical care and providing useful information to those involved.

[1107] Example 2

[1108] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1109] In modern medical settings, a large amount of data is managed as electronic medical records, and there is a need to utilize these records to improve diagnostic accuracy and develop treatment plans tailored to individual patients. It is also becoming increasingly important to provide medical care that takes into account the patient's emotions and psychological state. However, the format and quality of data collected from multiple medical institutions is not uniform, and unifying and analyzing this data requires a great deal of effort. Furthermore, there is a lack of technology to incorporate patient emotional data into diagnoses. Therefore, there is a need for a system that can efficiently collect, anonymize, preprocess, normalize, and analyze medical data, as well as utilize emotional data in a consistent manner.

[1110] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1111] In this invention, the server includes means for collecting medical data from multiple medical institutions, means for anonymizing personal information from the collected medical data, means for converting the anonymized medical data into structured data, means for analyzing the structured data using a generative AI model and extracting health status characteristics, means for compiling the analysis results into a report, means for distributing the report to various medical institutions, means for collecting emotional data, and means for customizing diagnostic information and advice using the emotional data.This enables efficient organization and analysis of medical data and realizes the provision of diagnostic information that takes patient emotions into consideration.

[1112] "Medical data" refers to information managed by medical institutions, including patient medical records, details of symptoms, prescription drug information, etc.

[1113] "Anonymization" is the process of removing or replacing a patient's personal information from medical data so that individuals cannot be identified.

[1114] "Structured data" is data that is organized and formatted according to certain rules, and includes database formats and JSON formats.

[1115] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques such as BERT and GPT, and is a technology for performing data analysis and pattern extraction.

[1116] "Analysis" is the process of extracting features and patterns from collected and anonymized medical data using a generative AI model.

[1117] A "report" is a document that summarizes the results of an analysis and provides specific data and its interpretation using graphs and charts.

[1118] "Various medical institutions" are organizations that require medical data, including medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[1119] "Emotional data" refers to data that indicates the emotional state of a patient, obtained from facial expressions and voice, and is collected using a camera and microphone.

[1120] "Diagnostic Information" means information about a patient's health condition that is based on collected and analyzed medical data.

[1121] "Advice" is treatment or care advice provided based on diagnostic information and emotional data.

[1122] "Preprocessing" is the process of checking the quality of medical data, filling in missing information, and correcting typos.

[1123] "Normalization" is the process of converting and organizing different forms of medical data into a unified format.

[1124] The present invention is a system for analyzing medical data collected from medical institutions and providing diagnostic support information. This system includes a function that improves the accuracy of diagnostic information and advice by taking into account patient emotional data. Specific embodiments of the system are described in detail below.

[1125] System Configuration Overview

[1126] Medical record data collection method

[1127] The user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system to CSV or XML format and sends them to the server using a secure communication protocol (such as HTTPS).

[1128] The server receives this data and performs a data consistency check, for example, to check whether the number of columns and data format of the CSV file match.

[1129] Medical record data anonymization method

[1130] The server anonymizes personal information from the received medical data by replacing the patient's name, address, etc. with a random ID, and also uses a personal information filtering algorithm to prevent personal information from being accidentally left behind.

[1131] Data preprocessing measures

[1132] The server automatically checks the quality of the medical data, for example by completing missing values ​​and correcting typos. It also uses natural language processing (NLP) technology to convert text data into structured data. Specifically, it uses open-source NLP libraries (e.g., spaCy and NLTK).

[1133] Data normalization measures

[1134] The server converts the structured data into a unified format using schema mapping techniques, for example, to convert data from different medical institutions into a common database format (SQL or NoSQL).

[1135] Analysis methods using generative AI

[1136] The server analyzes the data using generative AI (e.g., BERT or GPT models). This AI model extracts health features and hidden patterns from patient data. For example, it finds patterns such as "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[1137] Report Generation Method

[1138] The server generates a report based on the analysis results, including the diagnosis, details of any symptoms detected, and graphs and charts, using data visualization tools (e.g., Matplotlib or Tableau).

[1139] Report delivery method

[1140] The server distributes the generated reports to various medical institutions via secure email or a dedicated data distribution platform.

[1141] The terminal (computer or tablet at the medical institution) receives the report from the server and makes it available for the doctor to view during the consultation. The corresponding application is installed on the device.

[1142] Emotion engine emotion data collection method

[1143] The device (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to a server in real time. This data is then analyzed using emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[1144] How to use emotion data

[1145] The server then customizes diagnostic information and advice based on the emotional data, for example adding information that promotes relaxation if the patient expresses anxiety.

[1146] The user (doctor) uses the emotional data to adjust communication with the patient and provide appropriate treatment.

[1147] Examples and prompts

[1148] Examples of data collection and anonymization

[1149] The user (IT staff at Medical Institution A) exports "Patient A's medical records, symptom details, and prescription drug information" from the electronic medical record system and sends them to the server.

[1150] The server receives "Patient A's medical records" and anonymizes them by replacing personal information (name, address) with a random ID.

[1151] Specific examples of analysis and report generation using generative AI

[1152] The server uses an AI model to analyze a "dataset of hypertension patients" and calculate the "percentage of patients who have experienced a heart attack within the past five years."

[1153] Based on the results of this analysis, a "heart attack risk report for hypertensive patients" is generated and sent to medical institution B in PDF format.

[1154] Specific operation of the emotion engine

[1155] The terminal (a device with a camera) captures the patient's facial expression during the examination and transmits data indicating "anxiety" to the server in real time.

[1156] The server analyzes this emotional data and adds relaxation information (for example, deep breathing techniques) to the diagnostic information for patients who show "anxiety."

[1157] Prompt Sentence Examples

[1158] "Please tell me how to extract patient medical history, symptom details, and prescription drug information from medical institution medical record data, anonymize them, and then analyze them."

[1159] "Please explain the specific steps to extract disease characteristics and hidden patterns from medical record data using generative AI."

[1160] "If a patient is feeling anxious during a consultation, can you tell me specifically how we can use the emotion engine to customize the diagnostic information?"

[1161] This system enables consistent collection, analysis, and utilization of emotional data in medical data, improving the accuracy and efficiency of diagnostic support.

[1162] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1163] Step 1:

[1164] Medical record data collection

[1165] The user (IT staff at a medical institution) exports patient medical records from the electronic medical record system to CSV or XML format and sends them to the server. The input is the patient's medical data, and the output is the raw data received by the server.

[1166] The server receives this data and performs a data integrity check, specifically checking the file format and basic data validation.

[1167] Step 2:

[1168] Medical record data anonymization

[1169] The server processes the received medical data to remove or anonymize personal information. The input is the received medical record data, and the output is the anonymized data.

[1170] The server replaces personally identifiable information, such as the patient's name and address, with a random ID, and then uses an algorithm to remove personal information.

[1171] Step 3:

[1172] Data Preprocessing

[1173] The server checks the quality of the medical record data and automatically corrects missing information and typos. The input is anonymized medical data, and the output is quality-improved data.

[1174] Natural language processing (NLP) techniques are used to convert text into structured data. Specifically, open-source NLP libraries (e.g., spaCy and NLTK) are used to convert sentences such as "The patient has high blood pressure" into "{'Medical history': 'High blood pressure'}".

[1175] Step 4:

[1176] Data normalization

[1177] The server converts the structured data into a unified format. The input is the preprocessed structured data, and the output is the unified format data.

[1178] Perform schema mapping to convert data from different healthcare institutions into a common format (e.g., SQL or NoSQL database format).

[1179] Step 5:

[1180] Data analysis with generative AI

[1181] The server performs data analysis using generative AI (e.g., BERT or GPT models). The input is normalized structured data, and the output is the analysis results.

[1182] AI models extract health features and hidden patterns from input data, yielding insights such as "50% of people with high blood pressure have experienced a heart attack within the past five years."

[1183] Step 6:

[1184] Report Generation

[1185] The server creates a report based on the analysis results. The input is the analysis results from the AI ​​model, and the output is a report (e.g., in PDF format).

[1186] The report includes diagnostic findings, details of any symptoms found, and graphs and charts, and is generated using a data visualization tool (e.g., Matplotlib or Tableau).

[1187] Step 7:

[1188] Report Distribution

[1189] The server distributes the generated reports to various medical institutions. The input is the generated report and the output is the transmitted report.

[1190] Delivery will be via secure email or a dedicated data delivery platform.

[1191] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[1192] Step 8:

[1193] Emotion engine emotion data collection

[1194] The terminal (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to the server in real time. The input is the patient's facial expressions and voice, and the output is the captured emotional data.

[1195] The collected emotional data is analyzed by emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[1196] Step 9:

[1197] Utilizing Emotional Data

[1198] The server customizes diagnostic information and advice based on the emotional data. The input is the analyzed emotional data, and the output is customized diagnostic information.

[1199] For example, if the patient is showing signs of anxiety, add information that has a relaxing effect.

[1200] Based on this emotional data, the user (doctor) can adjust communication with the patient and provide appropriate treatment.

[1201] This system enables efficient collection and analysis of medical data and utilization of emotional data, improving the accuracy and efficiency of diagnostic support.

[1202] (Application example 2)

[1203] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1204] Conventional health management systems are limited to collecting and analyzing medical record data, making it difficult to provide flexible health support based on users' emotions and individual health conditions. Furthermore, healthcare information provided is often standardized, lacking specific advice tailored to each user's condition. This makes it difficult for users to properly manage their health, creating a demand for more personalized health support.

[1205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1206] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing the user's personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to related institutions, means for collecting user health condition data and providing health information based on the analysis results, emotion analysis means for recognizing the user's emotions, and means for customizing the health information using the emotion data. This makes it possible to provide health support information that is individually customized based on the user's health condition and emotions.

[1207] "Medical record data" refers to medical information such as medical records and prescription information collected by medical institutions during the course of treating patients.

[1208] "Anonymization" is the process of deleting or processing a patient's personal information from collected medical record data so that a specific individual cannot be identified.

[1209] "Structured data" is data that has been converted from vast amounts of medical information into a unified format or database format, making it easier to analyze.

[1210] "Generative AI" is a machine learning model that learns from large amounts of data and generates and analyzes new information and patterns. It is a technology used in natural language processing and image recognition.

[1211] A "report" is a document or data file that summarizes the analysis results of generative AI and the diagnostic information based on them, and is provided in a format that can be used by users and related institutions.

[1212] "Related institutions" include medical institutions, research institutions, pharmaceutical companies and medical information providers.

[1213] "Health status data" refers to information about a user's physical and mental health status, including vital signs and medical records.

[1214] "Emotion analysis" is a technology that detects and analyzes emotions from a user's facial expressions and voice, and is used to understand the user's psychological state.

[1215] "Customization" refers to individually adjusting and providing information and services that are tailored to the individual circumstances and needs of each user.

[1216] This invention is a system for the healthcare and food delivery fields that analyzes a user's health condition and emotional information and provides individually customized health information and meal plans. This system collects and analyzes medical record data from medical institutions and combines it with emotional analysis of the user to provide more personalized health support.

[1217] System Configuration

[1218] 1. Medical record data collection method

[1219] The user (IT staff at the medical institution) exports medical information from the electronic medical record system and sends it to the server. The medical record data includes medical history, symptoms, and prescription information.

[1220] The server receives the medical record data and checks the data for consistency and format.

[1221] 2. Anonymization measures

[1222] The server deletes or anonymizes personal information from the received medical record data, ensuring privacy and preventing third parties from identifying specific individuals.

[1223] 3. Data preprocessing methods

[1224] The server checks the quality of the medical record data, automatically corrects missing information and typos, and uses natural language processing (NLP) technology to convert text data into structured data.

[1225] 4. Data normalization methods

[1226] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[1227] 5. Analysis Methods Using Generative AI

[1228] The server uses generative AI (e.g., BERT or GPT models) to analyze health status and hidden patterns, and predicts the user's health status and risk.

[1229] 6. Report Generation Methods

[1230] The server compiles the analysis results from the generative AI into a report, which includes the characteristics of the disease and the analysis results along with graphs and charts.

[1231] 7. Report Delivery Method

[1232] The server transmits the generated report to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1233] The terminal (a computer or tablet at the medical institution) reads the report and displays it so that the doctor can view it during the examination.

[1234] 8. Emotion analysis method

[1235] The server incorporates an emotion engine that recognizes the user's emotions, and collects and analyzes emotion data using sensors such as cameras and microphones. The emotion engine determines the user's emotions from their facial expressions and voice.

[1236] 9. Means of customizing health information

[1237] The server then uses the data obtained through emotion analysis to customize health information and advice for the user. For example, if the user is feeling stressed, it will suggest relaxing meals and exercise.

[1238] Specific examples

[1239] Health analysis: Based on the user's health data, a low-salt meal plan is suggested for a user diagnosed with high blood pressure. Generative AI is also used to include analysis results in the report, such as "50% of high blood pressure patients have experienced a heart attack within the past five years."

[1240] Emotion analysis: If the user's facial expressions and voice data are captured during the consultation and they are feeling anxious or stressed, the system will suggest relaxing herbal tea or soothing music.

[1241] Prompt Sentence Examples

[1242] Suggest recipes that are best for users with high blood pressure.

[1243] If the user is feeling anxious, suggest a relaxing meal.

[1244] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1245] Step 1:

[1246] Medical record data collection

[1247] Input: Medical record data provided by medical institutions (CSV or XML format)

[1248] Operation: The user (IT staff at the medical institution) exports the target data from the electronic medical record system and sends it to the server.

[1249] Output: Medical record data saved on the server

[1250] Step 2:

[1251] Data anonymization

[1252] Input: Collected medical record data

[1253] How it works: The server removes or anonymizes personal information such as names and addresses from the patient records by replacing personal information fields in the database with random values.

[1254] Output: De-identified medical record data

[1255] Step 3:

[1256] Data Preprocessing

[1257] Input: De-identified medical record data

[1258] How it works: The server checks the quality of the data, corrects missing information and typos, and uses natural language processing (NLP) to structure an example like "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[1259] Output: Preprocessed structured data

[1260] Step 4:

[1261] Data normalization

[1262] Input: Preprocessed structured data

[1263] How it works: The server converts data provided in different formats into a unified format, making it easier to analyze and integrate diverse data sets.

[1264] Output: Normalized structured data

[1265] Step 5:

[1266] Analysis by generative AI

[1267] Input: Normalized structured data

[1268] How it works: The server analyzes data using a generative AI model (e.g., BERT or GPT) to detect disease signatures, commonly overlooked symptoms, and outliers. The generative AI leverages patterns and knowledge learned from the dataset to form a predictive model.

[1269] Output: Analysis results

[1270] Step 6:

[1271] Report Generation

[1272] Input: Analysis results of the generating AI

[1273] How it works: The server compiles the analysis results into a report, which includes discovered disease characteristics, proposed new treatments, and graphs and charts.

[1274] Output: Generated report

[1275] Step 7:

[1276] Report Distribution

[1277] Input: Generated report

[1278] Operation: The server distributes the generated report to related institutions (medical institutions, research institutions, pharmaceutical companies, medical information providers). The terminal (a PC or tablet at the medical institution) receives the report and displays it in a viewable format.

[1279] Output: Reports delivered to medical institutions and related organizations

[1280] Step 8:

[1281] Emotion analysis

[1282] Input: User's facial expressions and voice data

[1283] How it works: The device (a device with a camera and microphone) captures the user's facial expressions and voice during the consultation, and the server performs emotion analysis. Specifically, it recognizes faces from camera footage and analyzes emotional tones from voice data.

[1284] Output: Emotion analysis results

[1285] Step 9:

[1286] Customized health information

[1287] Input: Analysis results of the generation AI, emotion analysis results

[1288] How it works: The server customizes health information and advice based on your health and emotional state. For example, if a user has high blood pressure and is feeling anxious, it might suggest a relaxing meal plan and specific recipes.

[1289] Output: Customized health information and advice

[1290] Example prompt:

[1291] Suggest recipes that are best for users with high blood pressure.

[1292] If the user is feeling anxious, suggest a relaxing meal.

[1293] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1294] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1295] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1296] [Fourth embodiment]

[1297] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1298] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1299] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1300] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1301] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1302] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1303] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1304] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1305] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1306] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1307] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1308] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1309] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1310] The present invention is a system that collects medical record data from multiple medical institutions, stores it in a secure database, and analyzes it. The following components are required to implement this system:

[1311] System Configuration

[1312] 1. Medical record data collection method

[1313] The server is designed to receive medical record data sent from each medical institution. At the medical institution, the user (IT staff at the medical institution) converts the medical record data into a specific format and sends it to the server via a secure communication path.

[1314] 2. Anonymization measures

[1315] The server has a processing function that automatically removes or anonymizes personal information from the received medical record data, while appropriately protecting personally identifiable information such as the patient's name, address, and telephone number.

[1316] 3. Structured Data Conversion Methods

[1317] The server uses natural language processing (NLP) algorithms to convert textual medical record data into structured data that can be easily searched and analyzed in a database.

[1318] 4. Analysis Methods Using Generative AI

[1319] The server analyzes the structured data using generative AI (e.g., BERT or GPT models). The purpose of the analysis is to extract disease characteristics and hidden symptoms, which will contribute to improving diagnostic accuracy in medical settings and discovering new treatments.

[1320] 5. Report Generation Methods

[1321] The server generates a report based on the analysis results, including statistics and graphs from the analysis, and a list of discovered patterns and symptoms.

[1322] 6. Report Delivery Method

[1323] The server distributes the generated reports in an appropriate format to medical institutions, research institutes, pharmaceutical companies, and medical information providers. Terminals (e.g., doctors' computers or tablets) receive the distributed reports and make them viewable.

[1324] Specific examples

[1325] Examples of data collection

[1326] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, and past prescription drug information.

[1327] Examples of data anonymization

[1328] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects personal information.

[1329] Examples of data structuring

[1330] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[1331] Specific examples of data analysis

[1332] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[1333] Example of report generation and delivery

[1334] The server creates a report summarizing the analysis results in a chart and distributes it to related medical institutions (e.g., medical institutions A and B), research institutions, and pharmaceutical companies.

[1335] The terminal (medical institution B's personal computer) displays the received report to the doctor, who uses it as reference information for diagnosis and treatment planning.

[1336] In this way, the system of the present invention consistently performs the process from collecting medical record data to analyzing it and providing the information, thereby improving the quality of medical care and providing useful information to all parties involved in medical care.

[1337] The processing flow will be explained below.

[1338] Step 1: Data collection

[1339] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[1340] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[1341] Step 2: Anonymize the data

[1342] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[1343] Step 3: Preprocessing the data

[1344] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[1345] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[1346] Step 4: Normalize the data

[1347] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[1348] Step 5: Data analysis with generative AI

[1349] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[1350] Step 6: Generate reports

[1351] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[1352] Step 7: Report Distribution

[1353] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1354] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[1355] Step 8: Diagnostic support

[1356] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[1357] Through these steps, this system will consistently collect, analyze, and provide information on medical records, contributing to improved diagnostic accuracy and research progress throughout the medical industry.

[1358] Example 1

[1359] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1360] Conventional medical data management systems have problems with the efficiency of data management and analysis due to the lack of consistent anonymization, structuring, analysis, and feedback of results for medical record data collected from multiple medical institutions. Furthermore, the lack of a function to instruct the generative AI model on individual analysis content makes advanced medical analysis difficult. To solve these issues, a comprehensive and efficient system is needed.

[1361] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1362] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing patient personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI model and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for instructing the generative AI model on the analysis content using prompt statements. This makes it possible to consistently perform data anonymization, structuring, analysis, and feedback, thereby achieving efficient and advanced medical analysis.

[1363] The "collection means" refers to a device or program for receiving medical record data from multiple medical institutions.

[1364] An "anonymization means" is a device or program that removes personal information from collected medical record data or converts it into an unidentifiable form.

[1365] "Means for converting into structured data" refers to a device or program that formats text-based medical record data into a format that is easy to search and analyze in a database.

[1366] "Means for analyzing and extracting disease characteristics" refers to a device or program that uses a generative AI model to analyze structured data and identify disease characteristics and trends.

[1367] A "means for compiling a report" is a device or program that creates a report containing statistical data, graphs, patterns, etc. based on the analysis results.

[1368] The "distribution means" refers to a device or program that electronically transmits the generated report to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[1369] "Means for instructing a generative AI model on analysis content using a prompt sentence" refers to a device or program that uses input in the form of a sentence to instruct a generative AI model on specific analysis content.

[1370] MODE FOR CARRYING OUT THE INVENTION

[1371] The present invention provides a consistent system for collecting medical record data from multiple medical institutions, anonymizing the data, structuring it, analyzing it, and providing feedback. Specific embodiments of this system are described below.

[1372] (Collection of medical record data)

[1373] The server receives medical record data sent from each medical institution. To do this, the user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system and sends them to the server using a secure communication channel (e.g., HTTPS). The data sent includes medical history, details of symptoms, and past prescription drug information.

[1374] (Data anonymization)

[1375] The server automatically removes or anonymizes the patient's personal information (e.g., name, address, phone number) from the received medical record data, and in the process, utilizes NLP algorithms to accurately detect personal information within the data.

[1376] (Data structuring)

[1377] The server then uses natural language processing (NLP) to convert the anonymized medical record data into structured data. This structure makes it easier to search and analyze within the database. For example, a statement such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into the format "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[1378] (Data analysis using generative AI)

[1379] The server analyzes the structured data using a generative AI model (e.g., BERT or GPT). During this process, a prompt is used to instruct the generative AI model on specific analysis content. For example, by entering the prompt "Analyze the frequency of heart attacks in patients with high blood pressure," the AI ​​model will perform an analysis on patients with high blood pressure. The resulting statistical data is that "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[1380] (Report generation and distribution)

[1381] The server creates a report based on the analysis results of the generative AI model. This report includes analysis results, statistical data, graphs, medical treatment patterns, etc. The completed report is distributed in an appropriate format (e.g., PDF format) to medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[1382] (Receiving reports)

[1383] The device (for example, the doctor's computer or tablet) receives the delivered report and makes it viewable, allowing the doctor to use the report as reference information for diagnosis and treatment planning.

[1384] This system will improve the quality of medical care and provide useful information to related parties by consistently collecting, analyzing, and providing feedback on medical record data.

[1385] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1386] Step 1: Collect medical record data

[1387] The user (IT staff at the medical institution) exports patient medical records from the electronic medical record system. Then, this exported data is formatted into a specific format (e.g., CSV format). The formatted data is sent to the server using a security protocol (e.g., HTTPS). The input is raw medical record data, and the output is formatted medical record data. Specifically, the export function of the electronic medical record system is used to execute the data formatting script.

[1388] Step 2: Receiving the data

[1389] The server receives data sent via the HTTPS protocol and temporarily stores it. The input is formatted medical record data, and the output is medical record data stored on the server. Specifically, the receiving server program is executed to process the data.

[1390] Step 3: Anonymize data

[1391] The server automatically identifies personal information (e.g., name, address, phone number) from the received medical record data and deletes or anonymizes this information. The input is the received medical record data, and the output is the anonymized data. Specifically, it runs a personal information identification program using an NLP algorithm and replaces the identified personal information with, for example, "Patient X."

[1392] Step 4: Structuring your data

[1393] The server converts the anonymized medical record data into structured data using a natural language processing (NLP) algorithm. The input is the anonymized medical record data, and the output is structured data. Specifically, an NLP analysis program is run to convert the unstructured data into a format such as "{'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}".

[1394] Step 5: Analyze the data

[1395] The server analyzes the data using a generative AI model (e.g., BERT or GPT) based on the structured data. The input is the structured data and a prompt, and the output is the analysis result. Specifically, the prompt "Please analyze the frequency of heart attacks in patients with high blood pressure" is entered into the generative AI model, and the analysis begins when the analysis execution button is pressed. As a result, data such as "50% of patients with high blood pressure have experienced a heart attack within the past five years" is obtained.

[1396] Step 6: Generate reports

[1397] The server creates a report containing information such as statistical data, graphs, and medical treatment patterns based on the analysis results of the generative AI model. The input is the analysis results, and the output is the report. Specifically, the analysis results are input into a report generation program and formatted into a format (e.g., PDF).

[1398] Step 7: Report Distribution

[1399] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers via a secure communication protocol. The input is the generated report, and the output is the receiving system of the distribution destination. Specifically, it runs a program that uploads the report to email or a dedicated portal.

[1400] Step 8: Receive and view the report

[1401] The terminal (doctor's PC or tablet) receives the delivered report and makes it available for viewing by the user. The input is the delivered report, and the output is the displayed report. Specifically, a PDF reader or dedicated application is launched to display the received report.

[1402] (Application example 1)

[1403] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1404] Conventional medical data collection and analysis systems do not fully integrate the process of collecting data from multiple medical institutions, anonymizing it, converting it into structured data, and analyzing it using generative AI. This makes it difficult to display information in real time, especially when used on-site at medical facilities. This can lead to delays in timely diagnostic support and treatment planning.

[1405] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1406] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing personal patient information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to medical institutions, research institutions, pharmaceutical companies, and medical information providers, and means for collecting patient data and displaying the report in real time using a smartphone application running in medical facilities. This allows for the collection, analysis, and provision of medical record data all in one process, enabling real-time diagnostic support, particularly in medical facilities.

[1407] "Multiple medical institutions" refers to facilities that provide medical services, such as multiple different hospitals, clinics, and other medical facilities.

[1408] "Medical record data" refers to electronic or paper medical data that includes information such as a patient's medical records, medical history, prescription history, and test results.

[1409] "Anonymization" is the process of protecting personal information by removing or transforming information that can identify a specific individual (such as name, address, or phone number).

[1410] "Structured data" is data that is organized according to a specific format and can be easily searched and analyzed within a database.

[1411] "Generative AI" is artificial intelligence that uses natural language processing algorithms and deep learning models (e.g., BERT and GPT) to analyze large datasets and generate new information and patterns.

[1412] A "report" is a document created based on the results of analysis by generative AI, containing information such as statistics, graphs, and a list of discovered patterns and symptoms.

[1413] "Medical facilities" are facilities that provide medical services, such as hospitals, clinics, pharmacies, etc.

[1414] A "smartphone application" is software that runs on a smartphone and is designed to perform a specific task.

[1415] "Real-time" refers to data processing and information delivery occurring almost immediately.

[1416] This system collects medical record data from multiple medical institutions, anonymizes the data, converts it into structured data, analyzes it using generative AI, and generates and distributes reports. It also has the ability to collect patient data in real time and display the analysis results using a smartphone application running in medical facilities.

[1417] System Components

[1418] 1. Methods for collecting medical record data

[1419] The server receives the medical record data sent from each medical institution. This reception uses a secure communication protocol (e.g., HTTPS). At the medical institution, IT personnel export patient data from the electronic medical record system and send it to the server in a specified format (e.g., XML, JSON).

[1420] 2. Anonymization measures

[1421] The server automatically deletes or anonymizes personal information (such as names, addresses, and phone numbers) from the received medical record data. This process can be performed using a Python data processing library (e.g., Pandas).

[1422] 3. Structured Data Conversion Methods

[1423] The server uses natural language processing (NLP) algorithms to convert the text data from the medical records into structured data, using Hugging Face's Transformers library, which can then be used in a Named Entity Recognition (NER) pipeline to extract key medical information.

[1424] 4. Analysis Methods Using Generative AI

[1425] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[1426] 5. Report Generation Methods

[1427] The server generates a report based on the analysis results, which includes statistical data, graphs, and a list of symptoms and patterns discovered. Report generation can use libraries such as Pandas and Matplotlib.

[1428] 6. Report Delivery Method

[1429] The server distributes the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers. For this purpose, an email distribution service (e.g., SMTP server) or cloud storage (e.g., AWS S3) can be used.

[1430] 7. Smartphone Applications

[1431] The device (smartphone) will be used by doctors and staff in healthcare facilities to collect patient data and display reports in real time. This application can be built using mobile development frameworks such as React Native.

[1432] Specific examples

[1433] Examples of data collection

[1434] A user (IT staff at a medical institution) exports a patient's medical records from the electronic medical record system and sends them to the server, including medical history, details of symptoms, and past prescription drug information.

[1435] Examples of data anonymization

[1436] The server removes the patient's name, address, and phone number from the received medical record data and anonymizes it as "Patient A." This process protects privacy.

[1437] Examples of data structuring

[1438] The server converts the statement in the medical record, "The patient has high blood pressure and has had two heart attacks in the past year," into structured data like this: { 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': '1 year'}.

[1439] Specific examples of data analysis

[1440] The server uses generative AI to analyze the "frequency of heart attacks in patients with high blood pressure" and discovers that 50% of patients with high blood pressure have experienced a heart attack within the past five years.

[1441] Example of report generation and delivery

[1442] The server creates a report summarizing the analysis results in charts and graphs and distributes it to relevant medical institutions, research institutes, and pharmaceutical companies. The received report is displayed on the terminal (a PC or tablet at the medical institution) to doctors, who use it as reference information for diagnosis and treatment plans.

[1443] Example prompts for generative AI models

[1444] Please describe the symptoms of high blood pressure in detail.

[1445] Please also include the number of heart attacks you have had in the past year.

[1446] The above is the "Mode for Carrying Out the Invention."

[1447] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1448] Step 1:

[1449] A user exports a patient's medical record from an electronic medical record system and sends it to a server in a specified format (e.g., XML, JSON). The data exported by the user includes the patient's medical history, symptom details, and past prescription drug information. The input is the exported medical record data, and the output is the medical record data sent to the server.

[1450] Step 2:

[1451] The server automatically removes or anonymizes the patient's personal information (such as name, address, and phone number) from the received medical record data. This process uses the Python Pandas library. The input is the received medical record data, and the output is the data from which personal information has been removed or anonymized.

[1452] Step 3:

[1453] The server converts the anonymized medical record data into structured data using natural language processing (NLP) algorithms. To do this, it uses Hugging Face's Transformers library and extracts important medical information using a Named Entity Recognition (NER) pipeline. The input is anonymized medical record data, and the output is structured data.

[1454] Step 4:

[1455] The server analyzes the structured data using generative AI (e.g., BERT, GPT) to extract disease characteristics and hidden symptoms. This analysis can improve diagnostic accuracy and discover new treatments. The input is structured data, and the output is the analysis results.

[1456] Step 5:

[1457] The server generates a report based on the analysis results. The report includes statistical data, graphs, and a list of detected symptoms and patterns. Libraries such as Pandas and Matplotlib are used to generate the report. The input is the analysis results, and the output is the generated report.

[1458] Step 6:

[1459] The server distributes the generated reports to related medical institutions, research institutes, pharmaceutical companies, and medical information providers using email distribution services and cloud storage. The input is the generated report, and the output is the distributed report.

[1460] Step 7:

[1461] The device (smartphone) is used by doctors and staff at medical facilities to collect patient data and display reports in real time. It is built using mobile development frameworks such as React Native. This application allows users to input prompts to the generative AI model and quickly obtain analysis results. The input is the prompts to the generative AI model, and the output is the analysis results displayed in real time.

[1462] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1463] The present invention is a system that securely analyzes medical record data collected from medical institutions and provides diagnostic support information, and further improves the accuracy of diagnostic information and advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the present invention are described in detail below.

[1464] System Configuration

[1465] 1. Medical record data collection method

[1466] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[1467] The server receives the medical record data sent from the medical institution and checks the consistency and format of the data.

[1468] 2. Anonymization measures

[1469] The server then processes the received medical record data to remove or anonymize the patient's personal information, thereby ensuring patient privacy.

[1470] 3. Data preprocessing methods

[1471] The server checks the quality of the medical record data and automatically or semi-automatically corrects missing information and typos, using natural language processing (NLP) technology to convert text-based data into structured data.

[1472] 4. Data normalization methods

[1473] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[1474] 5. Analysis Methods Using Generative AI

[1475] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data. This analysis is expected to improve diagnostic accuracy and discover new treatments.

[1476] 6. Report Generation Methods

[1477] The server then generates a report based on the analysis results, which includes disease characteristics, newly discovered symptoms, graphs, and charts.

[1478] 7. Report Delivery Method

[1479] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1480] The terminal (a medical institution's computer or tablet) reads the report received from the server and displays it so that the doctor can easily view it during the examination.

[1481] 8. Emotion Engine

[1482] The server incorporates an emotion engine that recognizes the user's emotions and collects emotion data. The emotion engine uses sensors such as cameras and microphones to analyze emotions from the user's facial expressions and voice.

[1483] 9. Utilizing Emotional Data

[1484] The server uses the emotional data collected by the emotion engine to customize diagnostic information and advice, for example providing additional reassuring information if the user is feeling anxious.

[1485] The user (doctor) will use the emotion data to adjust communication with the patient and care plan. This information will be fed back to medical institutions and research institutes to help them further improve their services.

[1486] Specific examples

[1487] Examples of data collection and anonymization

[1488] A user (IT staff member at Medical Institution A) exports a patient's medical records from the electronic medical record system and sends them to the server. The data sent includes the patient's medical history, details of symptoms, prescription drug information, etc.

[1489] The server analyzes the received medical record data and anonymizes the patient's personal information, for example, removing the patient's name and address.

[1490] Examples of data preprocessing and normalization

[1491] The server corrects grammatical errors and missing values ​​from the medical record data and uses natural language processing technology to structure sentences such as "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[1492] The data is converted into a unified format, and data from different medical institutions is centralized in the same format.

[1493] Specific examples of analysis and report generation using generative AI

[1494] The server uses generative AI to extract analysis results such as "50% of hypertension patients have experienced a heart attack within the past five years."

[1495] The analysis results are generated as a report including graphs and charts.

[1496] Examples of emotion engines

[1497] The terminal (a device equipped with a camera and microphone used in medical institutions) captures the patient's facial expressions and voice during the examination and analyzes them using an emotion engine.

[1498] The server uses the data obtained from the emotion engine to customize diagnostic information, such as adding relaxation information to patients who show anxious expressions.

[1499] Specific examples of report distribution and diagnostic support

[1500] The server sends the generated report to Medical Institution B, a research institute, or a pharmaceutical company.

[1501] The terminal (a personal computer at Medical Institution B) presents the report to the doctor in a viewable format, and is used to assist in diagnosis and treatment planning.

[1502] This system will improve the quality of medical care and provide useful information to those involved by consistently collecting and analyzing medical records and utilizing emotional data.

[1503] The processing flow will be explained below.

[1504] Step 1: Data collection

[1505] The user (IT staff at the medical institution) converts the medical record data into a specific format (for example, CSV or XML format) and sends it to the server.

[1506] The server receives the medical record data sent from the medical institution and checks whether the data is consistent and in the correct format.

[1507] Step 2: Anonymize the data

[1508] The server removes or anonymizes personal information (such as name, address, and telephone number) from the received medical record data. This process protects the patient's personal information from being identified by third parties.

[1509] Step 3: Preprocessing the data

[1510] The server performs quality checks on the medical record data and automatically or semi-automatically corrects missing information and typos.

[1511] The server uses natural language processing (NLP) technology to analyze textual medical record data and convert it into structured data. For example, a sentence such as "The patient has high blood pressure and has experienced two heart attacks in the past year" is converted into a format such as "{ 'Medical history': 'High blood pressure', 'Number of heart attacks': 2, 'Duration': 1 year}".

[1512] Step 4: Normalize the data

[1513] The server normalizes the structured data and standardizes the data format from different medical institutions, facilitating subsequent analysis.

[1514] Step 5: Data analysis with generative AI

[1515] The server uses generative AI (e.g., BERT or GPT models) to extract disease features and hidden patterns from the structured data, such as obtaining statistical information like "50% of hypertension patients have experienced a heart attack within the past five years."

[1516] Step 6: Generate reports

[1517] The server then generates a report based on the generative AI's analysis results in an easy-to-understand format, including graphs and charts, that includes the characteristics of the disease, any symptoms detected, and a summary of the analysis results.

[1518] Step 7: Report Distribution

[1519] The server transmits the generated reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1520] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[1521] Step 8: Diagnostic support

[1522] The user (doctor) checks the report displayed on the device and makes a diagnosis and treatment plan based on the analysis results. For example, they may provide advice to the patient or perform additional tests based on trends and new characteristics of the disease.

[1523] Step 9: Collect emotion data

[1524] The terminal (a device equipped with a camera or microphone at the medical institution) captures the patient's facial expressions and voice during the examination and sends them to the server.

[1525] Step 10: Analyze the sentiment data

[1526] The server analyzes the received emotional data using an emotion engine. For example, it uses facial expression analysis and voice analysis technology to determine the patient's emotional state (e.g., anxiety, relief, anger, etc.).

[1527] Step 11: Customize diagnostic information

[1528] The server then customizes diagnostic information and advice based on the emotional data obtained from the emotion engine. For example, if the patient is anxious, it may provide additional relaxation information or reference materials.

[1529] Step 12: Feedback of Emotional Data

[1530] The server will then provide feedback to medical and research institutions on emotion data and customized diagnostic information based on it, which will be used for case studies and to improve treatment.

[1531] In this way, the system of the present invention performs consistent processing from collecting medical record data to analyzing it and utilizing emotional data, thereby improving the quality of medical care and providing useful information to those involved.

[1532] Example 2

[1533] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1534] In modern medical settings, a large amount of data is managed as electronic medical records, and there is a need to utilize these records to improve diagnostic accuracy and develop treatment plans tailored to individual patients. It is also becoming increasingly important to provide medical care that takes into account the patient's emotions and psychological state. However, the format and quality of data collected from multiple medical institutions is not uniform, and unifying and analyzing this data requires a great deal of effort. Furthermore, there is a lack of technology to incorporate patient emotional data into diagnoses. Therefore, there is a need for a system that can efficiently collect, anonymize, preprocess, normalize, and analyze medical data, as well as utilize emotional data in a consistent manner.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1536] In this invention, the server includes means for collecting medical data from multiple medical institutions, means for anonymizing personal information from the collected medical data, means for converting the anonymized medical data into structured data, means for analyzing the structured data using a generative AI model and extracting health status characteristics, means for compiling the analysis results into a report, means for distributing the report to various medical institutions, means for collecting emotional data, and means for customizing diagnostic information and advice using the emotional data.This enables efficient organization and analysis of medical data and realizes the provision of diagnostic information that takes patient emotions into consideration.

[1537] "Medical data" refers to information managed by medical institutions, including patient medical records, details of symptoms, prescription drug information, etc.

[1538] "Anonymization" is the process of removing or replacing a patient's personal information from medical data so that individuals cannot be identified.

[1539] "Structured data" is data that is organized and formatted according to certain rules, and includes database formats and JSON formats.

[1540] A "generative AI model" is an artificial intelligence model that uses natural language processing techniques such as BERT and GPT, and is a technology for performing data analysis and pattern extraction.

[1541] "Analysis" is the process of extracting features and patterns from collected and anonymized medical data using a generative AI model.

[1542] A "report" is a document that summarizes the results of an analysis and provides specific data and its interpretation using graphs and charts.

[1543] "Various medical institutions" are organizations that require medical data, including medical institutions, research institutions, pharmaceutical companies, and medical information providers.

[1544] "Emotional data" refers to data that indicates the emotional state of a patient, obtained from facial expressions and voice, and is collected using a camera and microphone.

[1545] "Diagnostic Information" means information about a patient's health condition that is based on collected and analyzed medical data.

[1546] "Advice" is treatment or care advice provided based on diagnostic information and emotional data.

[1547] "Preprocessing" is the process of checking the quality of medical data, filling in missing information, and correcting typos.

[1548] "Normalization" is the process of converting and organizing different forms of medical data into a unified format.

[1549] The present invention is a system for analyzing medical data collected from medical institutions and providing diagnostic support information. This system includes a function that improves the accuracy of diagnostic information and advice by taking into account patient emotional data. Specific embodiments of the system are described in detail below.

[1550] System Configuration Overview

[1551] Medical record data collection method

[1552] The user (IT staff at the medical institution) exports the patient's medical records from the electronic medical record system to CSV or XML format and sends them to the server using a secure communication protocol (such as HTTPS).

[1553] The server receives this data and performs a data consistency check, for example, to check whether the number of columns and data format of the CSV file match.

[1554] Medical record data anonymization method

[1555] The server anonymizes personal information from the received medical data by replacing the patient's name, address, etc. with a random ID, and also uses a personal information filtering algorithm to prevent personal information from being accidentally left behind.

[1556] Data preprocessing measures

[1557] The server automatically checks the quality of the medical data, for example by completing missing values ​​and correcting typos. It also uses natural language processing (NLP) technology to convert text data into structured data. Specifically, it uses open-source NLP libraries (e.g., spaCy and NLTK).

[1558] Data normalization measures

[1559] The server converts the structured data into a unified format using schema mapping techniques, for example, to convert data from different medical institutions into a common database format (SQL or NoSQL).

[1560] Analysis methods using generative AI

[1561] The server analyzes the data using generative AI (e.g., BERT or GPT models). This AI model extracts health features and hidden patterns from patient data. For example, it finds patterns such as "50% of patients with high blood pressure have experienced a heart attack within the past five years."

[1562] Report Generation Method

[1563] The server generates a report based on the analysis results, including the diagnosis, details of any symptoms detected, and graphs and charts, using data visualization tools (e.g., Matplotlib or Tableau).

[1564] Report delivery method

[1565] The server distributes the generated reports to various medical institutions via secure email or a dedicated data distribution platform.

[1566] The terminal (computer or tablet at the medical institution) receives the report from the server and makes it available for the doctor to view during the consultation. The corresponding application is installed on the device.

[1567] Emotion engine emotion data collection method

[1568] The device (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to a server in real time. This data is then analyzed using emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[1569] How to use emotion data

[1570] The server then customizes diagnostic information and advice based on the emotional data, for example adding information that promotes relaxation if the patient expresses anxiety.

[1571] The user (doctor) uses the emotional data to adjust communication with the patient and provide appropriate treatment.

[1572] Examples and prompts

[1573] Examples of data collection and anonymization

[1574] The user (IT staff at Medical Institution A) exports "Patient A's medical records, symptom details, and prescription drug information" from the electronic medical record system and sends them to the server.

[1575] The server receives "Patient A's medical records" and anonymizes them by replacing personal information (name, address) with a random ID.

[1576] Specific examples of analysis and report generation using generative AI

[1577] The server uses an AI model to analyze a "dataset of hypertension patients" and calculate the "percentage of patients who have experienced a heart attack within the past five years."

[1578] Based on the results of this analysis, a "heart attack risk report for hypertensive patients" is generated and sent to medical institution B in PDF format.

[1579] Specific operation of the emotion engine

[1580] The terminal (a device with a camera) captures the patient's facial expression during the examination and transmits data indicating "anxiety" to the server in real time.

[1581] The server analyzes this emotional data and adds relaxation information (for example, deep breathing techniques) to the diagnostic information for patients who show "anxiety."

[1582] Prompt Sentence Examples

[1583] "Please tell me how to extract patient medical history, symptom details, and prescription drug information from medical institution medical record data, anonymize them, and then analyze them."

[1584] "Please explain the specific steps to extract disease characteristics and hidden patterns from medical record data using generative AI."

[1585] "If a patient is feeling anxious during a consultation, can you tell me specifically how we can use the emotion engine to customize the diagnostic information?"

[1586] This system enables consistent collection, analysis, and utilization of emotional data in medical data, improving the accuracy and efficiency of diagnostic support.

[1587] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1588] Step 1:

[1589] Medical record data collection

[1590] The user (IT staff at a medical institution) exports patient medical records from the electronic medical record system to CSV or XML format and sends them to the server. The input is the patient's medical data, and the output is the raw data received by the server.

[1591] The server receives this data and performs a data integrity check, specifically checking the file format and basic data validation.

[1592] Step 2:

[1593] Medical record data anonymization

[1594] The server processes the received medical data to remove or anonymize personal information. The input is the received medical record data, and the output is the anonymized data.

[1595] The server replaces personally identifiable information, such as the patient's name and address, with a random ID, and then uses an algorithm to remove personal information.

[1596] Step 3:

[1597] Data Preprocessing

[1598] The server checks the quality of the medical record data and automatically corrects missing information and typos. The input is anonymized medical data, and the output is quality-improved data.

[1599] Natural language processing (NLP) techniques are used to convert text into structured data. Specifically, open-source NLP libraries (e.g., spaCy and NLTK) are used to convert sentences such as "The patient has high blood pressure" into "{'Medical history': 'High blood pressure'}".

[1600] Step 4:

[1601] Data normalization

[1602] The server converts the structured data into a unified format. The input is the preprocessed structured data, and the output is the unified format data.

[1603] Perform schema mapping to convert data from different healthcare institutions into a common format (e.g., SQL or NoSQL database format).

[1604] Step 5:

[1605] Data analysis with generative AI

[1606] The server performs data analysis using generative AI (e.g., BERT or GPT models). The input is normalized structured data, and the output is the analysis results.

[1607] AI models extract health features and hidden patterns from input data, yielding insights such as "50% of people with high blood pressure have experienced a heart attack within the past five years."

[1608] Step 6:

[1609] Report Generation

[1610] The server creates a report based on the analysis results. The input is the analysis results from the AI ​​model, and the output is a report (e.g., in PDF format).

[1611] The report includes diagnostic findings, details of any symptoms found, and graphs and charts, and is generated using a data visualization tool (e.g., Matplotlib or Tableau).

[1612] Step 7:

[1613] Report Distribution

[1614] The server distributes the generated reports to various medical institutions. The input is the generated report and the output is the transmitted report.

[1615] Delivery will be via secure email or a dedicated data delivery platform.

[1616] The terminal (a computer or tablet at the medical institution) reads the report received from the server and displays it so that the doctor can view it during the examination.

[1617] Step 8:

[1618] Emotion engine emotion data collection

[1619] The terminal (a device with a camera and microphone) captures the patient's facial expressions and voice and transmits them to the server in real time. The input is the patient's facial expressions and voice, and the output is the captured emotional data.

[1620] The collected emotional data is analyzed by emotion analysis algorithms, such as OpenFace for facial expression recognition and Praat for voice analysis.

[1621] Step 9:

[1622] Utilizing Emotional Data

[1623] The server customizes diagnostic information and advice based on the emotional data. The input is the analyzed emotional data, and the output is customized diagnostic information.

[1624] For example, if the patient is showing signs of anxiety, add information that has a relaxing effect.

[1625] Based on this emotional data, the user (doctor) can adjust communication with the patient and provide appropriate treatment.

[1626] This system enables efficient collection and analysis of medical data and utilization of emotional data, improving the accuracy and efficiency of diagnostic support.

[1627] (Application example 2)

[1628] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1629] Conventional health management systems are limited to collecting and analyzing medical record data, making it difficult to provide flexible health support based on users' emotions and individual health conditions. Furthermore, healthcare information provided is often standardized, lacking specific advice tailored to each user's condition. This makes it difficult for users to properly manage their health, creating a demand for more personalized health support.

[1630] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1631] In this invention, the server includes means for collecting medical record data from multiple medical institutions, means for anonymizing the user's personal information from the collected medical record data, means for converting the anonymized medical record data into structured data, means for analyzing the structured data using a generative AI and extracting disease characteristics, means for compiling the analysis results into a report, means for distributing the report to related institutions, means for collecting user health condition data and providing health information based on the analysis results, emotion analysis means for recognizing the user's emotions, and means for customizing the health information using the emotion data. This makes it possible to provide health support information that is individually customized based on the user's health condition and emotions.

[1632] "Medical record data" refers to medical information such as medical records and prescription information collected by medical institutions during the course of treating patients.

[1633] "Anonymization" is the process of deleting or processing a patient's personal information from collected medical record data so that a specific individual cannot be identified.

[1634] "Structured data" is data that has been converted from vast amounts of medical information into a unified format or database format, making it easier to analyze.

[1635] "Generative AI" is a machine learning model that learns from large amounts of data and generates and analyzes new information and patterns. It is a technology used in natural language processing and image recognition.

[1636] A "report" is a document or data file that summarizes the analysis results of generative AI and the diagnostic information based on them, and is provided in a format that can be used by users and related institutions.

[1637] "Related institutions" include medical institutions, research institutions, pharmaceutical companies and medical information providers.

[1638] "Health status data" refers to information about a user's physical and mental health status, including vital signs and medical records.

[1639] "Emotion analysis" is a technology that detects and analyzes emotions from a user's facial expressions and voice, and is used to understand the user's psychological state.

[1640] "Customization" refers to individually adjusting and providing information and services that are tailored to the individual circumstances and needs of each user.

[1641] This invention is a system for the healthcare and food delivery fields that analyzes a user's health condition and emotional information and provides individually customized health information and meal plans. This system collects and analyzes medical record data from medical institutions and combines it with emotional analysis of the user to provide more personalized health support.

[1642] System Configuration

[1643] 1. Medical record data collection method

[1644] The user (IT staff at the medical institution) exports medical information from the electronic medical record system and sends it to the server. The medical record data includes medical history, symptoms, and prescription information.

[1645] The server receives the medical record data and checks the data for consistency and format.

[1646] 2. Anonymization measures

[1647] The server deletes or anonymizes personal information from the received medical record data, ensuring privacy and preventing third parties from identifying specific individuals.

[1648] 3. Data preprocessing methods

[1649] The server checks the quality of the medical record data, automatically corrects missing information and typos, and uses natural language processing (NLP) technology to convert text data into structured data.

[1650] 4. Data normalization methods

[1651] The server converts the structured data into a unified format and integrates data from different medical institutions, making it easier to analyze.

[1652] 5. Analysis Methods Using Generative AI

[1653] The server uses generative AI (e.g., BERT or GPT models) to analyze health status and hidden patterns, and predicts the user's health status and risk.

[1654] 6. Report Generation Methods

[1655] The server compiles the analysis results from the generative AI into a report, which includes the characteristics of the disease and the analysis results along with graphs and charts.

[1656] 7. Report Delivery Method

[1657] The server transmits the generated report to medical institutions, research institutes, pharmaceutical companies, and medical information providers.

[1658] The terminal (a computer or tablet at the medical institution) reads the report and displays it so that the doctor can view it during the examination.

[1659] 8. Emotion analysis method

[1660] The server incorporates an emotion engine that recognizes the user's emotions, and collects and analyzes emotion data using sensors such as cameras and microphones. The emotion engine determines the user's emotions from their facial expressions and voice.

[1661] 9. Means of customizing health information

[1662] The server then uses the data obtained through emotion analysis to customize health information and advice for the user. For example, if the user is feeling stressed, it will suggest relaxing meals and exercise.

[1663] Specific examples

[1664] Health analysis: Based on the user's health data, a low-salt meal plan is suggested for a user diagnosed with high blood pressure. Generative AI is also used to include analysis results in the report, such as "50% of high blood pressure patients have experienced a heart attack within the past five years."

[1665] Emotion analysis: If the user's facial expressions and voice data are captured during the consultation and they are feeling anxious or stressed, the system will suggest relaxing herbal tea or soothing music.

[1666] Prompt Sentence Examples

[1667] Suggest recipes that are best for users with high blood pressure.

[1668] If the user is feeling anxious, suggest a relaxing meal.

[1669] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1670] Step 1:

[1671] Medical record data collection

[1672] Input: Medical record data provided by medical institutions (CSV or XML format)

[1673] Operation: The user (IT staff at the medical institution) exports the target data from the electronic medical record system and sends it to the server.

[1674] Output: Medical record data saved on the server

[1675] Step 2:

[1676] Data anonymization

[1677] Input: Collected medical record data

[1678] How it works: The server removes or anonymizes personal information such as names and addresses from the patient records by replacing personal information fields in the database with random values.

[1679] Output: De-identified medical record data

[1680] Step 3:

[1681] Data Preprocessing

[1682] Input: De-identified medical record data

[1683] How it works: The server checks the quality of the data, corrects missing information and typos, and uses natural language processing (NLP) to structure an example like "The patient has high blood pressure" into "{ 'Medical history': 'High blood pressure'}".

[1684] Output: Preprocessed structured data

[1685] Step 4:

[1686] Data normalization

[1687] Input: Preprocessed structured data

[1688] How it works: The server converts data provided in different formats into a unified format, making it easier to analyze and integrate diverse data sets.

[1689] Output: Normalized structured data

[1690] Step 5:

[1691] Analysis by generative AI

[1692] Input: Normalized structured data

[1693] How it works: The server analyzes data using a generative AI model (e.g., BERT or GPT) to detect disease signatures, commonly overlooked symptoms, and outliers. The generative AI leverages patterns and knowledge learned from the dataset to form a predictive model.

[1694] Output: Analysis results

[1695] Step 6:

[1696] Report Generation

[1697] Input: Analysis results of the generating AI

[1698] How it works: The server compiles the analysis results into a report, which includes discovered disease characteristics, proposed new treatments, and graphs and charts.

[1699] Output: Generated report

[1700] Step 7:

[1701] Report Distribution

[1702] Input: Generated report

[1703] Operation: The server distributes the generated report to related institutions (medical institutions, research institutions, pharmaceutical companies, medical information providers). The terminal (a PC or tablet at the medical institution) receives the report and displays it in a viewable format.

[1704] Output: Reports delivered to medical institutions and related organizations

[1705] Step 8:

[1706] Emotion analysis

[1707] Input: User's facial expressions and voice data

[1708] How it works: The device (a device with a camera and microphone) captures the user's facial expressions and voice during the consultation, and the server performs emotion analysis. Specifically, it recognizes faces from camera footage and analyzes emotional tones from voice data.

[1709] Output: Emotion analysis results

[1710] Step 9:

[1711] Customized health information

[1712] Input: Analysis results of the generation AI, emotion analysis results

[1713] How it works: The server customizes health information and advice based on your health and emotional state. For example, if a user has high blood pressure and is feeling anxious, it might suggest a relaxing meal plan and specific recipes.

[1714] Output: Customized health information and advice

[1715] Example prompt:

[1716] Suggest recipes that are best for users with high blood pressure.

[1717] If the user is feeling anxious, suggest a relaxing meal.

[1718] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1719] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1720] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1721] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1722] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1723] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1724] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1725] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1726] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1727] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1728] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1729] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1730] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1731] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1732] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1733] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1734] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1735] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1736] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1737] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1738] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1739] The following is further disclosed regarding the above embodiment.

[1740] (Claim 1)

[1741] A means for collecting medical record data from multiple medical institutions;

[1742] A means of anonymizing patient personal information from the collected medical record data;

[1743] A means for converting the anonymized medical record data into structured data;

[1744] A method for analyzing structured data using generative AI and extracting disease characteristics;

[1745] A means of summarizing the results of the analysis in a report,

[1746] A means of distributing the reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers;

[1747] A system including:

[1748] (Claim 2)

[1749] 2. The system of claim 1, further comprising means for checking the quality of the medical record data and automatically correcting missing information and typographical errors in the preprocessing.

[1750] (Claim 3)

[1751] The system of claim 1, further comprising means for automatically detecting symptoms and abnormal values ​​that are often overlooked through analysis using generative AI and including them in the report.

[1752] "Example 1"

[1753] (Claim 1)

[1754] A means for collecting medical record data from multiple medical institutions;

[1755] A means of anonymizing patient personal information from the collected medical record data;

[1756] A means for converting the anonymized medical record data into structured data;

[1757] A means of analyzing structured data using a generative AI model to extract disease characteristics;

[1758] A means of summarizing the results of the analysis in a report,

[1759] A means of distributing the reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers;

[1760] A means of instructing the generative AI model on the analysis content using prompt sentences;

[1761] A system including:

[1762] (Claim 2)

[1763] 2. The system of claim 1,...

Claims

1. A means for collecting medical record data from multiple medical institutions; A means of anonymizing patient personal information from the collected medical record data; A means for converting the anonymized medical record data into structured data; A method for analyzing structured data using generative AI and extracting disease characteristics; A means of summarizing the results of the analysis in a report, A means of distributing the reports to medical institutions, research institutes, pharmaceutical companies, and medical information providers; A system including:

2. 2. The system of claim 1, further comprising means for checking the quality of the medical record data and automatically correcting missing information and typographical errors in the preprocessing.

3. The system of claim 1 further comprising means for automatically detecting symptoms and abnormal values ​​that are often overlooked through analysis using generative AI and including them in the report.

Images