Medical information presentation system, management device, medical information presentation method, and medical information presentation program

Abstract

The present invention aims to provide appropriate answers to inquiries regarding medical reports. [Solution] The cloud management server 10 has a general-purpose LLM 16h, a database 15 that manages knowledge graph data 15a generated based on multiple vectorized information obtained by vectorizing text contained in multiple medical information, and an information presentation unit 16g that, when question data related to a medical report is received from a doctor's terminal 30, transmits answer data that responds to the question data in the medical report to the doctor's terminal 30 based on the database 15 and the LLM 16h.

Description

[Technical Field]

[0001] The present invention relates to a medical information presentation system, a management device, a medical information presentation method, and a medical information presentation program that can present appropriate answers to inquiries about medical reports. [Background technology]

[0002] 2. Description of the Related Art Conventionally, doctors and radiologists (hereinafter collectively referred to as "doctors") interpret examination images captured by modalities such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), and compile the results of the interpretation into a medical report.

[0003] A technology for supporting the creation of such medical reports is known (see, for example, Patent Document 1). In Patent Document 1, medical reports are managed by associating diagnosed disease names and lesion information with diagnostic reports. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-179319 Summary of the Invention [Problem to be solved by the invention]

[0005] However, while the technology disclosed in Patent Document 1 can reduce the effort required to input lesions and lesion-related medical information into medical reports, it cannot detect typographical errors, omissions, or possible inconsistencies in the written content contained in the medical reports, and therefore cannot sufficiently reduce the burden on doctors involved in preparing medical reports. Furthermore, there is a possibility that errors or mistakes in the written content of medical reports could have a negative impact on medical safety. Note that this problem is not limited to medical reports, but occurs similarly with various types of medical information.

[0006] The present invention has been made to solve the problems (issues) of the above-mentioned conventional technology, and aims to provide a medical information presentation system, management device, medical information presentation method, and medical information presentation program that can present appropriate answers to inquiries regarding medical reports. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the present invention provides a medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, the management device including a large-scale language model that learns a plurality of texts to understand and generate sentences, a database that manages a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information, and a predetermined terminal device that manages a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information. Requests for medical reports, including medical reports When the database as a search extension generation to obtain related information related to the medical report, and The large-scale language model Generated by an information presentation means for presenting, to the terminal device, response information responding to the request for the medical report; When the information presentation means receives a request for presentation of a continuation sentence following a sentence written in the medical report as a request related to the medical report, the information presentation means vectorizes the medical report, searches the knowledge graph network to acquire related information related to the medical report, inputs the acquired related information and the request for presentation of the continuation sentence of the medical report into the large-scale language model, and presents the medical report including the continuation sentence output from the large-scale language model to the terminal device as the response information. It is characterized by:

[0011] The present invention also provides A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, wherein the management device comprises: a large-scale language model that learns a plurality of texts to understand and generate sentences; a database that manages a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information; and an information presentation means that, when a request for a medical report including a medical report is received from a predetermined terminal device, uses the database as a search extension generation means to acquire related information related to the medical report, and presents to the terminal device response information that responds to the request for the medical report and that is generated by the large-scale language model using the request for the medical report and the related information; The information presentation means The medical report is a request for the medical report. When a request for additional information to be added to the information listed in the above is received, Report is vectorized and the knowledge graph network is searched to find the medical Acquire related information related to the report, and compare the acquired related information with the medical report. A request for presentation of an additional sentence is input to the large-scale language model, and the medical information including the additional sentence is output from the large-scale language model. Report is presented to the terminal device as the response information. Ruko It is characterized by the following.

[0012] The present invention also provides A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, wherein the management device comprises: a large-scale language model that learns a plurality of texts to understand and generate sentences; a database that manages a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information; and an information presentation means that, when a request for a medical report including a medical report is received from a predetermined terminal device, uses the database as a search extension generation means to acquire related information related to the medical report, and presents to the terminal device response information that responds to the request for the medical report and that is generated by the large-scale language model using the request for the medical report and the related information; The information presentation means As a request regarding the medical report The medical Report When a request is received to present a contradiction in the content of the text described in Reportis vectorized and the knowledge graph network is searched to find the medical Report and acquiring related information related to the medical device. Report A content contradiction presentation request is input to the large-scale language model, and the medical information is output from the large-scale language model. Report and presenting to the terminal device response information including the part of the content that is inconsistent with the content of the Ruko It is characterized by the following.

[0013] The present invention also provides A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, wherein the management device comprises: a large-scale language model that learns a plurality of texts to understand and generate sentences; a database that manages a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information; and an information presentation means that, when a request for a medical report including a medical report is received from a predetermined terminal device, uses the database as a search extension generation means to acquire related information related to the medical report, and presents to the terminal device response information that responds to the request for the medical report and that is generated by the large-scale language model using the request for the medical report and the related information; The information presentation means As a request regarding the medical report The medical Report If we receive a request to provide a typographical error in the text described in Report is vectorized and the knowledge graph network is searched to find the medical Report and acquiring related information related to the medical device. Report A request for displaying a misspelling is input to the large-scale language model, and the medical information output from the large-scale language model is Report The method is characterized in that response information including the typographical error is presented to the terminal device. Furthermore, in the above invention, the management device includes a label information extraction means for extracting each of a plurality of pieces of label information from text included in the plurality of pieces of medical information, a vectorization information generation means for vectorizing each of the plurality of pieces of label information to generate a plurality of pieces of vectorization information, and a graph network generation means for generating the knowledge graph network based on the plurality of pieces of vectorization information.

[0014] Furthermore, in the above invention, the database comprises a first knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of medical information acquired from a server device of a medical institution, and a second knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of publicly available information related to diseases whose number of infected people has increased by more than a predetermined amount within a predetermined period, and the information presenting means is configured to present the information to a predetermined terminal device. The medical report When the request is received, Report is vectorized to form the first knowledge graph network and the second knowledge graph network as a search extension generation to obtain related information corresponding to the request for the medical report, and The large-scale language model Generated by The response information is presented to the terminal device.

[0015] The present invention also provides a management device for managing medical information including medical images captured by a predetermined modality device and medical reports of the medical images, the management device comprising a large-scale language model that learns a plurality of texts to understand and generate sentences, a database that manages a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information, and a medical report management system that manages medical information from a predetermined terminal device. Report When a request is received, the database and the large-scale language model as a search extension generation to obtain related information corresponding to the request for the medical report, and The large-scale language model a request for the medical report generated by and an information presentation means for presenting response information to the terminal device. When the information presentation means receives a request for presentation of a continuation sentence following a sentence written in the medical report as a request related to the medical report, the information presentation means vectorizes the medical report, searches the knowledge graph network to acquire related information related to the medical report, inputs the acquired related information and the request for presentation of the continuation sentence of the medical report into the large-scale language model, and presents the medical report including the continuation sentence output from the large-scale language model to the terminal device as the response information. It is characterized by:

[0016] The present invention also provides a medical information presentation method in a medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, the management device receiving medical information from a predetermined terminal device. Medical report When a predetermined request relating to the above is received, a knowledge graph network is generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information. Utilizing search expansion generation to generate requests related to said medical report an acquisition step of acquiring related information related to the management device; using the relevant information and the request for the medical report; Large-scale language models that learn from multiple texts to understand and generate sentences responding to a request for the medical report generated by and a presentation step of presenting response information to the terminal device. and the presenting step, when receiving a request for presenting a continuation sentence following a sentence written in the medical report as a request related to the medical report, vectorizes the medical report, searches the knowledge graph network to acquire related information related to the medical report, inputs the acquired related information and the request for presenting the continuation sentence of the medical report to the large-scale language model, and presents the medical report including the continuation sentence output from the large-scale language model to the terminal device as the response information. It is characterized by the following.

[0017] The present invention also provides a medical information presentation program executed in a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, the medical information presentation program being ... Report When a request is received, a knowledge graph network is generated based on multiple vectorized information obtained by vectorizing text contained in multiple medical information. as a search extension generator for the medical report request. an acquisition procedure for acquiring relevant related information; using the relevant information and the request for the medical report; Large-scale language models that learn from multiple texts to understand and generate sentences responding to a request for the medical report generated bya presentation procedure for presenting response information to the terminal device; The presentation step includes, when a request for presentation of a continuation sentence following a sentence described in the medical report is received as a request related to the medical report, vectorizing the medical report, searching the knowledge graph network to acquire related information related to the medical report, inputting the acquired related information and the request for presentation of the continuation sentence of the medical report into the large-scale language model, and presenting the medical report including the continuation sentence output from the large-scale language model to the terminal device as the response information. The method is characterized by being executed by a computer. [Effects of the Invention]

[0018] According to the present invention, it is possible to provide an appropriate answer to an inquiry about a medical report. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram showing an overview of a medical information presentation system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing the system configuration of the medical information presentation system. [Figure 3] FIG. 3 is a functional block diagram illustrating the configuration of the cloud management server illustrated in FIG. [Figure 4] FIG. 4 is a diagram illustrating an example of vectorization processing. [Figure 5] FIG. 5 is a diagram showing an example of question data and answer data stored in the cloud management server shown in FIG. [Figure 6] FIG. 6 is a diagram (part 1) for explaining an example of the knowledge graph data shown in FIG. [Figure 7] FIG. 7 is a diagram (part 2) for explaining an example of the knowledge graph data shown in FIG. [Figure 8] FIG. 8 is a functional block diagram showing the configuration of the local server shown in FIG. [Figure 9] FIG. 9 is a diagram showing an example of disease name candidate presentation according to this embodiment. [Figure 10] FIG. 10 is a diagram showing an example of presentation of a continuation sentence according to this embodiment. [Figure 11] FIG. 11 is a diagram showing an example of presentation of an additional text according to this embodiment. [Figure 12] FIG. 12 is a diagram showing an example of content contradiction presentation according to this embodiment. [Figure 13] FIG. 13 is a diagram showing an example of the presentation of typos and omissions according to the first embodiment. [Figure 14] FIG. 14 is a sequence diagram showing the processing procedure for updating knowledge graph data in the medical information presentation system. [Figure 15] FIG. 15 is a sequence diagram showing a procedure for presenting response data by the cloud management server. [Figure 16] FIG. 16 is an explanatory diagram for explaining a case where a plurality of knowledge graph data are used. [Figure 17] FIG. 17 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0020] The following describes in detail embodiments of a medical information presentation system, a management device, a medical information presentation method, and a medical information presentation program according to the present invention, with reference to the accompanying drawings. In this embodiment, the medical information is mainly a medical report (such as a diagnostic report) prepared by a doctor, but the medical information can also be a radiological report prepared by a radiological doctor or other documents prepared by a doctor.

[0021] In this embodiment, "medical information" refers to information including medical images generated by modality devices, personal information of recipients (such as name, date of birth, and age) corresponding to the medical images added in accordance with the DICOM (Digital Imaging and Communications in Medicine) standard, and information related to image generation (such as the date and time of image generation, the name and number of the modality device that generated the image, the type of examination, and examination area). Furthermore, in this embodiment, a "medical report" refers to a diagnostic report or the like that includes text entered by a doctor in the findings section. Furthermore, in this embodiment, "medical data" refers to data including a medical report and medical images.

[0022] <Overview of the medical information presentation system> An overview of the medical information presentation system according to this embodiment will be described. FIG. 1 is an explanatory diagram for explaining the overview of the medical information presentation system according to this embodiment. Data 15a relating to a knowledge graph network (hereinafter referred to as "knowledge graph data") is stored in advance in a database 15 of a cloud management server 10 shown in FIG. 1. This knowledge graph data 15a is a graph network constructed based on vectorized information obtained by vectorizing multiple pieces of label information extracted from text contained in multiple pieces of anonymized medical information received from multiple medical institutions.

[0023] The medical information presentation system according to this embodiment is a system that can receive answer data to question data when question data including a medical report is sent from the doctor terminal 30 to the cloud management server 10. For example, when the doctor terminal 30 sends question data including the question "Suggest disease name candidates for the medical report" and the medical report, answer data of disease name candidates can be obtained.

[0024] Here, the cloud management server 10 uses the knowledge graph data 15a as retrieval-augmented generation (RAG). After acquiring related data searched from the knowledge graph data 15a, the cloud management server 10 provides the question data and related data to a large-scale language model (LLM) 16h to acquire answer data. Note that the LLM 16h does not use a large amount of medical information as training data, but is a general-purpose large-scale language model, and therefore uses the knowledge graph data 15a, which is retrieval-augmented generation (RAG).

[0025] As shown in FIG. 1, the doctor's terminal 30 transmits a medical report containing text in the findings section such as "Pain in the right buttock, numbness on the outside of the right lower leg, and pain in the buttocks in the same position for the past 2-3 years..." and question data including a question for "Suggestion of possible disease names" to the cloud management server 10 (S1).

[0026] When the cloud management server 10 receives the question data, it vectorizes the label information included in the question data to generate a question vector. Then, the information presenter 16g searches the knowledge graph data 15a using the question vector (S2) and acquires related data (S3).

[0027] Thereafter, the information presenter 16g inputs the question data and related data into the LLM 16h (S4), acquires answer data (S5), and transmits the answer data, including a disease name candidate such as "Lumbar spinal canal stenosis suspected," to the doctor terminal 30 (S6).

[0028] As described above, in the medical information presentation system according to this embodiment, the cloud management server 10 is configured to include a general-purpose LLM 16h, a database 15 that manages knowledge graph data 15a generated based on multiple vectorized information obtained by vectorizing text contained in multiple pieces of medical information, and an information presentation unit 16g that, when receiving question data related to a medical report from the doctor terminal 30, transmits answer data to the doctor terminal 30 that responds to the question data of the medical report based on the database 15 and the LLM 16h, thereby making it possible to present appropriate answers to inquiries regarding the medical report.

[0029] Here, we have given an example of searching for candidate disease names contained in a medical report, but it is also possible to present continuing sentences following the text in the findings section of the medical report, present additional sentences, present parts of the content that are inconsistent, present parts where there are typos, etc.

[0030] <System configuration of medical information presentation system> Next, the system configuration of the medical information presentation system will be described. Figure 2 is a diagram showing the system configuration of the medical information presentation system. As shown in Figure 2, this medical information presentation system has a cloud management server 10, which is a virtual management server that provides cloud computing services for storing and managing medical information on the cloud, and local servers 20 in multiple medical institutions 1, and the cloud management server 10 and the local servers 20 in the multiple medical institutions 1 are connected via a network N.

[0031] Furthermore, the plurality of medical institutions 1 further have doctor terminals 30 and modality devices (not shown), which are communicably connected to each other via a LAN within the medical institution 1. Furthermore, the doctor terminals 30 are communicably connected to the cloud management server 10 via the network N.

[0032] The local server 20 receives the medical report from the doctor terminal 30. The local server 20 also receives medical information from the modality device. Then, the local server 20 performs anonymization processing on the medical data (including the medical information and the medical report) within the medical institution 1, and transmits the anonymized data to the cloud management server 10 (S10). The reason why the local server 20 performs anonymization processing is to prevent personal information included in the medical information acquired within the medical institution 1 from being identified.

[0033] The cloud management server 10 extracts multiple pieces of label information from the medical information and text included in the medical report of the anonymized data received from the local server 20, and vectorizes each of the multiple pieces of label information to generate vectorized information (S11), and updates (generates) the knowledge graph data 15c based on the multiple vectorized information (S12).

[0034] Furthermore, when the cloud management server 10 receives question data (e.g., disease name candidate presentation) including a medical report from the doctor terminal 30 (S13), it searches the knowledge graph data 15a (S14) and acquires related data. Then, it inputs the question data and related data to the LLM 16h (S15), acquires answer data output from the LLM 16h (S16), and transmits the output of the LLM 16h, i.e., the answer data, to the doctor terminal 30 (S17).

[0035] <Configuration of Cloud Management Server 10> Next, the configuration of the cloud management server 10 of the medical information presentation system according to this embodiment will be described. Fig. 3 is a functional block diagram showing the configuration of the cloud management server 10 shown in Fig. 2. As shown in Fig. 3, the cloud management server 10 has a communication I / F unit 13, a storage unit 14, a database 15, and a control unit 16, and is connected to an input unit 11 and a display unit 12.

[0036] The input unit 11 is an input device such as a keyboard or a mouse, the display unit 12 is a display device such as an LCD panel, and the communication I / F unit 13 is a communication interface unit for connecting to the local server 20 and the doctor's terminal 30 via the network N so as to be able to communicate with them.

[0037] The storage unit 14 is a storage device such as a hard disk drive or nonvolatile memory, and stores anonymized data 14a, label data 14b, question data 14c, and answer data 14d. The anonymized data 14a is anonymized medical information data received from the local servers 20 of multiple medical institutions 1. The anonymized data 14a includes medical reports and examination images.

[0038] The label data 14b is data of label information extracted from text included in the anonymized data 14a. The label information is extracted for each input item, such as an examination request, findings, and diagnosis. The label information is information in which labels corresponding to the location, disease name, certainty of the disease name, size of the detected object, patient attributes, clinical background, and progress record are extracted from text included in multiple pieces of medical information, and labels corresponding to the model of examination equipment, presence or absence of contrast agent, and imaging conditions are extracted from text accompanying medical images included in the medical information.

[0039] The question data 14c is data including a question to the LLM 16h and text included in the medical report. The answer data 14d is data output by the LLM 16h as an answer when the question data 14c and related data are input to the LLM 16h.

[0040] The database 15 is a storage device such as a hard disk drive or nonvolatile memory, and stores knowledge graph data 15a. The knowledge graph data 15a is knowledge graph data that represents multiple labels and the relationships between the labels using nodes and edges. The knowledge graph is generated by vectorizing the label data 14b and the medical information linked to the label data 14b, and inputting the processed vectorized data into a natural language processing model. Note that each node in the knowledge graph data 15a is linked to related medical information.

[0041] The control unit 16 is a control unit that performs overall control of the cloud management server 10, and includes an anonymized data receiving unit 16a, a label generating unit 16b, a vectorization processing unit 16c, a knowledge graph data updating unit 16d, a knowledge graph data managing unit 16e, a request receiving unit 16f, an information presenting unit 16g, and an LLM 16h. In practice, by loading and executing these programs into a CPU, the anonymized data receiving unit 16a, the label generating unit 16b, the vectorization processing unit 16c, the knowledge graph data updating unit 16d, the knowledge graph data managing unit 16e, the request receiving unit 16f, the information presenting unit 16g, and the LLM 16h are caused to execute processes corresponding to the respective programs.

[0042] The anonymized data receiving unit 16a is a processing unit that receives anonymized data obtained by anonymizing medical information transmitted from the local server 20. The anonymized data receiving unit 16a stores the received anonymized data in the storage unit .

[0043] The label generation unit 16b is a processing unit that reads the anonymized data 14a from the storage unit 14 and generates label information from the text of medical information included in the anonymized data 14a. Examples of the label information include the location, disease name, certainty of the disease name (positive, negative, suspected), size of the detected object (tumor, etc.) included in the image diagnosis report, the model of the examination machine (modality), the presence or absence of a contrast agent, imaging conditions included in the information accompanying the examination image, patient attributes (gender, age, family history, medical history, treatment history, medication information, allergy information), clinical history, progress record, etc. included in the information in the clinician's diagnostic report. Note that the label generation unit 16b preferably automatically assigns labels using a machine learning trained model that has been trained in advance using an existing labeled dataset.

[0044] The vectorization processing unit 16c converts the multiple pieces of label information generated by the label generation unit 16b and the text of medical information included in the anonymized data 14a into corresponding vectorized information. For example, when vectorization processing is performed on data labeled in the organ category, data is output in which a name, a material ID, and a vector value correspond to an ID, as shown in FIG. 4(a). In this example, the name "gallbladder," material ID "32252," and vector values ​​"V1, V2, V3, ..." correspond to ID "844256450158," and the name "liver," material ID "32253," and vector values ​​"V5, V6, V7, ..." correspond to ID "844256450159."

[0045] Furthermore, the vectorization processing unit 16c outputs data in which page numbers, document IDs, and vector values ​​are associated with IDs as vector values ​​for the medical information text, for example, as shown in Fig. 4(b). Here, page "109", document ID "32252", and vector values ​​"V10, V11, V12, ..." are associated with ID "7", and page "110", document ID "32252", and vector values ​​"V15, V16, V17, ..." are associated with ID "8".

[0046] The knowledge graph data update unit 16d is a processing unit that extracts entities (such as body parts, symptoms, and disease names) from the vectorized data based on the medical information vectorized by the vectorization processing unit 16c, identifies the relationships between the extracted entities, and generates a knowledge graph with the entities as nodes and the relationships as edges.

[0047] The knowledge graph data management unit 16e is a processing unit that manages the generated knowledge graph data. Since new medical information is generated daily at the medical institution 1, the knowledge graph data 15c also needs to be updated according to the situation. For example, the knowledge graph data management unit 16e monitors updates to the anonymized data 14a, and when the anonymized data 14a is updated, performs label generation processing and vectorization processing on the newly updated differential data, thereby updating the knowledge graph data. Note that the knowledge graph data 15c may be updated all at once by performing batch processing overnight, or may be updated whenever the anonymized data 14a is updated.

[0048] The request receiving unit 16f is a processing unit that receives question data including a medical report transmitted from the doctor terminal 30. When the request receiving unit 16f receives the question data, the information presenting unit 16g is a processing unit that presents answer data corresponding to the question data to the doctor terminal 30 based on the knowledge graph data 15a and the LLM 16h.

[0049] Specifically, the request receiving unit 16f extracts and vectorizes text from the question data received, and then searches the knowledge graph data 15a to obtain related data. The information presenting unit 16g then inputs the question data including the related data into the LLM, obtains output from the LLM that answers the given question data as answer data, and presents the answer data on the doctor terminal 30.

[0050] LLM16h is a large-scale language model that outputs answers based on input. Here, LLM16h is assumed to be a general-purpose large-scale language model trained using a huge amount of training data. Therefore, the training data for LLM16h does not include detailed medical terms or medical reports.

[0051] Next, an example of the question data 14c and the answer data 14d stored in the storage unit 14 shown in FIG. 3 will be described. As shown in FIG. 5(a), the question data 14c is data in which a question ID, a question, a finding, and a vector are associated with each other. Here, a question ID "Q1234" is associated with a question "Disease name candidate search," a finding "A nodule shadow of approximately 2.5 cm was observed in the right upper lobe, ...," and a vector "V21, V22, ...." Also, a question ID "Q1235" is associated with a question "Continuation sentence generation," a finding "High accumulation was observed in the mediastinal lymph nodes, ...," and a vector "V25, V26, ...." In this way, the labels included in the text of the question and the finding are vectorized and stored as vectors.

[0052] As shown in FIG. 5(b), the answer data 14d is data in which an answer ID, a question ID, and an answer are associated with each other. Here, the question ID "Q1234" is associated with the answer ID "A1234," indicating that the answer ID "A1234" is an answer to the question ID "Q1234." The question ID "Q1235" is associated with the answer ID "A1235," indicating that the answer ID "A1235" is an answer to the question ID "Q1235." When the answer data is output from the LLM 16h, the answer data is transmitted to the requesting doctor terminal 30 and stored in the answer data 14d in the storage unit 14.

[0053] <Displaying knowledge graph> Next, an example of the knowledge graph data 15a shown in Fig. 3 will be described. Figs. 6 and 7 are diagrams showing an example of the knowledge graph data 15a shown in Fig. 3. As shown in Fig. 6, the knowledge graph data 15a represents a plurality of labels and the relationships between the labels using nodes and edges. Each node represents an entity of medical data, and displays labels such as a location, disease name, and symptom. Furthermore, edges indicate the relationship between the nodes, and display, for example, symptoms, location, suspicion, etc. Here, although node names and edge attributes are normally described, their display is omitted to simplify the explanation.

[0054] 7, knowledge graph data 15a, which is an enlarged view of region F of the knowledge graph, displays the following nodes: "gallbladder," "cholecystitis," "loss of potentiation," "necrotizing cholecystitis," "wall," "buoyant thickening," and "tension." The "gallbladder" and "cholecystitis" are connected by a "suspect" edge, the "gallbladder" and "loss of potentiation" are connected by a "symptom" edge, and the "loss of potentiation" is further connected to "necrotizing cholecystitis" by a "suspect" edge. The "gallbladder" and "wall" are connected by a "site" edge, the "wall" is further connected to "buoyant thickening" and a "symptom" edge, and the "gallbladder" and "tension" are connected by a "symptom" edge.

[0055] In this way, in this embodiment, the data in the database 15 used for generating an expanded search is converted into knowledge graph data, and the knowledge graph data is searched using vectors, thereby improving search accuracy and search efficiency.

[0056] <Configuration of local server 20> Next, the configuration of the local server 20 of the medical information presentation system will be described. Fig. 8 is a functional block diagram showing the configuration of the local server 20 shown in Fig. 2. As shown in Fig. 8, the local server 20 has a storage unit 25 and a control unit 26, and is connected to an input unit 21, a display unit 22, and a communication I / F unit 23.

[0057] The input unit 21 is an input device such as a keyboard or a mouse, the display unit 22 is a display device such as an LCD panel, and the communication I / F unit 23 is a communication interface unit for connecting to the cloud management server 10 and the doctor's terminal 30 via the network N.

[0058] The storage unit 25 is a storage device such as a hard disk drive or nonvolatile memory, and stores medical data 25a and anonymized data 25b. The medical data 25a includes medical information such as diagnostic reports, radiology reports, and examination images received from the doctor terminal 30 and modality devices in the medical institution 1. The anonymized data 25b is data from which personal information such as name, address, and telephone number has been deleted from the medical data 25a so that the personal information cannot be identified.

[0059] The control unit 26 is a control unit that performs overall control of the local server 20, and includes a medical data receiving unit 26a, an anonymization processing unit 26b, and anonymized data transmitting unit 26c. In practice, by loading and executing these programs into the CPU, the medical data receiving unit 26a, the anonymization processing unit 26b, and the anonymized data transmitting unit 26c execute the processes corresponding to them, respectively.

[0060] The medical data receiving unit 26a is a processing unit that receives medical data such as medical reports and examination images transmitted from the doctor terminal 30 and modality devices in the medical institution 1.

[0061] The anonymization processing unit 26b is a processing unit that performs anonymization processing on the medical data 25a. The anonymization processing unit 26b performs processes such as deleting direct identifying information such as the patient's name, address, and telephone number from the data, pseudonymization by replacing the patient's identifier with a unique code or ID, and data generalization by converting information such as age and address into a range or category.

[0062] The anonymized data transmission unit 26c performs a process of transmitting the anonymized data 25b to the cloud management server 10. The timing of transmission of the anonymized data 25b by the anonymized data transmission unit 26c is preferably, for example, at night when outpatient consultations at the medical institution 1 have been completed and there is less work to create diagnostic reports and capture examination images. However, the local server 20 may receive the medical data, perform anonymization processing sequentially, and transmit the anonymized medical data to the cloud management server 10 once the anonymization processing is complete.

[0063] <Example of disease name suggestions> Next, an example of disease name candidate presentation according to this embodiment will be described. Fig. 9 is a diagram showing an example of disease name candidate presentation according to this embodiment. In the disease name candidate presentation according to this embodiment, disease name candidates that are highly related to the input data input at the doctor terminal 30 are presented.

[0064] For example, as shown in Figure 9(a), if the input data is "Pain in the right buttock, numbness on the outside of the right lower leg, pain in the buttocks in the same position for the past 2-3 years, pain from about a week ago, dull pain in the leg...", the suggested diagnosis is "Suspected lumbar spinal canal stenosis."

[0065] Furthermore, as shown in Figure 9(b), if the input data is "Fluid accumulation and increased fatty tissue density are observed around the pancreas and around the base of the mesentery. The spread of inflammation beyond the lower pole of the left kidney has improved slightly, but...", the suggested diagnosis is "suspected uterine fibroids."

[0066] <An example of presenting a continuation sentence> Next, an example of the continuous sentence presentation according to this embodiment will be described. Fig. 10 is a diagram showing an example of the continuous sentence presentation according to this embodiment. In the continuous sentence presentation according to this embodiment, candidates of sentences that are likely to be input following the text input on the doctor terminal 30 are presented.

[0067] For example, as shown in Figure 10, if the input text is "CK20 positive and CDX2 positive findings are present," then the following sentences are presented as possible continuation sentences: "These findings suggest a tumor of gastrointestinal origin, particularly colon cancer," "A tumor of intestinal origin is strongly suspected," "A primary malignant tumor of the gastrointestinal tract is considered," and "This suggests a high possibility of a tumor of intestinal epithelium origin."

[0068] <Example of additional text presentation> Next, an example of the presentation of additional text according to this embodiment will be described. Fig. 11 is a diagram showing an example of the presentation of additional text according to this embodiment. In the presentation of additional text according to this embodiment, candidates of sentences that are likely to be added to the text inputted at the doctor terminal 30 are presented.

[0069] For example, as shown in Figure 11(a), if the input data is "Cancer metastasis was found during endoscopic examination," then the following are suggested as possible additional sentences: "Pathological diagnosis of biopsy tissue was positive for ...A," "Metastasis was observed in the lesser curvature of the stomach and ...," "Lymph node enlargement is accompanied by ...histological evaluation will be performed," "Extensive erosive changes are accompanied by ...characteristics," and "Multiple lesions have formed, and ...examination is required."

[0070] It should be noted that instead of presenting candidates for sentences to follow the input data, sentences that provide reasons leading to the conclusion of the input data may be added to the input data.

[0071] For example, as shown in Figure 11(b), the following sentences are presented as candidates for additional text: "An endoscopic examination revealed multiple irregular ulcers in the gastric mucosa, and a biopsy confirmed cancer metastasis."; "An endoscopic examination revealed elevated lesions in the colonic mucosa, and a biopsy confirmed cancer metastasis."; "An endoscopic examination revealed diffuse redness and erosion in the esophageal mucosa, and a biopsy confirmed cancer metastasis."; "An endoscopic examination revealed an irregular mass in the duodenal papilla, and cancer metastasis was confirmed."; or "An endoscopic examination revealed a lesion with an induration in the rectal mucosa, and a biopsy confirmed cancer metastasis." The parts enclosed in " " are the parts to be added.

[0072] <An example of a contradiction in content> Next, an example of content contradiction presentation according to this embodiment will be described. Fig. 12 is a diagram showing an example of content contradiction presentation according to this embodiment. In content contradiction presentation according to this embodiment, a part of content that is contradictory in a medical report, which is content contradiction presentation target data, is presented.

[0073] For example, as shown in Figure 12(a), if the clinical report data is "Due to persistent coughing for over a month, a small nodular shadow is present in the left middle lung field and dilated left renal calyx is suspected," the inconsistent part "left middle lung field" is underlined and presented because of the inconsistency with the test subject.

[0074] Also, as shown in Figure 12(b), if the clinical report data is "A small nodule is seen in the right tongue segment, but this is a non-specific finding and may be a granuloma...", there is a contradiction with the subject of the test, so the contradictory part "right tongue segment" is presented with " ".

[0075] <Example of typo notification> Next, an example of typographical error and omission presentation according to this embodiment will be described. Fig. 13 is a diagram showing an example of typographical error and omission presentation according to the first embodiment. In the typographical error and omission presentation according to this embodiment, typographical errors and omissions in a medical report, which is data for which typographical error and omission presentation is to be performed, are presented. Note that, for the typographical error and omission presentation, input to the LLM16h can be omitted.

[0076] For example, as shown in Figure 13, if the medical report is "Medical advice about lifestyle-related diseases: Obesity and visceral fat obesity have been confirmed... Obesity has been confirmed as a lifestyle weekly disease... Gout and kidney disorders,...", it will be presented as "Medical advice about lifestyle-related diseases: Obesity and 'visceral' fat obesity have been confirmed... Obesity has been confirmed as a lifestyle 'weekly' disease... Gout and kidney disorders,..." with "" placed around typos.

[0077] <Knowledge graph data update procedure> Next, the processing procedure for updating knowledge graph data in the medical information presentation system will be described. Fig. 14 is a sequence diagram showing the processing procedure for updating knowledge graph data in the medical information presentation system. As shown in Fig. 14, the local server 20 receives medical data from the doctor terminal 30 and modality devices in the medical institution 1 (step S101).

[0078] Then, the local server 20 anonymizes the medical data (step S102). After that, the local server 20 transmits the anonymized data to the cloud management server 10 (step S103). The cloud management server 10 receives the anonymized data transmitted from the local server 20 (step S104).

[0079] Then, the cloud management server 10 generates label data from the anonymized data (step S105). After that, the cloud management server 10 vectorizes the anonymized data and the label data (step S106). Then, the cloud management server 10 updates the knowledge graph data (step S107). If there is already constructed knowledge graph data, the cloud management server 10 updates the knowledge graph data using new data, and if there is no already constructed knowledge graph data, the cloud management server 10 generates new knowledge graph data.

[0080] <Procedure for presenting response data> Next, a description will be given of the procedure for presenting answer data by cloud management server 10. Fig. 15 is a sequence diagram showing the procedure for presenting answer data by cloud management server 10. As shown in Fig. 15, cloud management server 10 is in a state of waiting for question sentence data (including medical reports) (step S201; No).

[0081] Then, when question data is received (step S201; Yes), a label is generated from the text included in the question data (step S202), and the generated label is vectorized to generate a question vector (step S203).

[0082] Then, the knowledge graph data 15a is searched using this question vector (step S204) to acquire related data. After that, the question data and related data are input to the LLM 16h to acquire answer data (step S206), and the acquired answer data is sent to the doctor terminal 30 (step S207).

[0083] By performing the above series of processes, it is possible to output answer data corresponding to a question regarding a doctor report using the knowledge graph data 15a and the LLM 16h.

[0084] <Modification> Incidentally, in the above embodiment, a case where a single piece of knowledge graph data 15a is used is shown, but the present invention is not limited to this, and can also be applied to a case where a plurality of pieces of knowledge graph data are used as a multi-RAG.

[0085] 16 is an explanatory diagram for explaining a case where a plurality of knowledge graph data are used. As shown in FIG. 16, a cloud management server 40 has data 41 that stores knowledge graph data 42 and knowledge graph data 43.

[0086] The knowledge graph data 42 is a graph network generated based on multiple vectorized information obtained by vectorizing texts contained in multiple pieces of medical information acquired from local servers of medical institutions. In other words, the knowledge graph data 42 is generated based on highly reliable data, namely, medical information acquired at medical institutions.

[0087] The knowledge graph data 43 is a knowledge graph network generated based on multiple vectorized information obtained by vectorizing text contained in multiple publicly available information related to diseases that have seen a specified increase in the number of infected people within a specified period of time. In other words, the knowledge graph data 43 is generated based on data related to epidemic diseases, although its reliability is lower than that of medical information.

[0088] When the information presentation unit 16g receives question data related to a medical report from the doctor terminal 30, it searches the knowledge graph data 42 and the knowledge graph data 43 to obtain related data related to the question data, inputs the obtained related data and the medical report into the LLM, and transmits the answer data output from the LLM to the doctor terminal 30.

[0089] In medical institutions, highly contagious diseases such as the novel coronavirus disease (COVID-19) can spread, causing outbreaks, epidemics, and pandemics. Before many cases of these diseases appear, they are only published in a few journals and papers, and are not reflected in knowledge graph data 42. For this reason, it is desirable to introduce new knowledge graph data 43. However, if knowledge graph data 42 and knowledge graph data 43 are combined to create a single knowledge graph data, inaccurate information may be mixed into the knowledge graph data, resulting in a deterioration in the quality of the knowledge graph data. For this reason, if knowledge graph data 42 and LLM cannot provide valid answer data, it is desirable to further refer to knowledge graph data 43 to obtain answer data from LLM.

[0090] <Relationship with hardware> Next, the correspondence between the cloud management server 10 of the medical information presentation system according to this embodiment and the main hardware configuration of the computer will be described. Fig. 17 is a diagram showing an example of the hardware configuration.

[0091] Generally, a computer is configured such that a CPU 81, a ROM 82, a RAM 83, and a non-volatile memory 84 are connected via a bus 85. A hard disk drive may be provided instead of the non-volatile memory 84. For the sake of convenience of explanation, only the basic hardware configuration is shown.

[0092] Here, the ROM 82 or non-volatile memory 84 stores programs required to start the operating system (hereinafter simply referred to as "OS"), and the CPU 81 reads and executes the OS program from the ROM 82 or non-volatile memory 84 when the power is turned on.

[0093] On the other hand, various application programs executed on the OS are stored in non-volatile memory 84, and the CPU 81 executes the application programs while using RAM 83 as the main memory, thereby executing processes corresponding to the applications.

[0094] The medical information presentation program of the cloud management server 10 of the medical information presentation system according to this embodiment is also stored in the nonvolatile memory 84 or the like, like other application programs, and the CPU 81 loads and executes the management program. In the case of the cloud management server 10 of the medical information presentation system according to this embodiment, a medical information presentation program including routines corresponding to the anonymized data receiving unit 16a, the label generating unit 16b, the vectorization processing unit 16c, the knowledge graph data updating unit 16d, the knowledge graph data managing unit 16e, the request receiving unit 16f, the information presenting unit 16g, and the LLM 16h shown in FIG. 3 is stored in the nonvolatile memory 84 or the like. The CPU 81 generates medical information presentation processes corresponding to the anonymized data receiving unit 16a, the label generating unit 16b, the vectorization processing unit 16c, the knowledge graph data updating unit 16d, the knowledge graph data managing unit 16e, the request receiving unit 16f, the information presenting unit 16g, and the LLM 16h.

[0095] As described above, in this embodiment, the cloud management server 10 is configured to have a general-purpose LLM 16h, a database 15 that manages knowledge graph data 15a generated based on multiple vectorized information obtained by vectorizing text contained in multiple pieces of medical information, and an information presentation unit 16g that, when receiving question data related to a medical report from the doctor terminal 30, transmits answer data that responds to the question data of the medical report based on the database 15 and the LLM 16h to the doctor terminal 30, thereby making it possible to present appropriate answers to inquiries related to the medical report.

[0096] The components illustrated in the above embodiments are merely functional schematics and are not necessarily physically configured as shown. In other words, the distribution and integration of each device is not limited to the illustrated configuration, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. [Industrial Applicability]

[0097] The medical information presentation system, management device, medical information presentation method, and medical information presentation program according to the present invention are suitable for presenting appropriate answers to inquiries about medical information. [Explanation of symbols]

[0098] 1. Medical institutions 10,40 Cloud Management Server 11 Input section 12 Display section 13 Communication I / F section 14 Storage section 14a Anonymized Data 14b Label Data 14c Question data 14d Response data 15,41 Database 15a, 42, 43 Knowledge graph data 16 Control Unit 16a Anonymized data receiving unit 16b Label generation unit 16c Vectorization processing section 16d Knowledge graph data update section 16e Knowledge Graph Data Management Department 16f Request Reception Section 16g Information presentation section 16h LLM 20 Local Server 21 Input section 22 Display section 23 Communication I / F section 25 Memory section 25a Medical Data 25b Anonymized Data 26 Control Unit 26a Medical data receiver 26b Anonymization processing unit 26c Anonymized data transmission unit 30 Doctor's terminal 51 CPU 52 ROM 53 RAM 54 Non-volatile memory

Claims

1. A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, The management device A large-scale language model that learns from multiple texts to understand and generate sentences, a database for managing a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of medical information; an information presentation means for, when receiving a request for a medical report including a medical report from a predetermined terminal device, acquiring related information related to the medical report by using the database as a search extension generation means, and presenting to the terminal device response information in response to the request for the medical report, which is generated by the large-scale language model using the request for the medical report and the related information; Equipped with The information presentation means When a request for presentation of a continuation sentence following a sentence written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of the continuation sentence of the medical report are input to the large-scale language model, and the medical report including the continuation sentence output from the large-scale language model is presented to the terminal device as the response information. A medical information presentation system comprising:

2. A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, The management device A large-scale language model that learns from multiple texts to understand and generate sentences, a database for managing a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of medical information; an information presentation means for, when receiving a request for a medical report including a medical report from a predetermined terminal device, acquiring related information related to the medical report by using the database as a search extension generation means, and presenting to the terminal device response information in response to the request for the medical report, which is generated by the large-scale language model using the request for the medical report and the related information; Equipped with The information presentation means When a request for presentation of an additional sentence to be added to the text written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of the additional sentence in the medical report are input to the large-scale language model, and the medical report including the additional sentence output from the large-scale language model is presented to the terminal device as the response information. A medical information presentation system comprising:

3. A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, The management device A large-scale language model that learns from multiple texts to understand and generate sentences, a database for managing a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of medical information; an information presentation means for, when receiving a request for a medical report including a medical report from a predetermined terminal device, acquiring related information related to the medical report by using the database as a search extension generation means, and presenting to the terminal device response information in response to the request for the medical report, which is generated by the large-scale language model using the request for the medical report and the related information; Equipped with The information presentation means When a request for presentation of content contradictions in sentences written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of content contradictions in the medical report are input to the large-scale language model, and response information including content contradictions in the medical report output from the large-scale language model is presented to the terminal device. A medical information presentation system comprising:

4. A medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, The management device A large-scale language model that learns from multiple texts to understand and generate sentences, a database for managing a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of medical information; an information presentation means for, when receiving a request for a medical report including a medical report from a predetermined terminal device, acquiring related information related to the medical report by using the database as a search extension generation means, and presenting to the terminal device response information in response to the request for the medical report, which is generated by the large-scale language model using the request for the medical report and the related information; Equipped with The information presentation means When a request for presentation of clerical errors in a sentence written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of clerical errors in the medical report are input to the large-scale language model, and response information including clerical errors in the medical report output from the large-scale language model is presented to the terminal device. A medical information presentation system comprising:

5. The management device a label information extraction means for extracting a plurality of pieces of label information from text included in the plurality of pieces of medical information; vectorized information generating means for vectorizing each of the plurality of pieces of label information to generate a plurality of pieces of vectorized information; a graph network generation means for generating the knowledge graph network based on the plurality of vectorized information; 5. The medical information presentation system according to claim 1, further comprising:

6. The database comprises: a first knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information acquired from a server device of a medical institution; a second knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing text included in a plurality of publicly available information related to a disease whose number of infected people has increased by a predetermined amount or more within a predetermined period of time; The information presentation means When a request related to the medical report is received from a predetermined terminal device, the medical report is vectorized, and related information corresponding to the request related to the medical report is acquired by using the first knowledge graph network and the second knowledge graph network as search extension generation, and the response information generated by the large-scale language model using the request related to the medical report and the related information is presented to the terminal device.

5. The medical information presentation system according to claim 1, wherein the medical information presentation system is a system for displaying medical information.

7. A management device for managing medical information including medical images captured by a predetermined modality device and medical reports of the medical images, A large-scale language model that learns from multiple texts to understand and generate sentences, a database for managing a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts contained in a plurality of medical information; an information presentation means for, when receiving a request for a medical report from a predetermined terminal device, acquiring related information corresponding to the request for the medical report by using the database and the large-scale language model as search extension generation, and presenting to the terminal device response information responding to the request for the medical report, which is generated by the large-scale language model using the request for the medical report and the related information; Equipped with The information presentation means When a request for presentation of a continuation sentence following a sentence written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of the continuation sentence of the medical report are input to the large-scale language model, and the medical report including the continuation sentence output from the large-scale language model is presented to the terminal device as the response information. A management device characterized by:

8. A medical information presentation method in a medical information presentation system having a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, comprising: When the management device receives a predetermined request related to a medical report from a predetermined terminal device, the management device acquires related information related to the request related to the medical report by using a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information as a search extension generation; a presentation step in which the management device presents, to the terminal device, response information in response to the request for the medical report, the response information being generated by a large-scale language model that understands and generates sentences by learning a plurality of texts, using the related information and the request for the medical report; Including, The presenting step includes: When a request for presentation of a continuation sentence following a sentence written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of the continuation sentence of the medical report are input to the large-scale language model, and the medical report including the continuation sentence output from the large-scale language model is presented to the terminal device as the response information. A medical information presentation method comprising:

9. A medical information presentation program executed in a management device that manages medical information including medical images captured by a predetermined modality device and medical reports of the medical images, comprising: an acquisition step of acquiring, when a request for a medical report is received from a predetermined terminal device, related information related to the request for the medical report by using a knowledge graph network generated based on a plurality of vectorized information obtained by vectorizing texts included in a plurality of medical information as a search extension generation; a presentation step of presenting, on the terminal device, response information in response to the request related to the medical report, which is generated by a large-scale language model that learns a plurality of texts and understands and generates sentences using the related information and the request related to the medical report; Including, The presentation procedure includes: When a request for presentation of a continuation sentence following a sentence written in the medical report is received as a request related to the medical report, the medical report is vectorized, the knowledge graph network is searched to acquire related information related to the medical report, the acquired related information and the request for presentation of the continuation sentence of the medical report are input to the large-scale language model, and the medical report including the continuation sentence output from the large-scale language model is presented to the terminal device as the response information. A medical information presentation program that causes a computer to execute processing.

Images