Information processing device, information processing method, and program

Abstract

To provide an information processor, a method for processing information, and a program that can reduce a work load of updating a database including a plurality of templates related to a document.SOLUTION: An information processor includes: at least one processor; and at least one storage for storing a database including a plurality of templates on a document. The templates include a fill-in-the-blank sentence and a candidate of a word to fill the blank. The at least one processor receives a sentence, analyzes the received sentence, determines whether updating the database is necessary on the basis of the result of analysis of the sentence, and updates the database on the basis of the result of the determination.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program, and particularly relates to a technique for assisting in creating a document typified by an image diagnosis report.

Background Art

[0002] A radiologist reads medical images and creates a reading report. For the purpose of reducing the workload of the reading report creation task, a system that stores a finding template in advance in the system's database and presents the finding template to the radiologist has become widespread.

[0003] Patent Document 1 and Patent Document 2 show examples of creating a reading report using a template. Further, Patent Document 3 discloses a mechanism for managing a finding template together with identification information and updating the template.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, since the text of the finding template prepared in advance may differ from the content that the doctor wants to describe, it may be necessary to modify the finding template in the database or add a new template, and the burden of this work is large.

[0006] This disclosure is made in view of these circumstances and aims to provide an information processing device, an information processing method, and a program that can reduce the workload of updating a database containing multiple templates related to documents. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present disclosure is an information processing device comprising one or more processors and one or more storage devices that store a database containing a plurality of templates relating to documents, wherein the plurality of templates include fill-in-the-blank sentences and candidate words or phrases that can be inserted into the fill-in-the-blank portions, and the one or more processors receive a document, analyze the received document, determine whether or not to update the database based on the results of the document analysis, and update the database based on the determination result.

[0008] According to this embodiment, since the updating of the template database can be automated, the workload of updating the database by the user can be reduced.

[0009] In other embodiments of the present disclosure, one or more processors may be configured to determine whether or not a database needs to be updated by determining, based on the results of parsing a document, whether or not the document can be represented using a template in the database.

[0010] In information processing devices relating to other aspects of this disclosure, the document may be a medical document.

[0011] Medical documents may also relate to the results of imaging diagnostics. For example, imaging diagnostic reports, such as image interpretation reports, are included in medical documents. Medical documents may also relate to the patient's medical records.

[0012] In other embodiments of the present disclosure, one or more processors may be configured to update the database by performing at least one of the following processes: adding new fill-in-the-blank sentences; modifying existing fill-in-the-blank sentences; adding new word or phrase candidates associated with the fill-in-the-blank portion; and modifying existing word or phrase candidates associated with the fill-in-the-blank portion.

[0013] Possible forms of updating the database include adding new fill-in-the-blank sentences that are not currently in the database, modifying existing fill-in-the-blank sentences that are already in the database, adding new word candidates that are not currently in the database, and changing existing word candidates that are already in the database to other words such as synonyms.

[0014] In other embodiments of the present disclosure, one or more processors may be configured to generate fill-in-the-blank sentences corresponding to the expression of a received sentence, and to add the generated fill-in-the-blank sentences to a database and update the database.

[0015] In other embodiments of the present disclosure, the information processing device may be configured such that the number of blanks in the fill-in-the-blank text generated in response to the expression of the received text is increased compared to the existing fill-in-the-blank text stored in the database.

[0016] The fill-in-the-blank section may include a section in which a word or phrase representing at least one of the following is included: the characteristics of the lesion and the name of the disease.

[0017] In other embodiments of the present disclosure, one or more processors may be configured to determine whether a document can be represented using a template in a database, based on the degree of matching of at least one of the characteristics of a lesion and a disease name.

[0018] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to generate a fill-in-the-blank sentence corresponding to the expression of the received sentence, and determine whether the sentence can be expressed using a template in a database based on at least one of the number of fill-in-the-blank parts, the types of fill-in-the-blank parts, and the order.

[0019] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to structure the received sentence by natural language processing, and determine whether the sentence can be expressed using a template in a database based on the result of the structuring.

[0020] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to update the database by identifying a template similar to the received sentence among a plurality of templates stored in the database, and overwriting and storing the template after changing a part of the identified template.

[0021] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to overwrite and store the template after changing the style of the identified template.

[0022] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to overwrite and store the template after changing the word order of the fill-in-the-blank parts of the identified template.

[0023] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to determine a word or phrase associated with a fill-in-the-blank part of a fill-in-the-blank sentence corresponding to the expression of the received sentence based on the received sentence, and add the determined word or phrase to the database as a candidate for a word or phrase that can be inserted into the fill-in-the-blank part.

[0024] In an information processing apparatus according to another aspect of the present disclosure, the blanks in the filling text may be configured to include a first type filled by the processing result of image processing and a second type not filled by the processing result of image processing.

[0025] The blanks of the first type include portions filled by inserting a phrase representing at least one of a location and a quantity, and the blanks of the second type may include portions filled by inserting a phrase representing a disease name.

[0026] In an information processing apparatus according to another aspect of the present disclosure, one or more processors generate one or more text candidates in which the blanks of the first type are filled based on the processing result of image processing and a template stored in a database, present the one or more text candidates as options, and receive an instruction to select one of the text candidates.

[0027] In an information processing apparatus according to another aspect of the present disclosure, one or more processors receive an input to rewrite a text candidate, determine whether to update the database based on the analysis result of the text stored after rewriting, and update the database based on the determination result.

[0028] In an information processing apparatus according to another aspect of the present disclosure, one or more processors present phrase candidates that can be inserted into the blanks of the second type based on a template stored in a database and receive an instruction to select one of the phrase candidates.

[0029] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to receive a text input using an input device.

[0030] An information processing method relating to another aspect of this disclosure is an information processing method in which one or more processors update a database including a plurality of templates relating to a document, wherein the plurality of templates include fill-in-the-blank sentences and candidate words or phrases that can be inserted into the fill-in-the-blanks, and the one or more processors include receiving a document, analyzing the received document, determining whether or not to update the database based on the results of the document analysis, and updating the database based on the determination result.

[0031] A program relating to another aspect of this disclosure is a program that enables a computer to perform a function to update a database containing multiple templates relating to documents, wherein the multiple templates include fill-in-the-blank sentences and candidate words or phrases that can be inserted into the blanks, and enables the computer to perform a function to receive documents, a function to analyze the received documents, a function to determine whether or not to update the database based on the results of the document analysis, and a function to update the database based on the determination results. [Effects of the Invention]

[0032] According to this disclosure, the workload of updating a database containing multiple templates for documents can be reduced. [Brief explanation of the drawing]

[0033] [Figure 1] Figure 1 is a block diagram showing an example configuration of a medical information system including an information processing device according to an embodiment. [Figure 2] Figure 2 is a block diagram schematically showing an example of the hardware configuration of an information processing device. [Figure 3] Figure 3 is a functional block diagram of the information processing device. [Figure 4] Figure 4 is an explanatory diagram showing an example of a template for pulmonary nodules. [Figure 5] Figure 5 is an explanatory diagram showing an example of a template for liver tumors. [Figure 6] Figure 6 is an explanatory diagram showing an example template for liver cysts. [Figure 7] Figure 7 is an explanatory diagram showing an example of a template related to lymphadenopathy. [Figure 8] Figure 8 is an explanatory diagram showing an example of a template for comparing pulmonary nodules over time. [Figure 9] Figure 9 is a flowchart showing an example of processing performed by an information processing device. [Figure 10] Figure 10 shows an example of a medical image being observed. [Figure 11] Figure 11 is an explanatory diagram showing an example of a candidate for the findings statement displayed on the screen of the display device. [Figure 12] Figure 12 is an explanatory diagram showing an example of a report written using free-form descriptions. [Figure 13] Figure 13 is an explanatory diagram illustrating an example of the process when word candidates are added, and shows an example of a finding sentence regarding lymph node enlargement. [Figure 14] Figure 14 is an explanatory diagram illustrating an example of the process when word candidates are added, and shows an example of updating the template related to lymph node enlargement. [Figure 15] Figure 15 is an explanatory diagram illustrating an example of the process when word candidates are changed, and shows an example of a finding sentence related to liver tumors. [Figure 16] Figure 16 is an explanatory diagram illustrating an example of the process when word candidates are changed, and shows an example of updating a template related to liver tumors. [Figure 17] Figure 17 is an explanatory diagram illustrating an example of how to handle cases where the word order of a fill-in-the-blank sentence is changed, and shows an example of a finding sentence regarding pulmonary nodules. [Figure 18] Figure 18 is an explanatory diagram illustrating an example of how to handle cases where the word order of fill-in-the-blank sentences is changed, and shows an example of updating a template related to pulmonary nodules. [Figure 19] Figure 19 is an explanatory diagram illustrating an example of how to handle cases where the wording of fill-in-the-blank sentences is changed, and shows an example of a finding sentence regarding liver cysts. [Figure 20] Figure 20 shows an example of an updated template for liver cysts. [Figure 21]Figure 21 is an explanatory diagram illustrating an example of how to handle cases where blanks are added to a fill-in-the-blank text, and shows an example of a findings text regarding a time-series comparison of pulmonary nodules. [Figure 22] Figure 22 is an explanatory diagram illustrating an example of the process when blanks are added to a fill-in-the-blank text, and shows an example of updating a template for a time-series comparison of pulmonary nodules. [Figure 23] Figure 23 is an explanatory diagram showing an example of a medical report written by a physician. [Figure 24] Figure 24 is an explanatory diagram illustrating an example of a new template generated based on the findings text in Figure 23. [Figure 25] Figure 25 is an explanatory diagram showing an example of a template-based presentation of candidate findings. [Figure 26] Figure 26 is an explanatory diagram showing example 1 of a physician's rewriting of a medical report. [Figure 27] Figure 27 is an explanatory diagram showing example 2 of a physician's rewriting of a medical report. [Figure 28] Figure 28 is an explanatory diagram showing example 3 of a physician's rewriting of a medical report. [Figure 29] Figure 29 is an explanatory diagram showing example 4 of how a physician can rewrite a medical report. [Modes for carrying out the invention]

[0034] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings.

[0035] Examples of medical information system configurations Figure 1 is a block diagram showing an example configuration of a medical information system 200 including an information processing device 10 according to an embodiment of this disclosure. The medical information system 200 is implemented as a computer network built in a medical institution such as a hospital. The medical information system 200 includes an electronic medical record system 202, a CT scanner 204, an MRI scanner 206, an image storage server 210, an information processing device 10, and a viewer terminal 230, and these elements are connected via a communication line 240. The communication line 240 may be an in-house communication line within the medical institution. A portion of the communication line 240 may also include a wide-area communication line. Some elements of the medical information system 200 may be configured using cloud computing.

[0036] Figure 1 illustrates CT scanner 204 and MRI scanner 206 as examples of modalities. However, medical imaging equipment is not limited to CT scanner 204 and MRI scanner 206. Various other diagnostic devices are possible, including ultrasound diagnostic equipment (not shown), PET (Positron Emission Tomography) equipment, mammography equipment, X-ray diagnostic equipment, X-ray fluoroscopy diagnostic equipment, and endoscopes. The types and number of modalities connected to the communication line 240 can vary from one medical institution to another.

[0037] The image storage server 210 may be, for example, a DICOM server operating according to the DICOM (Digital Imaging and Communications in Medicine) specification. The image storage server 210 is a computer that stores and manages various data, including images acquired using various modalities such as the CT scanner 204 and the MRI scanner 206, and is equipped with a large-capacity external storage device and a database management program. The image storage server 210 communicates with other devices via the communication line 240 and sends and receives various data, including image data. The image storage server 210 receives various data, including images generated by modalities such as the CT scanner 204, via the communication line 240, and stores and manages them on a recording medium such as a large-capacity external storage device. The storage format of the image data and communication between each device via the communication line 240 are based on the DICOM protocol.

[0038] The information processing device 10 can acquire data from an image storage server 210 or the like via a communication line 240. The information processing device 10 can be implemented using computer hardware and software. The form of the information processing device 10 is not particularly limited; it may be a server computer, a workstation, a personal computer, or a tablet terminal. The information processing device 10 may also include an input device 12 and a display device 14.

[0039] The input device 12 may be, for example, a keyboard, mouse, multi-touch panel, or other pointing device, or an audio input device, or an appropriate combination thereof. The display device 14 may be, for example, a liquid crystal display, an organic electro-luminescence (OEL) display, or a projector, or an appropriate combination thereof. The input device 12 and the display device 14 may be configured as an integrated unit, such as a touch panel. The input device 12 and the display device 14 may be included in the information processing device 10, or the information processing device 10, the input device 12, and the display device 14 may be configured as an integrated unit.

[0040] The information processing device 10 performs image analysis of medical images captured by modalities, assists in the creation of image interpretation reports, and performs various other processing. An image interpretation report is a report concerning the results of image diagnosis. The information processing device 10 is configured to perform various computer-aided diagnosis (Computer-Aided Diagnosis, Computer-Aided Detection: CAD) and other analysis processing, such as processing to recognize lesion areas from images, processing to identify classifications such as disease names, or segmentation processing to recognize areas such as organs. The information processing device 10 also performs processing to assist in the creation of image interpretation reports. The information processing device 10 can send the processing results of image processing and the processing results of report creation to the image storage server 210 and the viewer terminal 230. Some or all of the processing functions of the information processing device 10 may be incorporated into the image storage server 210 or into the viewer terminal 230.

[0041] Various data stored in the image database 212 of the image storage server 210, as well as various information including processing results generated by the information processing device 10, can be displayed on the display device 234 of the viewer terminal 230. In the diagram, "DB" stands for database.

[0042] The viewer terminal 230 may be an image viewing terminal called a PACS (Picture Archiving and Communication Systems) viewer or a DICOM viewer. Although Figure 1 illustrates one viewer terminal 230, multiple viewer terminals 230 may be connected to the communication line 240. The form of the viewer terminal 230 is not particularly limited; it may be a personal computer, a workstation, or a tablet terminal. The viewer terminal 230 includes an input device 232 and a display device 234. The input device 232 and the display device 234 may have the same configuration as the input device 12 and the display device 14 of the information processing device 10.

[0043] [Example Hardware Configuration of Information Processing Device 10] Figure 2 is a block diagram schematically showing an example of the hardware configuration of the information processing device 10. The information processing device 10 can be implemented by a computer system consisting of one or more computers. Here, we describe an example in which various functions of the information processing device 10 are realized by a single computer executing a program.

[0044] The information processing device 10 includes a processor 102, a computer-readable medium 104 which is a non-temporary tangible object, a communication interface 106, an input / output interface 108, and a bus 110.

[0045] The processor 102 includes a CPU (Central Processing Unit). The processor 102 may also include a GPU (Graphics Processing Unit). The processor 102 is connected to a computer-readable medium 104, a communication interface 106, and an input / output interface 108 via a bus 110. The processor 102 reads various programs and data stored in the computer-readable medium 104 and executes various processes. The term "program" includes the concept of a program module and includes instructions equivalent to a program.

[0046] The computer-readable medium 104 is a storage device that includes, for example, a main memory 122 and an auxiliary storage device 124. The storage device 124 is configured using, for example, a hard disk drive (HDD), a solid state drive (SSD), an optical disk, a magneto-optical disk, or a semiconductor memory, or an appropriate combination thereof. Various programs and data are stored in the storage device 124. The memory 122 and the storage device 124 are examples of "storage devices" in this disclosure.

[0047] Memory 122 is used as a workspace for the processor 102 and as a storage unit for temporarily storing programs read from storage 124 and various data. When a program stored in storage 124 is loaded into memory 122, and the processor 102 executes the program's instructions, the processor 102 functions as a means for performing various processes defined by the program. Memory 122 stores programs such as an image processing program 140, a report creation support program 150, and a display control program 160, which are executed by the processor 102, as well as various data. Storage 124 includes a template database 130 that stores multiple templates for the findings to be included in the image interpretation report. The template database 130 is an example of a "database" in this disclosure.

[0048] The image processing program 140 includes a CAD module that processes an input image and, for example, detects areas such as lesions within the image and performs characteristic analysis of the lesions. The image processing program 140 may also include, for example, a program that detects lesions such as nodules from CT images of the lungs, and / or a program that performs characteristic analysis of liver tumors from dynamic contrast-enhanced CT images of the liver. The image processing program 140 may be configured using a trained model trained by machine learning to output processing results such as the desired characteristic analysis from the input image.

[0049] The image processing program 140 may include other CAD modules, such as organ recognition, programming, and lesion detection programs, in addition to characterization analysis. It may also include a program that performs segmentation of lesion regions and calculates the size of the lesion region based on the results.

[0050] The report creation support program 150 includes a report candidate generation program 152, a text editing program 154, a text analysis program 156, and a template update program 158. The report candidate generation program 152 is a program that causes the processor 102 to perform a process of automatically generating one or more report candidate sentences based on the image processing results from the image processing program 140 and templates stored in the template database 130.

[0051] The document editing program 154 is a program that causes the processor 102 to execute the process of creating and editing the medical report based on operations such as text input by the physician (user). The document analysis program 156 is a program that causes the processor 102 to execute the process of analyzing and structuring the document created by the physician using natural language processing (NLP) technology.

[0052] The template update program 158 is a program that causes the processor 102 to execute a process to update the template database 130 based on the processing results of the text analysis program 156.

[0053] The display control program 160 is a program that generates display signals necessary for display output to the display device 14 and causes the processor 102 to execute the process of controlling the display of the display device 14.

[0054] The communication interface 106 performs communication processing with external devices via wired or wireless connections and exchanges information with external devices. The information processing device 10 is connected to the communication line 240 via the communication interface 106 (see Figure 1), and can exchange data with devices such as the image storage server 210 and the viewer terminal 230. The communication interface 106 can also act as a data acquisition unit that accepts input data such as images.

[0055] The input device 12 and the display device 14 are connected to the bus 110 via the input / output interface 108.

[0056] Operation of the information processing device 10 Figure 3 is a functional block diagram of the information processing device 10. The information processing device 10 includes a data acquisition unit 312, a communication unit 316, an image processing unit 340, a report creation support unit 350, a template database 130, a report storage unit 134, and a display control unit 360.

[0057] The data acquisition unit 312 acquires the medical image to be processed. The medical image may be a two-dimensional image or a three-dimensional image. In modalities such as the CT scanner 204 or the MRI scanner 206, three-dimensional data showing the three-dimensional form of the object is obtained by continuously acquiring two-dimensional slice images. In this specification, the term "three-dimensional image" includes the concept of a collection of continuously acquired two-dimensional slice images (two-dimensional image sequence).

[0058] The data acquisition unit 312 may acquire a combination of multiple medical images. For example, the data acquisition unit 312 may acquire a combination of images from multiple time phases taken by a dynamic contrast-enhanced examination. The data acquisition unit 312 may be configured to include a communication interface for receiving images from an external device such as an imaging device or an image storage server 210, or it may be configured to include a media interface for reading images from removable media such as a memory card. The data acquisition unit 312 may also be configured to include an image acquisition program for automatically acquiring images from an external device. Images acquired via the data acquisition unit 312 are sent to the image processing unit 340.

[0059] The communication unit 316 includes a communication interface for communicating with external devices. The communication unit 316 may also include a buffer memory.

[0060] The image processing unit 340 processes the input image and performs tasks such as organ detection, lesion detection, lesion region segmentation, and size measurement. For example, the image processing unit 340 performs lesion region extraction processing, which automatically extracts lesion regions from medical images acquired via the data acquisition unit 312 using image recognition. The image processing unit 340 extracts lesion regions by performing image segmentation on the input medical image using a trained model that has been trained on the task of image segmentation using machine learning, such as deep learning.

[0061] For example, a convolutional neural network (CNN) can be used as a learning model for image segmentation. The image processing unit 340 may be configured to perform segmentation that classifies the lesion region from other regions, or it may be configured to classify each region of the lesion region from multiple surrounding tissues. Furthermore, the image processing unit 340 may be configured to include a classification model that analyzes the characteristics of the lesion.

[0062] The report creation support unit 350 includes a candidate observation generation unit 352, a text editing unit 354, a text analysis unit 356, and a template update unit 358.

[0063] The findings statement candidate generation unit 352, based on the image processing results from the image processing unit 340, refers to a template suitable for the findings statement from the template database and generates one or more, preferably multiple, findings statement candidates. The findings statement candidates generated by the findings statement candidate generation unit 352 are displayed on the display device 14 or the like and presented to the user (physician).

[0064] The user can select a desired sentence from the presented list of findings. The user can then use the selected sentence as the text to be included in the findings section of the diagnostic imaging report.

[0065] When a user selects a candidate observation using input device 12 or input device 232, the selected candidate observation is copied to the observation field. The user can rewrite the content of the candidate observation copied to the observation field. Alternatively, the user can choose not to adopt a candidate observation and enter free text in the observation field. The user can rewrite the candidate observation or enter arbitrary text using input device 12. The text editing unit 354 performs processing such as rewriting the candidate observation and creating new text according to instructions from input device 12 or input device 232.

[0066] After text is entered in the comments field, the comments are saved to the report storage unit 134 when a predetermined operation is performed, such as clicking the confirm button or save button (not shown). The report storage unit 134 may be composed of a portion of the storage area of ​​the storage unit 124.

[0067] The text analysis unit 356 analyzes and structures the saved observation text using natural language processing. The text analysis unit 356 may be configured to perform structuring using a trained model that has been trained to structure text using machine learning. For example, a Transformer model can be used as the trained model for natural language processing.

[0068] The template update unit 358 determines whether or not to update the template database 130 based on the structuring results from the text analysis unit 356, and updates the template database 130 according to the determination result. The template update unit 358 determines whether or not the saved observation text can be expressed using an existing template in the template database 130. If it can be expressed using an existing template, it is determined that no update is necessary; if it cannot be expressed, it is determined that an update is necessary.

[0069] The display control unit 360 performs a process to generate display data necessary for displaying the processing results and controls the display on the display device 14.

[0070] 《Template Description》 In this embodiment, the "template" for the observation statement includes a fill-in-the-blank text and candidate words or phrases that can be inserted into the blanks (hereinafter referred to as "word candidates"). The "fill-in-the-blanks" referred to here are of two types: those that are automatically entered based on image processing and those that are not automatically entered by image processing. For each fill-in-the-blank section (part) that is not automatically entered by image processing, at least one candidate word or phrase is associated with it.

[0071] Among the blank-filling sections of a fill-in-the-blank text, the first type of blank-filling section, which is filled in by the results of image processing, includes, for example, location and size. Here, location includes the concept of anatomical position. Size may be quantitative information including measured values ​​such as the long or short diameter of a lesion, or it may indicate a size classification that is divided into levels according to the size of the area.

[0072] Among the blank-filling sections of a fill-in-the-blank text, a second type of blank-filling section that is not filled in by the image processing results includes, for example, disease names. For this second type of blank-filling section, word candidates may be stored within the system and presented to the doctor as options when inputting, or the system may present the blank-filling section with the word candidates already inserted.

[0073] Figures 4 to 8 show examples of templates for findings statements. Figure 4 is an example of a template for a pulmonary nodule. Figure 4 shows three types of fill-in-the-blank sentences and examples of word candidates that can actually be entered in the "Disease Name" placeholder. The symbols such as [SN001] in the figures are identification symbols for each fill-in-the-blank sentence. Also, the parenthetical notations such as "(Location)", "(Longest diameter ○○ cm)", and "(Disease Name)" in the fill-in-the-blank sentences represent placeholders for the blanks.

[0074] As templates for pulmonary nodule findings, for example, three types of fill-in-the-blank sentences can be stored in the template database 130: "A nodule was found at (location).", "A nodule with a (longest diameter of XX cm) was found at (location).", and "A nodule with a (longest diameter of XX cm) was found at (location). (Disease name) is suspected.", along with four types of words to replace the placeholder for "(Disease name)": "Primary lung cancer," "Metastatic lung cancer," "Benign," and "Malignant." In this case, the blanks for "(location)" and "(longest diameter of XX cm)" in each fill-in-the-blank sentence are automatically entered based on image processing, and for the blank for "(Disease name)," four types of words—"Primary lung cancer," "Metastatic lung cancer," "Benign," and "Malignant"—can be presented as options to replace the placeholder for "(Disease name)." These four types of words are examples of "word candidates" in this disclosure.

[0075] Figure 5 shows an example of a template for liver tumors. As a template for liver tumor findings, for example, three types of fill-in-the-blank sentences can be stored in the template database 130: "A low-attenuation area is observed at (location).", "A low-attenuation area with a longest diameter of (○○ cm) is observed at (location).", and "A low-attenuation area with a longest diameter of (○○ cm) is observed at (location). (Disease name) is suspected.", along with three types of words to replace the placeholder "(Disease name)": "hepatocellular carcinoma," "metastatic liver cancer," and "hemangioma."

[0076] Figure 6 shows an example of a template for liver cysts. Two types of fill-in-the-blank sentences can be stored in the template database 130 as templates for liver cyst findings, for example, "A cyst was found at (location)" and "A microcyst was found at (location)."

[0077] Figure 7 shows an example of a template for lymph node enlargement. As a template for the findings of lymph node enlargement, four types of fill-in-the-blank sentences may be stored in the template database 130, such as "Lymph node enlargement was observed at (location)," "Lymph node enlargement of (short diameter XX cm) was observed at (location)," and "Lymph node enlargement of (long diameter XX cm) was observed at (location). (Disease name) is suspected.", along with one word candidate to replace the placeholder "(Disease name)," such as "malignant lymphoma."

[0078] Figure 8 shows an example of a template for a time-series comparison of pulmonary nodules. As a template for a report on a time-series comparison of pulmonary nodules, for example, one type of fill-in-the-blank sentence, "The nodule at (location) has (changed)," and three types of phrases as word candidates to replace the placeholder "(changed)"—"No significant change," "It is increasing," and "It is decreasing"—can be stored in the template database 130.

[0079] Examples of information processing methods Figure 9 is a flowchart showing an example of processing performed by the information processing device 10. Here, an example using the input device 12 and display device 14 of the information processing device 10 is described, but the input device 232 and display device 234 of the viewer terminal 230 may be used instead of the input device 12 and display device 14.

[0080] In step S11, the processor 102 receives a medical image input and displays the input image on the display device 14. For example, when a physician specifies an image to be read using the input device 12, the specified image is sent from the image storage server 210 to the information processing device 10, and the information processing device 10 acquires the image. The acquired image is displayed on the display device 14.

[0081] In step S12, the processor 102 accepts the designation of a region of interest, such as a lesion. The physician can observe the image displayed on the display device 14 and use the input device 12 to specify the region of interest. Figure 10 shows an example of an image IMG to be observed. Figure 10 shows an example of a CT image of the lung. For example, the physician can specify a specific location within the image IMG by clicking on the location of the region of interest on the image IMG using the mouse 12A as the input device 12.

[0082] In step S13 of Figure 9, the processor 102 performs image processing on the input image using the image processing program 140. Through image processing, for example, the location and size of the lesion are identified. The processor 102 can obtain information indicating the size of the lesion based on the segmentation results of the lesion region.

[0083] In step S14, the processor 102 uses the template database 130 to display multiple candidate findings.

[0084] Figure 11 shows examples of candidate findings statements presented on the screen of the display device 14. The candidate findings statements shown in Figure 11 are examples of "sentence candidates" in this disclosure. Here, we show an example where the information processing device 10 performs image processing and obtains information that the location of the lesion is "right lung S3" and the size is "3 cm in major diameter". In this case, the physician is presented with several candidate findings statements (here, six candidates from [1] to [6]). [1] A nodule was found in the right lung S3. [2] A nodule with a major diameter of 3 cm was found in the right lung S3. [3] A nodule with a major diameter of 3 cm was found in the right lung S3. Primary lung cancer is suspected. [4] A nodule with a major diameter of 3 cm was found in the right lung S3. Metastatic lung cancer is suspected. [5] A nodule with a major diameter of 3 cm was found in the right lung S3. Benign is suspected. [6] A nodule with a major diameter of 3 cm was found in the right lung S3. Malignant is suspected.

[0085] The physician can select one of these multiple candidate findings and input an instruction to "adopt (decide)" it, thereby transcribing the text of the adopted candidate finding into the radiology report. In other words, the processor 102 accepts an instruction to select one of the multiple candidate findings presented.

[0086] For example, if the doctor wants to write a finding statement [3], the doctor selects the finding statement candidate [3] using the input device 12, and the text [3] is then copied into the findings section of the radiology report.

[0087] Furthermore, the processor 102 not only accepts instructions to select one of several presented candidate findings, but also accepts input of freely written findings by the physician.

[0088] In step S15 of Figure 9, the processor 102 determines whether or not one of the multiple candidate findings has been selected. If the physician has selected one of the candidate findings and the result of the determination in step S15 is Yes, the processor 102 proceeds to step S16 and performs the process of transcribing the text of the candidate finding selected by the physician into the radiology report.

[0089] On the other hand, if none of the multiple candidate findings displayed in step S14 contain the findings the doctor wants to write, and the result of step S15 is a No, then processor 102 proceeds to step S17.

[0090] In step S17, the processor 102 accepts input of a freely written report from the physician. If the physician does not receive a candidate for the report they wish to write, they can use the input device 12 to create a report using free writing. Figure 12 shows an example of a report created using free writing. For example, as shown in the upper part of Figure 12, the physician can create the report themselves in the report entry field, which reads, "A 3 cm long, poorly defined nodule is found in the right lung S3. Primary lung cancer is suspected."

[0091] Next, in step S18 of Figure 9, the processor 102 structures the findings written by the physician using natural language processing technology. This structuring process is performed using the text analysis program 156. For example, the result of structuring the sentence "A poorly defined nodule with a longest diameter of 3 cm was found in the right lung S3. Primary lung cancer is suspected." using natural language processing is as shown in the middle of Figure 12. The classification items used in the structuring include, for example, organ, location, lesion, factuality, disease name, factuality, size, and characteristics.

[0092] Subsequently, in step S19, the processor 102 determines, based on the structuring results, whether the text can be expressed using an existing template. If the determination result in step S19 is No, the processor 102 proceeds to step S20 and updates the findings text template database 130. For example, from the structuring results shown in Figure 12, the fill-in-the-blank sentence that serves as a template necessary to express the findings text that the doctor wants to write is the fill-in-the-blank sentence, "A nodule of (longest diameter ○○ cm) with (characteristics) is found at (location). (Disease name) is suspected." The placeholder for "characteristics" is automatically entered with the words "clearly defined border" or "unclear border" based on image processing. The processor 102 generates such new template addition candidates. The processor 102 then checks whether such additional candidate templates are already saved in the template database 130, and if not, adds them to the template database 130 as new templates.

[0093] Regarding the determination of whether or not the template database 130 needs to be updated, for example, the processor 102 determines whether the findings can be expressed using an existing template in the template database 130, based on the degree of agreement of at least one of the characteristics of the lesion and the disease name.

[0094] Furthermore, the processor 102 may determine whether the findings can be expressed using an existing template based on at least one of the following: the number of fill-in-the-blank sections in the additional candidate template, the type of fill-in-the-blank sections, and their order.

[0095] In this example, the template capable of representing the observation statement shown in the upper part of Figure 12 is not stored in the template database 130. Therefore, the processor 102 performs the process of adding this new template to the template database 130. For example, the processor 102 adds a new template to the template database 130, as shown in the lower part of Figure 12. The added template exemplified in the lower part of Figure 12 is a fill-in-the-blank text with an increased number of fill-in-the-blank sections compared to the existing template (see Figure 4).

[0096] After step S20, processor 102 terminates the flowchart in Figure 9. Also, after step S16, or if the result of step S19 is a Yes determination, processor 102 terminates the flowchart in Figure 9 without updating the template database 130.

[0097] 《Another specific example 1: Adding word suggestions》 Figure 13 is an explanatory diagram illustrating an example of the process when word candidates are added. Figure 13 shows an example of a finding sentence related to lymph node enlargement. The processor 102 reflects the image processing results and presents multiple finding sentence candidates, including the finding sentences [1] to [3], as candidate findings for lymph node enlargement, for example, as shown in the upper part of Figure 13.

[0098] Suppose the doctor rejected all of the proposed findings and wrote his own findings. For example, as shown in the middle of Figure 13, the doctor wrote the following sentence: "A lymph node enlargement measuring 1.3 cm in diameter was observed in the hilar region. A reactive lymph node is suspected."

[0099] Upon receiving such a medical report from the physician, the processor 102 structures the text of the report using natural language processing technology to obtain the result shown in the lower part of Figure 13. Based on the structuring result, the processor 102 determines whether the medical report can be represented using an existing template in the template database 130, based on the degree of agreement of at least one of the characteristics of the lesion and the disease name.

[0100] From the structuring results shown in the lower part of Figure 13, it is clear that "reactive lymph node" should be added as a disease name. Therefore, processor 102 determines that the template database 130 needs to be updated and performs the process of adding "reactive lymph node" to the word candidates for "disease name" in the lymph node enlargement template. That is, processor 102 updates the lymph node enlargement template as shown in Figure 14. In Figure 14, the added word candidate "reactive lymph node" is underlined. As is clear when compared with the template before the update shown in Figure 7, the updated template (Figure 14) has "reactive lymph node" added to the word candidates.

[0101] 《Another specific example 2: Changing word suggestions》 Figure 15 is an explanatory diagram illustrating an example of processing when word candidates are changed. Figure 15 shows an example of a finding sentence related to a liver tumor. The processor 102 reflects the image processing results and presents multiple candidates as candidate finding sentences for liver tumors, as shown in the upper part of Figure 15. Here, three examples of candidate finding sentences, [1] to [3], are shown.

[0102] For example, the physician selected the finding statement [3], "A low-attenuation area measuring 2 cm in length was observed in liver segment S1. Hepatocellular carcinoma is suspected." However, he changed the word "hepatocellular carcinoma" to "HCC," and as shown in the middle of Figure 15, he rewrote the statement to "A low-attenuation area measuring 2 cm in length was observed in liver segment S1. HCC is suspected." and saved this rewritten finding statement. Note that "HCC" is an abbreviation for Hepatocellular Carcinoma.

[0103] Upon receiving such rewriting (editing) of the medical report by the physician, the processor 102 structures the medical report saved by the physician after rewriting using natural language processing technology, obtaining the result shown in the lower part of Figure 15.

[0104] From this structuring, it can be seen that the disease name is "HCC". By having a thesaurus in the system, processor 102 can see that "hepatocellular carcinoma" and "HCC" are synonyms, and so processor 102 changes "hepatocellular carcinoma" to "HCC" among the word candidates for "disease name" in the liver tumor template. That is, processor 102 updates the liver tumor template as shown in Figure 16. In Figure 16, the changed word candidate "HCC" is underlined. As is clear when compared with the template before the update shown in Figure 5, the updated template (Figure 15) now includes "HCC" among the word candidates.

[0105] 《Another specific example 3: Changing word order》 Figure 17 is an explanatory diagram illustrating an example of how to handle cases where the word order of a fill-in-the-blank sentence is changed. Figure 17 shows an example of a finding sentence regarding a pulmonary nodule.

[0106] The processor 102 reflects the image processing results and presents multiple candidate findings for the lung nodule, as shown in the upper part of Figure 17. Here, we show an example where three candidate findings [1] to [6] are presented based on the template explained in Figure 4.

[0107] For example, suppose a physician selects the finding statement [2] "A nodule with a major diameter of 3 cm was found in the right lung S7" from the candidate findings statements shown in the upper part of Figure 17, but changes the word order in the sentence to rewrite it as "A nodule was found in the right lung S7. Its major diameter is 3 cm," as shown in the middle part of Figure 17, and saves this rewritten finding statement. Upon receiving this rewritten and saved finding statement by the physician, processor 102 structures this saved finding statement using natural language processing technology to obtain the result shown in the lower part of Figure 17.

[0108] The structured results show that they are within the range that can be expressed by the original [2] candidate observations.

[0109] Based on the structuring results, processor 102 identifies the fill-in-the-blank sentence of the template that served as the basis for the candidate findings in [2] as a similar template, and processes the existing fill-in-the-blank sentence to change it to the sentence "A nodule was found at (location). Its longest diameter is XX cm." That is, processor 102 updates the pulmonary nodule template as shown in Figure 18. In Figure 18, the changed fill-in-the-blank sentence (identification code [SN002]) is underlined. As is clear from comparing it with the template before the update shown in Figure 4, the updated template (Figure 18) has a changed word order in the fill-in-the-blank sentence.

[0110] 《Another specific example 4: Changes in expression》 Figure 19 is an explanatory diagram illustrating an example of processing when the wording of the fill-in-the-blank sentences is changed. Figure 19 shows an example of a finding sentence regarding a liver cyst. The processor 102 reflects the image processing results and presents multiple candidate findings sentences for the liver cyst, as shown in the upper part of Figure 19. Here, we show an example in which two candidate findings sentences, [1] and [2], are presented based on the template described in Figure 6.

[0111] Suppose a physician selects the finding statement, for example, [1] "A cyst was found in liver segment S1," but modifies the wording to rewrite it as "A cyst was found in liver segment S1," as shown in the middle of Figure 19, and saves this modified finding statement. Upon receiving this modified and saved finding statement from the physician, processor 102 structures the saved finding statement using natural language processing technology to obtain the result shown in the lower part of Figure 19.

[0112] The structuring results indicate that the content is within the range that can be expressed by the original [1] candidate findings. Therefore, in this case, processor 102 updates the hepatic cyst template as shown in the upper part of Figure 20. As is clear when compared with the unupdated template shown in Figure 6, the updated template (Figure 20) has the fill-in-the-blank sentences changed from a polite "desu / masu" style to a more formal "dearu" style.

[0113] The physician selected and modified the candidate findings shown in the upper part of Figure 19 [1], but as shown in the upper part of Figure 20, the system may also update all of its hepatic cyst templates to reflect the stylistic changes.

[0114] Furthermore, if a physician replaces the sentence "A cyst was found in liver S1," as shown in the middle of Figure 19, with a nominalized expression such as "There is a cyst in liver S1," the processor 102 updates the liver cyst template with a fill-in-the-blank sentence using the nominalized expression, as shown in the lower part of Figure 20.

[0115] 《Another example 5: Adding fill-in-the-blanks》 Figure 21 is an explanatory diagram illustrating an example of processing when a blank is added to a fill-in-the-blank sentence. Figure 21 shows an example of a finding statement regarding a time-series comparison of pulmonary nodules. The processor 102 reflects the image processing results and presents multiple candidate findings statements for the time-series comparison of pulmonary nodules, as shown in the upper part of Figure 21. Here, we show an example in which three candidate findings statements, [1] to [3], are presented based on the template explained in Figure 8.

[0116] For example, suppose a physician selects the finding statement [2] "The nodule in left lung S1 has increased in size" from among several candidate findings statements shown in the upper part of Figure 21, but adds a description regarding the size and rewrites it as shown in the middle part of Figure 21, "The nodule in left lung S1 has increased in length to 2 cm," and saves this rewritten finding statement. Upon receiving such a rewritten and saved finding statement by the physician, processor 102 structures this saved finding statement using natural language processing technology and obtains the result shown in the lower part of Figure 21.

[0117] The structuring results indicate that information about size has been added. Therefore, in this case, processor 102 generates a new template that includes the size fill-in section, for example, "The nodule at (location) has increased to (longest diameter ○○ cm)," and adds this to the template database 130. In other words, processor 102 updates the template for the time-series comparison of lung nodules as shown in Figure 22. As is clear from comparing it with the template before the update shown in Figure 8, the updated template (Figure 22) includes the size fill-in section and the fill-in section (identification code [SN042]).

[0118] 《Another specific example 6: An example of a free-response comment by a doctor》 Figure 23 shows an example of a medical report written by a physician using free-form text. If the physician does not find a suitable report among the candidates presented by the information processing device 10, they can ignore the candidates and complete the report manually. This is the case where step S15 in Figure 9 is judged as NO.

[0119] For example, suppose a doctor manually enters a finding statement into an input device 12, such as, "Abnormalities of ground-glass opacities with internal cavities attached to the chest wall are observed throughout both lungs. Diagnosed as ILD." ILD is an abbreviation for Interstitial lung disease.

[0120] In this case, the information processing device 10 analyzes the manually entered text using natural language processing technology and obtains the analysis results shown in the lower part of Figure 23. The information processing device 10 determines the words extracted from the text (extracted words) and the item names for each word. The determination unit in the information processing device 10, which determines whether or not a template update is necessary, may determine that the disease name is ILD and is the same as the disease name determined by CAD, but that some of the item names have changed compared to the candidate findings sentences presented to the doctor. For example, if the fill-in-the-blank sentence in the existing template does not have a fill-in-the-blank part for the item name "(size)", the determination unit will determine that the sentence cannot be expressed in the existing template and will generate a new template corresponding to the manually entered findings sentence.

[0121] Figure 24 shows an example of the new template. The new template corresponding to the manually entered findings (example in Figure 23) will be a fill-in-the-blank sentence as shown in Figure 24, which will be "(Organ) (Location) (Size), (Properties 1), (Properties 2), (Properties 3) (Disease Name) (Factual Information 1). (Disease Name) and (Factual Information 2)." This fill-in-the-blank sentence will have the "(Size)" part added, which was not present in the existing template.

[0122] The information processing device 10 may add such fill-in-the-blank sentences as new ILD templates, or it may overwrite an existing ILD template with the template shown in Figure 24 and save it.

[0123] Whether to add a new entry or overwrite an existing one can be determined by an appropriate method. For example, the physician may be asked to decide whether to "add a new entry," "overwrite an existing entry," or "do not register." When the physician inputs an instruction of "add a new entry," "overwrite an existing entry," or "do not register" using a user interface such as the input device 12, the information processing device 10 performs the corresponding processing.

[0124] Furthermore, the registration process may be automatically determined not only by a physician's decision, but also based on factors such as the severity of the disease name, which may be prepared separately. For example, if the disease name is serious or life-threatening, an additional registration may be performed, while in other cases, the registration may be overwritten.

[0125] Alternatively, the system could be structured so that templates based on documents written by specific, highly experienced physicians are added, while others are overwritten, depending on the physician's experience level or position within the hospital.

[0126] Examples of candidate reports based on templates. Figure 25 is an explanatory diagram showing an example of a template-based presentation of candidate findings. The upper part of Figure 25 shows an example of a template, and the lower part of Figure 25 shows an example of a candidate finding that has been automatically filled in by the image processing results.

[0127] In the template's fill-in-the-blank text, the placeholders for "(location)" and "(longest diameter XX cm)" are automatically filled in based on the image processing results. On the other hand, the placeholder for "(disease name)" is a fill-in-the-blank section for which word suggestions are provided.

[0128] Based on the image processing results, the physician is presented with several candidate findings, for example, as shown in the lower part of Figure 25. Here, an example is shown where a pulmonary nodule with a major diameter of 3 cm is detected in lung S1 through image processing.

[0129] Example 1 of a doctor's rewriting A physician can select one of several proposed findings and either use it as is or rewrite parts of it. Figure 26 shows an example of rewriting by a physician. For example, suppose a physician uses one of the several proposed findings shown in the lower part of Figure 25, [2] "A 3 cm longest diameter pulmonary nodule was found in lung S1. Lung cancer is suspected," as a base and rewrites it to the sentence shown in the upper part of Figure 26, "An irregularly shaped pulmonary nodule 3 cm longest diameter was found in lung S1. Lung cancer is suspected," and saves it. In this case, based on natural language processing, it is understood that the expression "irregularly shaped" has been added as a phrase describing the nature of the pulmonary nodule.

[0130] As a result, the information processing device 10 updates the template database 130 by adding a new fill-in-the-blank sentence as a new template, as shown in the middle of Figure 26: "A pulmonary nodule of (length XX cm) with (characteristics) is found at (location); (disease name) is suspected."

[0131] After adding the template, five types of candidate findings [1] to [5] may be presented for the same image processing result, as shown in the lower part of Figure 26.

[0132] 《Example of rewriting by a doctor, Part 2》 Figure 27 shows an example of rewriting by a physician, as shown in the upper part of Figure 27. For example, suppose a physician takes one of the multiple candidate findings shown in the lower part of Figure 25, candidate [1], and rewrites it to the sentence, "A pulmonary nodule measuring 3 cm in its longest diameter was found in lung S1. Hamartoma is suspected," and saves it. In this case, based on natural language processing, it is understood that the expression "hamartoma" has been added as a phrase representing the disease name.

[0133] As a result, the information processing device 10 updates the template database 130 by adding "hamartoma" to the list of disease name word candidates as a new template, as shown in the middle of Figure 27. After the update, for the same image processing result, four types of candidate findings [1] to [4] may be presented, including the candidate finding [4] "A pulmonary nodule with a longest diameter of 3 cm is found in lung S1. Hamartoma is suspected." as shown in the lower part of Figure 27.

[0134] 《Example 3 of rewriting by a doctor》 Figure 28 shows example 3 of rewriting by a physician. For example, suppose a physician uses candidate [3] of the multiple candidate findings shown in the lower part of Figure 25 as a base and rewrites it as shown in the upper part of Figure 28 to the sentence, "A pulmonary nodule measuring 3 cm in its longest diameter was found in lung S1. Metastatic lung cancer is suspected." and saves it. In this case, based on natural language processing, it can be understood that the expressions "found" and "suspected" are used as phrases to express factual accuracy.

[0135] As a result, the information processing device 10 updates the template database 130 by changing the style of the fill-in-the-blank sentences, as shown in the middle section of Figure 28. After the update, if a similar case is found from images of another patient, three types of candidate findings [1] to [3] may be presented based on the updated template, as shown in the lower section of Figure 28.

[0136] 《Example 4 of rewriting by a doctor》 Figure 29 shows example 4 of rewriting by a physician. For example, suppose a physician takes one of the multiple candidate findings shown in the lower part of Figure 25, [3] "A 3 cm longest diameter pulmonary nodule was found in lung S1. Metastatic lung cancer is suspected," as a base and rewrites it to the sentence shown in the upper part of Figure 29, "A 3 cm longest diameter pulmonary nodule was found in lung S1. Metastatic lung cancer is suspected," and saves it. In this case, based on natural language processing, it can be seen that the word order of size and location has been changed.

[0137] As a result, the information processing device 10 generates a new fill-in-the-blank sentence with the size and location word order changed, as shown in the middle of Figure 29, which is "A lung nodule is found at (location) with a longest diameter of ○○ cm. (Disease name) is suspected." and updates the template database 130 by adding or overwriting the template.

[0138] After the update, if a similar case is found in images of another patient, etc., three types of candidate findings [1] to [3] may be presented based on the updated template, as shown in the lower part of Figure 29.

[0139] Regarding the method of presenting candidate reports: When presenting the user with multiple candidate findings that have been filled in based on the image processing results, the candidate findings may be displayed in order of the frequency of use of the templates. The order of frequency of use may be in order of the number of times used, or it may be the number of times used in the most recent predetermined period. The information processing device 10 can prioritize the display of candidate findings that are likely to be used by recording the usage history of the templates adopted for writing the findings in the image interpretation report.

[0140] Regarding programs that operate computers: It is possible to record a program that enables a computer to implement some or all of the processing functions of the information processing device 10 on a computer-readable medium, such as an optical disk, magnetic disk, or semiconductor memory, which is a tangible, non-temporary information storage medium, and to provide the program through this information storage medium.

[0141] Alternatively, instead of providing programs by storing them on tangible, non-temporary computer-readable media, it is also possible to provide program signals as a download service using telecommunication lines such as the Internet.

[0142] Furthermore, some or all of the processing functions in the information processing device 10 may be implemented by cloud computing, and Sa aIt can also be offered as a Software as a Service (S).

[0143] Regarding the hardware configuration of each processing unit: The hardware structure of the processing units that perform various processes in the information processing device 10, such as the data acquisition unit 312, image processing unit 340, report creation support unit 350, opinion text candidate generation unit 352, text editing unit 354, text analysis unit 356, template update unit 358, and display control unit 360, is, for example, various processors as shown below.

[0144] Various types of processors include CPUs, which are general-purpose processors that execute programs and function as various processing units; GPUs, which are processors specialized for image processing; Programmable Logic Devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing; and Dedicated Electrical Circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to perform particular processing.

[0145] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type. For example, a single processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU. Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as clients and servers. Secondly, a configuration where a processor is used that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0146] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit composed of circuit elements such as semiconductor devices.

[0147] Advantages of the Embodiment According to the information processing device 10 of this embodiment, the updating of the template database 130 can be automated, thereby reducing the workload of users in updating the database. Furthermore, the information processing device 10 can utilize templates stored in the template database 130 to present useful candidate findings, thereby streamlining the report creation process by physicians.

[0148] Other application examples While the above-described embodiment illustrates an example of a system that assists in the creation of image interpretation reports, the technology of this disclosure can be applied to systems that assist in the creation of various medical documents, not limited to image diagnostic reports such as image interpretation reports, but also including documents related to patient medical records. Examples of documents related to patient medical records include interim summaries and discharge summaries. Furthermore, the technology of this disclosure can be applied to updating databases containing templates for documents in various fields, regardless of the type or use of the object, such as inspection result reports for industrial products, inspection result reports for buildings, or various expert reports.

[0149] "others" This disclosure is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the technical idea of ​​this disclosure. [Explanation of symbols]

[0150] 10 Information Processing Devices 12 Input devices 12A Mouse 14 Display device 102 processors 104 Computer-readable media 106 Communication Interface 108 Input / Output Interfaces 110 Bus 122 memory 124 storage 130 Template Databases 134 Report Storage Section 140 Image Processing Programs 150 Report Writing Support Program 152 Report Candidate Generation Program 154 Text editing programs 156 Text Analysis Program 158 Template Update 160 Display Control Program 200 Medical Information Systems 202 Electronic Medical Record System 204 CT device 206 MRI machine 210 Image storage server 212 Image Database 230 Viewer Terminals 232 Input device 234 Display device 240 communication lines 312 Data Acquisition Unit 316 Communications Department 340 Image Processing Unit 350 Report Creation Support Department 352 Observation candidate generation unit 354 Editorial Department 356 Sentence Analysis Department 358 Template Update Section 360 Display Control Unit IMG image S11-S20 Steps of the information processing method

Claims

1. One or more processors, One or more storage devices that store a database containing multiple templates related to documents, An information processing device comprising, The aforementioned multiple templates include fill-in-the-blank sentences and candidate words or phrases that can be placed in the blanks, The one or more processors described above are Obtain at least one processing result from image processing, including organ detection, lesion detection, lesion region segmentation, and size measurement. Based on the image processing results and the template stored in the database, one or more candidate sentences are generated in which at least a portion of the blanks are filled in by the image processing results. One or more of the above-mentioned candidate sentences are presented, The input device accepts text different from the aforementioned text candidates. The received text is analyzed and structured, Based on the analysis results of the document including the structuring results, it is determined whether the document can be represented using the template in the database. If it can be represented, it is determined that updating the database is unnecessary. If it cannot be represented, it is determined that updating the database is necessary. When it is determined that the database needs to be updated, the database is updated by performing at least one of the following processes: adding new fill-in-the-blank sentences, modifying existing fill-in-the-blank sentences, adding new word candidates associated with the fill-in-the-blank portion, and modifying existing word candidates associated with the fill-in-the-blank portion. Information processing device.

2. The aforementioned document is a medical document. The information processing apparatus according to claim 1.

3. The aforementioned medical document relates to the results of imaging diagnostics. The information processing apparatus according to claim 2.

4. The aforementioned medical document pertains to the patient's medical record. The information processing apparatus according to claim 2.

5. The one or more processors described above are The system generates fill-in-the-blank sentences corresponding to the expressions in the received text, The generated fill-in-the-blank sentences are added to the database and the database is updated. The information processing apparatus according to any one of claims 1 to 4.

6. When the received text contains a phrase of a structured classification item corresponding to the fill-in-the-blank portion contained in an existing fill-in-the-blank text stored in the database, and also contains a phrase of a classification item not contained in the existing fill-in-the-blank text, the fill-in-the-blank text generated in response to the expression of the received text has an increased number of fill-in-the-blank portions compared to an existing fill-in-the-blank text stored in the database. The information processing apparatus according to claim 5.

7. The aforementioned fill-in-the-blank section includes a portion in which a phrase representing at least one of the characteristics of the lesion and the name of the disease is inserted. The information processing apparatus according to claim 6.

8. The one or more processors described above are Based on the degree of agreement between at least one of the characteristics of the lesion and the disease name in the received document and at least one of the characteristics of the lesion and the disease name in the template in the database, it is determined whether the document can be expressed using the template in the database. The information processing apparatus according to claim 7.

9. The one or more processors described above are The system generates fill-in-the-blank sentences corresponding to the expressions in the received text, The number of blanks, the type of blanks, and the order of the blanks are compared with the blank-filled text of the template in the database to determine whether the text can be expressed using the template in the database. The information processing apparatus according to any one of claims 1 to 8.

10. The one or more processors described above are The received text is structured using natural language processing. An information processing apparatus according to any one of claims 1 to 9.

11. The one or more processors described above are From among the multiple templates stored in the database, identify a template that is similar to the received document. The database is updated by modifying a portion of the identified template and overwriting and saving the template. The information processing apparatus according to any one of claims 1 to 10.

12. The one or more processors described above are The style of the identified template is changed and the template is overwritten and saved. The information processing apparatus according to claim 11.

13. The one or more processors described above are The word order of the fill-in-the-blank sentences in the identified template is changed, and the template is overwritten and saved. The information processing apparatus according to claim 11.

14. The one or more processors described above are Based on the structuring results of the received text, the words associated with the blanks in the fill-in-the-blank sentences that can represent the text in the database are determined. The determined phrase is added to the database as a candidate phrase that can be inserted into the blank. The information processing apparatus according to any one of claims 1 to 13.

15. The blank portion in the aforementioned fill-in-the-blank text is of a first type, which is filled in by the result of the image processing, and a second type, which is not filled in by the result of the image processing. The information processing apparatus according to any one of claims 1 to 14.

16. The first type of fill-in-the-blank portion includes a portion to be filled in with a phrase representing at least one of place and quantity, The second type of fill-in-the-blank section includes a section in which a word or phrase representing the name of a disease is to be inserted to fill in the blank. The information processing apparatus according to claim 15.

17. The one or more processors described above are Based on the image processing results and the template stored in the database, one or more candidate sentences are generated in which the first type of fill-in-the-blank portion is filled in. One or more of the above-mentioned candidate sentences are presented as options, The system accepts instructions to select one of the aforementioned sentence candidates. The information processing apparatus according to claim 15 or 16.

18. The one or more processors described above are The system accepts input to rewrite the aforementioned suggested sentences. Based on the analysis results of the document saved after the rewriting, it is determined whether the document can be expressed using the template in the database, and if it is determined that it cannot be expressed and it is determined that the database needs to be updated, the database is updated. The information processing apparatus according to claim 17.

19. The one or more processors described above are Based on the template stored in the database, the system presents candidate words that can be inserted into the second type of fill-in-the-blank section. The system accepts instructions to select one of the aforementioned word / phrase candidates. The information processing apparatus according to any one of claims 15 to 18.

20. An information processing method in which one or more processors update a database containing multiple templates for documents, The aforementioned multiple templates include fill-in-the-blank sentences and candidate words or phrases that can be placed in the blanks, The one or more processors described above Obtaining at least one processing result from image processing, including organ detection, lesion detection, lesion region segmentation, and size measurement, Based on the image processing results and the template stored in the database, one or more candidate sentences are generated in which at least a portion of the blanks are filled in by the image processing results. To present one or more of the above-mentioned candidate sentences, The system accepts input of text different from the aforementioned text candidates via an input device, The process involves analyzing the received text and structuring it, Based on the analysis results of the document including the structuring results, it is determined whether the document can be represented using the template in the database. If it can be represented, it is determined that updating the database is unnecessary. If it cannot be represented, it is determined that updating the database is necessary. When it is determined that the database needs to be updated, the database is updated by performing at least one of the following processes: adding new fill-in-the-blank sentences, modifying existing fill-in-the-blank sentences, adding new word candidates associated with the fill-in-the-blank portion, and modifying existing word candidates associated with the fill-in-the-blank portion. Information processing methods including

21. A program that enables a computer to update a database containing multiple templates related to documents, The aforementioned multiple templates include fill-in-the-blank sentences and candidate words or phrases that can be placed in the blanks, To the aforementioned computer, A function to acquire at least one processing result from image processing, including organ detection, lesion detection, lesion region segmentation, and size measurement. A function that generates one or more sentence candidates in which at least a portion of the blanks are filled in by the image processing result, based on the image processing result and the template stored in the database, The function of presenting one or more of the aforementioned sentence candidates, A function that accepts input of text different from the aforementioned text candidates via an input device, The function includes analyzing the received text and structuring the text, A function that determines whether the document can be represented using the template in the database based on the analysis results of the document including the structuring results, determines that updating the database is unnecessary if it can be represented, and determines that updating the database is necessary if it cannot be represented, A function to update the database by performing at least one of the following processes when it is determined that the database needs to be updated: adding new fill-in-the-blank sentences, modifying existing fill-in-the-blank sentences, adding new word candidates associated with the fill-in-the-blank portion, and modifying existing word candidates associated with the fill-in-the-blank portion. A program that makes this possible.

Images