Information processor, method for processing information, and program

Abstract

To provide an information processor, a method for processing information, and a program that can analyze a sentence accurately.SOLUTION: An information processor includes: at least one processor; and at least one memory for storing an order executed by the at least one processor. The at least one processor acquires a sentence, classifies an attribute of information described in the sentence on a constant unit of the sentence, analyzes the sentence for same classification on the basis of the result of the classification, and outputs the result of the analysis.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program, and particularly relates to natural language processing technology for analyzing text.

Background Art

[0002] The text created in the medical field is text freely described by medical personnel including doctors, and it is unstructured data that is difficult to reuse statistically or analyze content as it is. In a radiology report, which is one type of medical document, a doctor observes images taken by medical devices such as a CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device, and the results of grasping the properties such as the location, size, shape, or internal structure of each disease are described. In order to obtain this information described in the report, there is an increasing need for technology to structure the text of the report.

[0003] Patent Document 1 discloses an information processing apparatus that analyzes free-form text, obtains and classifies term expressions such as medical terms based on the analysis results of the text, and presents the classification results in a unified manner. Morphological analysis and dependency analysis are used for the analysis process in the information processing apparatus described in Patent Document 1, and statistical information or co-occurrence relationships are used for the classification process.

[0004] Patent Document 2 discloses a report creation apparatus including a structural analysis unit that analyzes a radiology report and identifies each character string included in the text in terms of description units.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Non-Patent Documents

[0006] [Non-Patent Document 1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”<https: / / arxiv.org / pdf / 1810.04805> [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] Medical documents contain not only descriptions of various organs or diseases, but also texts unrelated to those diseases, and the relationships between terms are often confined to the same organ or disease. However, conventional technologies cannot adequately grasp information about the relationships between descriptions in texts containing descriptions of multiple things, resulting in challenges such as inaccurate extraction of terminology, acquisition of medically incorrect relationships, or failure to acquire necessary relationships.

[0008] This disclosure is made in view of these circumstances and aims to provide an information processing device, an information processing method, and a program that can perform text analysis with high accuracy. [Means for solving the problem]

[0009] An information processing device according to one aspect of the present disclosure comprises one or more processors and one or more memories in which instructions executed by the one or more processors are stored, wherein the one or more processors acquire a document, classify the attributes of the information described in the document into certain units of the document, analyze the document for each classification based on the results of the classification, and output the results of the analysis.

[0010] According to this embodiment, the attributes of the information described in the text are classified, and text analysis is performed within each classification. Therefore, compared to analyzing texts containing a mix of different classifications together, the accuracy of the analysis can be improved.

[0011] In information processing devices relating to other aspects of this disclosure, the certain unit may be a sentence unit, a phrase unit, a word unit, or a character unit.

[0012] In information processing devices relating to other aspects of this disclosure, the text may be medical text.

[0013] In other embodiments of this disclosure, the information classification items may include one or more of the following: human body parts, organs, types of diseases, types of medical processes, and the presence or absence of disease. This embodiment makes it possible to extract medically appropriate terminology and to obtain medically correct relationships between terms.

[0014] In other aspects of the information processing device relating to this disclosure, the type of medical process may be configured to include at least one of findings, diagnosis, and past comparison.

[0015] In information processing devices relating to other aspects of this disclosure, the process of analyzing text for each category may include a process of extracting terms.

[0016] In other embodiments of the present disclosure, the process for analyzing text for each category may include a process for extracting terms and a process for obtaining the relationships between terms. According to this embodiment, the correct relationships between terms in a text can be obtained.

[0017] In other embodiments of the present disclosure, the term extraction may include the acquisition of term expressions and the determination of the type of term.

[0018] In an information processing apparatus according to another aspect of the present disclosure, the term extraction process may be configured to be performed using a prediction model that has been machine-learned in advance.

[0019] In an information processing apparatus according to another aspect of the present disclosure, at least one of the term extraction process and the process of obtaining the relationship between terms may be configured to be performed using a prediction model that has been machine-learned in advance.

[0020] In an information processing apparatus according to another aspect of the present disclosure, different prediction models may be used according to the classification results.

[0021] In an information processing apparatus according to another aspect of the present disclosure, as inputs to the prediction model, sentences for each same classification and classification information of the same classification may be used.

[0022] In an information processing apparatus according to another aspect of the present disclosure, it may further include an input device that receives an input of a sentence and a display device that displays the analysis result.

[0023] In an information processing apparatus according to another aspect of the present disclosure, one or more processors may be configured to save the classification result and perform a process of visibly displaying the classification result when displaying the obtained sentence.

[0024] An information processing method according to another aspect of the present disclosure is an information processing method executed by one or more processors, and includes: one or more processors obtaining a sentence, classifying the attributes of the information described in the sentence in a certain unit of the sentence, analyzing the sentence for each same classification based on the classification result, and outputting the analyzed result.

[0025] A program according to another aspect of the present disclosure causes a computer to realize functions of obtaining a sentence, classifying the attributes of the information described in the sentence in a certain unit of the sentence, analyzing the sentence for each same classification based on the classification result, and outputting the analyzed result. [Effects of the Invention]

[0026] According to this disclosure, the accuracy of text analysis can be improved. [Brief explanation of the drawing]

[0027] [Figure 1] Figure 1 is a block diagram showing an example configuration of a medical information system including an information processing device according to an embodiment. [Figure 2] Figure 2 is a functional block diagram showing an overview of the processing functions of the information processing device. [Figure 3] Figure 3 is a block diagram schematically showing an example of the hardware configuration of an information processing device. [Figure 4] Figure 4 is a flowchart showing an example of an information processing method performed by an information processing device. [Figure 5] Figure 5 is an explanatory diagram showing an example of text included in an image interpretation report acquired by an information processing device. [Figure 6] Figure 6 is an explanatory diagram showing an example of the results of determining the type of medical process on a sentence-by-sentence basis for the text of the image interpretation report shown in Figure 5. [Figure 7] Figure 7 is an explanatory diagram illustrating an example of a process that determines the type of organ on a sentence-by-sentence basis and performs analysis on sentences that refer to the same organ. [Figure 8] Figure 8 is an explanatory diagram showing an example of the configuration of a term extraction unit in an information processing device. [Figure 9] Figure 9 is a graph showing an example of a probability distribution for named entity recognition (NER) labels. [Figure 10] Figure 10 is an explanatory diagram showing an example of the configuration of a relationship acquisition unit in an information processing device. [Figure 11] Figure 11 is an explanatory diagram illustrating an example of a process that performs organ recognition at the sentence level. [Figure 12] Figure 12 is a diagram showing an example of structured information obtained by extracting terms from sentences for each organ based on the organ recognition results shown in Figure 11. [Figure 13] Figure 13 is an explanatory diagram illustrating an example of a process that performs organ recognition at the phrase level and extracts terms for each organ phrase. [Figure 14] Figure 14 is a diagram showing an example of structured information obtained by performing term extraction and relationship extraction on sentences for each organ based on the organ recognition results shown in Figure 11. [Figure 15] Figure 15 is an explanatory diagram illustrating an example of a process that performs organ recognition at the sentence level and analyzes the text using different models for each organ in sentences related to that organ. [Figure 16] Figure 16 is a block diagram showing an example of the configuration of the text analysis unit, which uses different models for each organ. [Figure 17] Figure 17 is a diagram illustrating an example of structured information obtained through text analysis using the machine learning model trained for cardiac studies shown in Figure 16. [Figure 18] Figure 18 is a diagram illustrating an example of structured information obtained through text analysis using the machine learning model trained for lung function shown in Figure 16. [Figure 19] Figure 19 is a conceptual diagram of a text analysis unit that uses a model to input organ recognition results as auxiliary information. [Figure 20] Figure 20 is an explanatory diagram illustrating an example of processing using the machine learning model shown in Figure 19. [Figure 21] Figure 21 is an explanatory diagram illustrating an example of a process for structuring factory maintenance reports. [Modes for carrying out the invention]

[0028] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings.

[0029] Examples of medical information system configurations This section describes an example of an information processing device 10 that analyzes the text of a radiology report, a type of medical document. Figure 1 is a block diagram showing an example configuration of a medical information system 200 including the information processing device 10 according to this embodiment. The medical information system 200 is implemented as a computer network built in a medical institution such as a hospital. The medical information system 200 includes an electronic medical record system 202, an examination order system 203, a CT scanner 204, an MRI scanner 206, an image storage server 210, a report server 220, the information processing device 10, and a terminal device 230, and these elements are connected via a communication line 240.

[0030] The electronic medical record system 202 and the laboratory order system 203 may each utilize a server-type computer. The server-type computer may be configured as a network of multiple computers working together. The communication line 240 may be an on-site communication line within the medical institution. Furthermore, a portion of the communication line 240 may include a wide-area communication line. Some elements of the medical information system 200 may be configured using cloud computing.

[0031] Figure 1 illustrates CT scanner 204 and MRI scanner 206 as examples of modalities. However, medical imaging equipment is not limited to CT scanner 204 and MRI scanner 206. Various other diagnostic devices are possible, including ultrasound diagnostic equipment (not shown), PET (Positron Emission Tomography) equipment, mammography equipment, X-ray diagnostic equipment, X-ray fluoroscopy diagnostic equipment, and endoscopes. The types and number of modalities connected to the communication line 240 can vary from one medical institution to another.

[0032] The electronic medical record system 202 manages electronic medical records for each patient. The electronic medical record system 202 includes an electronic medical record storage device that stores the electronic medical records. The electronic medical record system 202 stores patient identification information in association with the electronic medical records and can retrieve patient-specific electronic medical records using patient identification information as a parameter. In response to a read request transmitted from a terminal device 230 or information processing device 10, the electronic medical record system 202 searches for the electronic medical records and transmits various information contained in the electronic medical records corresponding to the read request to the requesting terminal device 230, etc.

[0033] The examination order system 203 manages examination orders issued based on examination order requests issued by physicians. Examination orders include various information related to the examination, such as patient identification information (e.g., patient ID), physician identification information (e.g., attending physician ID), and the type of examination. The examination order system 203 is equipped with an examination order storage device for storing examination orders.

[0034] The image storage server 210 may be, for example, a DICOM server operating according to the DICOM (Digital Imaging and Communications in Medicine) specification. The image storage server 210 is a computer that stores and manages various data, including images acquired using various modalities such as the CT scanner 204 and the MRI scanner 206, and is equipped with a large-capacity external storage device and a database management program. The image storage server 210 communicates with other devices via the communication line 240 and sends and receives various data, including image data. The image storage server 210 receives various data, including images generated by modalities such as the CT scanner 204, via the communication line 240, and stores and manages them on a recording medium such as a large-capacity external storage device. The storage format of the image data and communication between each device via the communication line 240 are based on the DICOM protocol.

[0035] The report server 220 is a computer that stores and manages various medical documents, including image interpretation reports. These medical documents include reports on diagnostic imaging results, such as image interpretation reports, as well as patient medical records. While this embodiment primarily uses image interpretation reports as an example, the documents covered are not limited to image interpretation reports. The explanation regarding image interpretation reports can be interpreted and applied to various other types of documents.

[0036] The report server 220 stores electronic medical records, medical images, and image interpretation reports in association with each other. The report server 220 may also be equipped with a program to assist in the creation of image interpretation reports. The report server 220 communicates with other devices via the communication line 240 to send and receive various data such as image interpretation reports.

[0037] The information processing device 10 can acquire data from the report server 220, etc., via the communication line 240. The information processing device 10 processes the text data contained in the report and performs text analysis such as classifying descriptive information and extracting terms. Details of the processing functions of the information processing device 10 will be described later. The information processing device 10 can be implemented using computer hardware and software. The form of the information processing device 10 is not particularly limited and may be a server computer, a workstation, a personal computer, or a tablet terminal. In this embodiment, an example is described in which the information processing device 10 and the report server 220 are separate devices, but some or all of the processing functions of the information processing device 10 may be incorporated into another computer, such as the report server 220.

[0038] The information processing device 10 may include an input device 22 and a display device 24. The input device 22 may be, for example, a keyboard, mouse, multi-touch panel, or other pointing device, or an audio input device, or an appropriate combination thereof. The display device 24 may be, for example, a liquid crystal display, an organic electro-luminescence (OEL) display, or a projector, or an appropriate combination thereof. The input device 22 and the display device 24 may be configured as an integrated unit, such as a touch panel. The input device 22 and the display device 24 may be included in the information processing device 10, or the information processing device 10, the input device 22, and the display device 24 may be configured as an integrated unit.

[0039] The information processing device 10 can send the results of the text analysis to other devices such as the report server 220 and the terminal device 230.

[0040] The terminal device 230 may be a viewer terminal for viewing images, such as a PACS (Picture Archiving and Communication Systems) viewer or a DICOM viewer. Although Figure 1 illustrates one terminal device 230, multiple terminal devices 230 may be connected to the communication line 240. The form of the terminal device 230 is not particularly limited; it may be a personal computer, a workstation, or a tablet terminal. The terminal device 230 includes an input device 232 and a display device 234. The input device 232 and the display device 234 may have the same configuration as the input device 22 and the display device 24 of the information processing device 10.

[0041] Various data stored in the image database of the image storage server 210, as well as various information including processing results generated by the information processing device 10, can be displayed on the display device 234 of the terminal device 230.

[0042] Furthermore, the medical information system 200 may include an image processing device (not shown). The image processing device is equipped with an image processing program that performs image analysis on medical images captured by a modality. The image processing device may be configured to perform various computer-aided diagnosis (Computer-Aided Diagnosis, Computer-Aided Detection: CAD) analysis processes, such as recognizing lesion areas from input images, identifying classifications such as disease names, or segmentation processes that recognize areas such as organs. It may also perform processes that support the creation of image interpretation reports using the image processing results. The processing functions of the image processing device may be incorporated into the information processing device 10.

[0043] Overview of the processing functions of the information processing device 10 Figure 2 is a functional block diagram showing an overview of the processing functions of the information processing device 10. The information processing device 10 comprises a document acquisition unit 12, a descriptive information classification unit 14, a document analysis unit 16, an analysis result output unit 18, and a data storage unit 20. The document acquisition unit 12 acquires documents to be processed. The document acquisition unit 12 may accept input from image interpretation reports stored in a report server 220 or the like, or it may acquire documents entered by a physician using an input device 22. The document acquisition unit 12 may also receive documents from other devices such as a terminal device 230.

[0044] The document acquisition unit 12 may include a communication interface for receiving image interpretation reports from an external device such as a report server 220, or it may include a media interface for reading image interpretation reports from removable media such as a memory card. The document acquisition unit 12 may also read the documents to be processed from the data storage unit 20 within the information processing device 10. The document acquisition unit 12 may include a document acquisition program for automatically acquiring documents from the data storage unit 20 or an external device. Documents acquired via the document acquisition unit 12 are sent to the descriptive information classification unit 14. Data such as image interpretation reports input from an external device such as a report server 220 are stored in the data storage unit 20.

[0045] The descriptive information classification unit 14 performs a process of classifying the attributes of the information described in the input text (hereinafter referred to as "text description information") in fixed units. The fixed unit may be a sentence unit, a phrase unit, a word unit, or a character unit. Text in a radiology report often contains multiple sentences, and each sentence is often relatively short. A sentence unit is preferable for the fixed unit. "Attributes" include the concept of type. Attributes of text description information include, for example, one or more of the following: type of human body part, type of organ, type of disease or lesion, type of medical process, and presence or absence of disease or lesion. Types of medical processes may include, for example, findings, diagnosis, past comparison, and contact information. When handling radiology reports, it is desirable for the descriptive information classification unit 14 to perform a process of classifying the type of medical process and a process of classifying the type of organ. The classification results by the descriptive information classification unit 14 are stored in the classification result storage unit 50 of the data storage unit 20.

[0046] The text analysis unit 16 analyzes texts for each category based on the classification results from the descriptive information classification unit 14. The text analysis unit 16 includes a term extraction unit 30 and a relationship acquisition unit 32, and structures the text. The term extraction unit 30 extracts term expressions from the input text and determines the type (attribute) of each term. The relationship acquisition unit 32 acquires the relationships between the terms extracted by the term extraction unit 30. The relationship acquisition unit 32 determines whether there is a relationship between a subject and an object based on information about the subject terms, object terms, and the span between these terms in the text. The term extraction unit 30 and the relationship acquisition unit 32 may each be configured to use a pre-machine-learned predictive model to perform term extraction or relationship acquisition processing.

[0047] The analysis result output unit 18 outputs the analysis results from the document analysis unit 16. The analysis results include structured information indicating the structuring results. The analysis result output unit 18 converts the analysis results from the document analysis unit 16 into data in a format suitable for the output method and outputs it. Possible output methods include display, transmission, and storage. The analysis result output unit 18 may also include a display control unit 40 and a communication unit 42. The display control unit 40 controls the display of the display device 24. The display control unit 40 generates display data to be applied to the display device 24 and outputs the display data to the display device 24. As a result, the structured information of the analysis results is displayed on the display device 24. The display device 24 can also display information such as the input document (unstructured document) and the classification results from the descriptive information classification unit 14.

[0048] The communication unit 42 generates data for communication using the communication line 240 and transmits the data to an external device such as the report server 220. This allows the analysis results to be saved to the report server 220 or displayed on the display device 234 of the terminal device 230.

[0049] [Example Hardware Configuration of Information Processing Device 10] Figure 3 is a block diagram schematically showing an example of the hardware configuration of the information processing device 10. The information processing device 10 can be implemented by a computer system consisting of one or more computers. Here, we describe an example in which various functions of the information processing device 10 are realized by a single computer executing a program.

[0050] The information processing device 10 includes a processor 102, a computer-readable medium 104 which is a non-temporary tangible object, a communication interface 106, an input / output interface 108, and a bus 110.

[0051] The processor 102 includes a CPU (Central Processing Unit). The processor 102 may also include a GPU (Graphics Processing Unit). The processor 102 is connected to a computer-readable medium 104, a communication interface 106, and an input / output interface 108 via a bus 110. The processor 102 reads various programs and data stored in the computer-readable medium 104 and executes various processes. The term "program" includes the concept of a program module and includes instructions equivalent to a program.

[0052] The computer-readable medium 104 is a storage device that includes, for example, a main memory 122 and an auxiliary storage device, a storage 124. The storage 124 is configured using, for example, a hard disk drive (HDD), a solid state drive (SSD), an optical disk, a magneto-optical disk, or a semiconductor memory, or an appropriate combination thereof. Various programs and data are stored in the storage 124. The storage 124 includes the storage areas of the classification result storage unit 50 and the analysis result storage unit 52, and can function as a data storage unit 20 (see Figure 2).

[0053] Memory 122 is used as a workspace for the processor 102 and serves as a temporary storage unit for programs read from storage 124 and various types of data. When a program stored in storage 124 is loaded into memory 122, and the processor 102 executes the program's instructions, the processor 102 functions as a means of performing various processes defined by the program. Memory 122 stores programs such as the descriptive information classification program 140, the text analysis program 160, the display control program 180, and the communication control program 182, as well as various types of data, which are executed by the processor 102.

[0054] The descriptive information classification program 140 enables the processor 102 to perform processing functions as a descriptive information classification unit 14. The text analysis program 160 enables the processor 102 to perform processing functions as a text analysis unit 16. The text analysis program 160 includes a term extraction program 162 and a relationship acquisition program 164. The term extraction program 162 and the relationship acquisition program 164 enable the processor 102 to perform processing functions as a term extraction unit 30 and a relationship acquisition unit 32.

[0055] The display control program 180 causes the processor 102 to implement processing functions as the display control unit 40. The communication control program 182 causes the processor 102 to implement processing functions for communicating with external devices via the communication interface 106.

[0056] The communication interface 106 performs communication processing with external devices via wired or wireless connections and exchanges information with external devices. The information processing device 10 is connected to the communication line 240 via the communication interface 106 (see Figure 1), and can exchange data with devices such as the image storage server 210, the report server 220, and the terminal device 230. The communication interface 106 can also function as a document acquisition unit 12 (Figure 2) that accepts input of text data such as image interpretation reports, and as a communication unit 42 that outputs analysis results.

[0057] The input device 22 and the display device 24 are connected to the bus 110 via the input / output interface 108.

[0058] Examples of operation of the information processing device 10 Figures 4 to 7 illustrate an example of the operation of the information processing device 10. Figure 4 is a flowchart illustrating an example of an information processing method performed by the information processing device 10. In step ST1, the processor 102 receives input of a radiology report and acquires the radiology report. The processor 102 reads the radiology report, which is an unstructured document (free-form text) stored in the report server 220 or the like, and can display the retrieved radiology report on the display device 24. Figure 5 shows an example of text contained in the acquired radiology report. The text shown in Figure 5 is an example of medical text.

[0059] Next, in step ST2 of Figure 4, the processor 102 determines the type of medical process for each sentence contained in the acquired text. Here, an example is shown in which the medical processes are classified into three types: "findings," "diagnosis," and "other." "Other" is a category to which things that do not fall under either "findings" or "diagnosis" are classified; for example, past comparisons or contact information fall under "other." The processor 102 can determine the type of medical process on a sentence-by-sentence basis and assign a label to each sentence indicating the type of medical process. Through such labeling, each sentence is classified in terms of the type of medical process. Determining the type of medical process on a sentence-by-sentence basis is an example of classifying the attributes of text description information in a fixed unit. The processing in step ST2 is carried out in the description information classification unit 14.

[0060] Figure 6 shows an example of the results of determining the type of medical process on a sentence-by-sentence basis for the text of the radiology report shown in Figure 5. As shown in Figure 6, each sentence in the section "Compare with 10 / 15. CT <simultaneous thoracoabdomen>" is a description related to "other" and is classified as "other". Each sentence in the section "An irregular nodule measuring 1.5 cm in diameter is observed in left lung S6. It is accompanied by calcification inside. A bulla is observed in right lung S3." is a description related to "findings" and is classified as "findings". The sentence "A mass measuring 6 cm in diameter is observed in liver S3, showing early enhancement and washout, suggesting HCC" contains descriptions related to both "findings" and "diagnosis", so it may be labeled with both the "findings" label and the "diagnosis" label, or with only one of them. The sentence "A mass measuring 5 mm in diameter with early enhancement is also observed in liver S4, and it is persistent" is a description related to "findings" and is classified as "findings". Each sentence in the phrase "We suspect a hemangioma. There is a calcifying fibroid in the uterus." is a description of the "diagnosis" and is classified under "diagnosis." The phrase "Please follow up." is a description of contact information and is classified under "other." Bulla is a type of cystic lung disease. HCC is an abbreviation for Hepatocellular Carcinoma, which means liver cell carcinoma.

[0061] When considering the secondary use of radiology reports, the most important descriptive elements in the report are the findings and diagnosis, and it is desirable to structure the text related to the findings and diagnosis. Therefore, when the processor 102 determines the type of medical process on a sentence-by-sentence basis, it may label "findings" and "diagnosis" together as a single medical process type, "findings or diagnosis," without distinguishing between them. Furthermore, based on the determination (classification) result of the medical process type, the processor 102 may exclude text classified as "other" from the analysis processing by the text analysis unit 16, and include text related to findings or diagnosis in the analysis processing by the text analysis unit 16.

[0062] Furthermore, when the processor 102 displays the unstructured image interpretation report on the display device 24 or the like, it may display the determination of the type of medical process (classification result) in a way that allows for identification. In the example shown in Figure 6, the text in the parts classified as "Other" is displayed in gray, and the text in the parts classified as "Findings" is displayed in black, thus using different display colors. In addition to using different display colors to relatively reduce the visibility of the text parts that are excluded from the analysis process, various display methods can be adopted, such as enclosing each group of the same classification with a border, adding line breaks for each group of the same classification, displaying annotations indicating the classification result, or any appropriate combination thereof.

[0063] Next, in step ST3 of Figure 4, the processor 102 determines the type of organ for each sentence contained in the acquired text. However, the processor 102 may exclude sentences that fall under "other" in the determination of the type of medical process from the processing in step ST3. The processor 102 determines the type of organ on a sentence-by-sentence basis and classifies each sentence by the type of organ according to the determination result. Determining the type of organ on a sentence-by-sentence basis is an example of classifying the attributes of text description information in a fixed unit. The processing in step ST3 is carried out in the description information classification unit 14.

[0064] The upper part of Figure 7 shows examples of organ type determination results on a sentence-by-sentence basis. The three sentences, "An irregular nodule measuring 1.5 cm in diameter is observed in left lung S6. It is accompanied by calcification inside. A bulla is observed in right lung S3," describe the lungs, and each of these sentences is determined to be the organ of "lung." The three sentences, "A mass measuring 6 cm in diameter is observed in liver S3, exhibiting early staining and washout, suggesting HCC. A mass measuring 5 mm in diameter with early staining is also observed in liver S4, and it is persistent. A hemangioma is suspected," describe the liver, and each of these sentences is determined to be the organ of "liver." The sentence, "A calcifying fibroid is present in the uterus," describes the uterus, and this sentence is determined to be the organ of "uterus."

[0065] Furthermore, when the processor 102 displays the image interpretation report, which is an unstructured document, on the display device 24 or the like, it displays the results of the determination of the type of organ (classification results) in a way that allows for identification. Various display methods can be adopted to make the classification results identifiable, such as using different colors for the characters displayed for each classification, enclosing each group of the same classification with a border, adding a line break for each group of the same classification, or displaying text information indicating the classification results, or any appropriate combination thereof.

[0066] In the example shown in the upper part of Figure 7, the text of sentences classified as "lungs" is displayed in orange, for example; the text of sentences classified as "liver" is displayed in blue, for example; and the text of sentences classified as "uterus" is displayed in purple, for example; thus, the display colors of the text are different.

[0067] Next, in step ST4, the processor 102 analyzes the text for each classification based on the determination of the type of organ and performs term extraction. The processing in step ST4 is carried out by the term extraction unit 30 of the text analysis unit 16.

[0068] Furthermore, in step ST5, the processor 102 acquires the relationships between the extracted terms. The processing in step ST5 is carried out by the relationship acquisition unit 32 of the text analysis unit 16. The processor 102 receives input of the type of medical process, the type of organ, and the extracted term expressions and their types as text description information, determines whether there is a relationship between the terms and acquires the relationship. After excluding those where the type of medical process is "other," the processor 102 performs the relationship acquisition process within sets of sentences with the same type of organ (sets of sentences of the same classification).

[0069] The middle section of Figure 7 shows an example of term extraction and relationship acquisition performed on sentences classified as "liver." In the middle section of Figure 7, underlined terms in the sentences are terms that function as objects or subjects, and terms connected by dashed lines indicate relationships between them. Here, it can be seen that terms related to the lesion term "tumor," such as location, quantity, characteristics, and disease name, are described in the sentences. When displaying the sentences, the processor 102 may color-code the terms extracted by term extraction according to their type.

[0070] Although not shown in Figure 7, steps ST4 and ST5 are also performed for sentences classified as "lungs" and sentences classified as "uterus."

[0071] Next, in step ST6, the processor 102 displays the processing results of term extraction and relationship acquisition as structured results. The lower part of Figure 7 shows an example of the display of structured results for sentences classified as "liver".

[0072] [Example of the configuration of the descriptive information classification unit 14] The descriptive information classification unit 14 can be constructed using, for example, a natural language processing model called BERT (Bidirectional Encoder Representations from Transformers) as described in Non-Patent Document 1. Note that other machine learning models, such as Recurrent Neural Networks (RNNs) or Support Vector Machines (SVMs), may also be applied, not limited to BERT.

[0073] The descriptive information classification unit 14 uses text classification technology to determine what the input text is about in certain units, for example, at the sentence level. In the case of a text containing multiple sentences, the descriptive information classification unit 14 determines which classification each sentence belongs to based on the relationships between the sentences within the text.

[0074] When structuring sentences by organ, the descriptive information classification unit 14 uses sentence classification technology to determine the type of organ each sentence in the report belongs to. When inputting sentences into the descriptive information classification unit 14, surrounding sentences of the target sentence may also be input to determine the type of organ. Surrounding sentences may consist of N sentences before the target sentence and M sentences after the target sentence. N represents an integer of 1 or more, and M represents an integer of 0 or more. For example, in the sentence, "An irregular nodule measuring 1.5 cm in diameter was observed in the left lung S6. It was accompanied by calcification inside," if the target sentence is "It was accompanied by calcification inside," then by inputting the two sentences, including the preceding sentence, into the BERT model, the target sentence can be determined to be a sentence about the "lung."

[0075] [Example of the configuration of the term extraction unit 30] The term extraction unit 30 uses Named Entity Recognition (NER) technology to acquire term expressions and determine their type. The tasks of the term extraction unit 30 include a classification task that accepts a sequence of tokens as input and predicts a label for each token in the input, and a task that distinguishes the start and end of a Named Entity (NE) using the BIO method. The BIO method is a method that identifies the start and end of a Named Entity using tags: "Begin" to indicate the start of a Named Entity, "Inside" to indicate a sequence (internal) of Named Entities, and "other" to indicate something that is not a Named Entity (everything else).

[0076] The term extraction unit 30 can be configured, for example, using an encoder-decoder model. Figure 8 shows an example of a term extraction model 300 applied to the term extraction unit 30. The term extraction model 300 includes an encoder 302 and a decoder 304. The encoder 302 is configured using a neural network such as BERT or LSTM (Long Short Term Memory). The encoder 302 obtains a distributed representation from the input text. The decoder 304 predicts a probability distribution for a label from the distributed representation of each token output from the encoder 302 using a fully connected layer. NER handles NER labels, which are a combination of BIO labels and NE labels.

[0077] For example, the label "Section B" shown in Figure 8 is an NER label that combines the BIO label "Begin" and the NE label "Section," and the label "I Lesion" is an NER label that combines the BIO label "Inside" and the NE label "Lesion."

[0078] Figure 9 shows an example of the probability distribution of NER labels obtained by the decoder 304. The term extraction model 300 is trained to minimize the cross-entropy of the softmax function output of the decoder 304.

[0079] [Example of the configuration of the relationship acquisition unit 32] Figure 10 is a block diagram showing an example configuration of the relationship acquisition unit 32. The relationship acquisition model 320 applied to the relationship acquisition unit 32 has a structure in which a pooling layer 324, a coupling layer 326, and a linear layer 328 are added to the output of the BERT model 322.

[0080] The relationship acquisition unit 32 performs a binary classification of the input text, determining whether a relationship exists or not, based on the subject terms, object terms, and the span between these terms.

[0081] In the example text shown in Figure 10, "liver S3" is a subject term, "tumor" is an object term, and the string "7mm in diameter" between these terms is an example of a span. When text is input to the BERT model 322, the BERT model 322 provides output values ​​t1, t2, t3, etc. for each token. In Figure 10, the rounded rectangles "○○○" represent the output values ​​corresponding to each token. For example, in Figure 10, output value t1 represents the output value corresponding to the "Liver S3" token in the input token sequence. Output value t2 represents the output value corresponding to the span token, and output value t3 represents the output value corresponding to "tumor".

[0082] The output of the BERT model 322 is input to the pooling layer 324, where it is pooled. The output of the pooling layer 324 is input to the concatenation layer 326, where it is concatenated. The output of the concatenation layer 326 is linearly transformed by the linear layer 328. Based on the values ​​output from the linear layer 328, binarization is performed to indicate whether there is a relationship or not.

[0083] Various application examples The information processing device 10 described above is configured to perform term extraction and relationship acquisition by analyzing text for each category. However, the configuration may also include performing only term extraction as the processing content of the text analysis performed for each category. Furthermore, the classification is not limited to sentence units, but may also be at the phrase unit, etc. Below, variations of specific application examples will be described as Examples 1 to 7.

[0084] [Example 1] Example 1 is an example of using sentence-level organ recognition results to extract and structure terms for sentences related to each organ. Here, as shown in the upper part of Figure 11, an example of structuring a medical document that reads, "An irregular nodule measuring 1.5 cm in diameter is observed in the left lung S6. It is accompanied by calcification inside. A bulla is observed in the right lung S3. A mass measuring 6 cm in diameter is observed in the liver S3, exhibiting early staining and washout, suggesting HCC."

[0085] In this case, processor 102 performs organ recognition on each of the first to fourth sentences and obtains the result "lungs, lungs, lungs, liver". The notation "lungs, lungs, lungs, liver" indicates that the first to third sentences are each related to the lungs, and the fourth sentence is related to the liver. In the lower part of Figure 11, the organ recognition results for each sentence are shown by notations such as " / [lungs]".

[0086] Based on the classification results of each sentence, processor 102 performs term extraction on sentences for each organ, thereby obtaining structured information such as "Organ: Lung, Location: Left lung S6, Right lung S3, Quantity: 1.5 cm in diameter, Lesion (+): Nodule, Bulla, Characteristics (+): Irregular, Calcification" and "Organ: Liver, Location: Liver S3, Quantity: 6 cm in diameter, Lesion (+): Tumor, Characteristics (+): Early staining, Washout, Disease name: HCC". The type of lesion and disease name, such as nodule, bulla, or HCC, are examples of disease types.

[0087] [Example 2] Example 2 is an example of using the organ recognition results at the phrase level to perform term extraction and structuring for each organ phrase. Here, as shown in the upper part of Figure 13, an example of structuring a medical document that reads, "A cyst was found in liver S3, and no ascites was observed." Processor 102 performs organ recognition on each phrase of this sentence and obtains the classification results that "A cyst was found in liver S3" is liver and "No ascites was observed" is abdominal cavity. In the middle part of Figure 13, the organ recognition results for each phrase are shown by notations such as " / [liver]".

[0088] Processor 102 performs term extraction on each organ-specific phrase, thereby obtaining structured information such as "Organ: Liver, Location: Liver S3, Lesion (+): Cyst" and "Organ: Abdominal cavity, Lesion (-): Ascites," as shown in the lower part of Figure 13.

[0089] [Example 3] Example 3 is an example of using sentence-level organ recognition results to extract terms and obtain relationships for sentences related to each organ, and then structuring the text. Here, similar to Example 1, we will explain how to structure a medical document that reads: "An irregular nodule measuring 1.5 cm in diameter is observed in the left lung S6. It is accompanied by calcification inside. A bulla is observed in the right lung S3. A mass measuring 6 cm in diameter is observed in the liver S3, showing early staining and washout, suggesting HCC."

[0090] Processor 102 performs organ recognition on each sentence and obtains the result "lung, lung, lung, liver". Then, for each sentence for each organ, Processor 102 performs term extraction and relationship acquisition to obtain structured information as shown in Figure 14, such as "Organ: Lung, Location: Left lung S6, Quantity: 1.5 cm in diameter, Lesion (+): Nodule, Characteristics (+): Irregular, Calcification", "Organ: Lung, Location: Right lung S3, Lesion (+): Bulla", "Organ: Liver, Location: Liver S3, Quantity: 6 cm in diameter, Lesion (+): Tumor, Characteristics (+): Early staining, Washout, Disease name: HCC". Alternatively, if a model that can consider both sentences for each organ and the type of organ is used, the type of organ may be input to the model in addition to the sentences for each organ, and term extraction and relationship acquisition may be performed.

[0091] [Example 4] Example 4 is an example of using sentence-level organ recognition results to perform term extraction, relationship acquisition, and structuring for each organ-specific sentence. Here, as shown in the upper part of Figure 15, an example of structuring a medical document that reads, "Right coronary artery: calcification (+). Left coronary artery: stenosis (-). An irregular nodule measuring 1.5 cm in diameter is observed in left lung S6."

[0092] Processor 102 performs organ recognition on each sentence and obtains the classification result "heart, heart, lungs". In the lower part of Figure 15, the organ recognition results for each sentence are shown by notations such as " / [heart]".

[0093] Based on the results of organ recognition, processor 102 uses a machine learning model trained for the heart for sentences describing the heart, and a machine learning model trained for the lungs for sentences describing the lungs, to perform term extraction and relationship acquisition.

[0094] Figure 16 is a conceptual diagram of the text analysis unit 16 applied to Embodiment 4. The text analysis unit 16 shown in Figure 16 includes a machine learning model 410 trained for the heart, a machine learning model 420 trained for the lungs, and a model selection unit 430 for selecting the model to use. The machine learning model 410 trained for the heart is a trained model that has been trained using sentences describing the heart as training data, and includes a term extraction model 412 trained to perform the term extraction task and a relationship acquisition model 414 trained to perform the relationship acquisition task.

[0095] The machine learning model 420 trained for the lungs is a trained model that has been trained using sentences describing the lungs as training data, and includes a term extraction model 422 trained to perform the term extraction task and a relationship acquisition model 424 trained to perform the relationship acquisition task. Although not shown in Figure 16, the text analysis unit 16 may also include models trained for other organs, such as a machine learning model trained for the liver.

[0096] The model selection unit 430 selectively switches the machine learning model to be used based on the classification information obtained as a result of organ recognition. For example, if the classification information of the input text indicates the label "heart," the model selection unit 430 selects the machine learning model 410 trained for the heart as the model to be used for text analysis. For texts related to the heart, term extraction and relationship acquisition are performed using the machine learning model 410 trained for the heart, and structured information of texts related to the heart is obtained. Also, if the classification information of the input text indicates the label "lung," the model selection unit 430 selects the machine learning model 420 trained for the lung as the model to be used for text analysis. For texts related to the lung, term extraction and relationship acquisition are performed using the machine learning model 420 trained for the lung, and structured information of texts related to the lung is obtained.

[0097] Figure 17 shows an example of structured information obtained by text analysis using machine learning model 410, which was trained for the heart. Figure 18 shows an example of structured information obtained by text analysis using machine learning model 420, which was trained for the lungs.

[0098] As explained in Figure 16, by performing text analysis using an appropriate model according to the type of organ, more accurate analysis is possible. With this configuration, even when organs are written differently, highly accurate structuring can be performed, and structured information such as "Organ: Heart, Location: Right coronary artery, Lesion (+): Calcification", "Organ: Heart, Location: Left coronary artery, Lesion (-): Stenosis", and "Organ: Lung, Location: Left lung S6, Quantity: 1.5 cm in diameter, Lesion (+): Nodule, Characteristics (+): Irregular" can be obtained, as shown in Figures 17 and 18.

[0099] Furthermore, the configuration is not limited to preparing separate machine learning models for each organ. For example, for organs with unique descriptions, such as the heart, a machine learning model 410 trained specifically for the heart could be prepared, while for organs other than the heart, a machine learning model trained for organs other than the heart could be applied.

[0100] [Example 5] Example 5 is an example mentioned as an alternative form of Example 3, and it uses organ recognition results at the sentence level. When extracting terms and obtaining relationships for sentences related to each organ, the model uses the organ recognition results as input as auxiliary information.

[0101] Figure 19 is a conceptual diagram of the text analysis unit 16 applied to Example 5. The text analysis unit 16 shown in Figure 19 includes a machine learning model 450 that processes texts about various organs. The machine learning model 450 includes a term extraction model 452 that performs the task of term extraction and a relationship acquisition model 454 that performs the task of relationship acquisition. The machine learning model 450 is input with texts for each organ and organ type information (classification information) that indicates the organ recognition result of the text. By inputting the organ classification information as auxiliary information, the term extraction model 452 and the relationship acquisition model 454 can each perform highly accurate term extraction and relationship acquisition.

[0102] Furthermore, when training the machine learning model 450, which is commonly used for texts about various organs, the training data includes texts about various organs and information about the type of organ that classifies those texts.

[0103] Figure 20 is an explanatory diagram illustrating an example of processing by machine learning model 450. As shown in Figure 20, the machine learning model 450 is input with the following information: for example, "An irregular nodule measuring 1.5 cm in diameter is observed in left lung S6. It is accompanied by calcification. A bulla is present in right lung S3." and the auxiliary information "lung" indicating the type of organ. The machine learning model 450 then outputs the processing result. From the processing result of machine learning model 450, structured information is obtained such as "Organ: Lung, Location: Left lung S6, Quantity: 1.5 cm in diameter, Lesion (+): Nodule, Characteristics (+): Irregular, Calcification", and "Organ: Lung, Location: Right lung S3, Lesion (+): Bulla".

[0104] [Example 6] Up to this point, we have described an information processing device 10 that handles image interpretation reports. However, the technology disclosed herein can be applied to systems that handle various medical documents, not limited to image diagnostic reports such as image interpretation reports, but also including documents related to patient medical records. Examples of documents related to patient medical records include interim summaries and discharge summaries. Furthermore, the technology disclosed herein can be applied to processing documents in various fields, regardless of the type or use of the object, such as factory maintenance reports, inspection result reports for industrial products, inspection result reports for buildings, or various expert reports.

[0105] Example 6 describes an example of handling factory maintenance reports. Figure 21 shows an example of structuring a factory maintenance report by classifying its content into sentences and extracting terms for each classification. Here, as shown in the upper part of Figure 21, we show an example of structuring a maintenance report that reads, "An error was detected on the air flow meter. The current value was found to be 3.8mA. Upon checking with the manufacturer, it was suggested that there might be a problem with the circuit board or diagram. After investigation, it was found that the air flow meter was faulty. Therefore, it was replaced with a successor model."

[0106] The configuration of the information processing device for processing such text may be the same as the configuration of the information processing device 10 described in Figure 2, but instead of determining the type of organ, it is configured to determine whether each sentence contains "symptoms," "causes," or "countermeasures." In this case, the classification result of each sentence by the descriptive information classification unit 14 is "symptoms, symptoms, cause, cause, countermeasures," as shown in the middle of Figure 21.

[0107] The text analysis unit 16 then extracts terms from sentences categorized by content. This allows for the acquisition of structured information such as "Content: Symptom, Part: Air flow meter, Phenomenon: Error display, Measurement item: Current value, Measurement value: 3.8mA", "Content: Cause, Part: Circuit board, Diagram, Air flow meter, Phenomenon: Malfunction, Failure", and "Content: Countermeasure, Part: Successor model, Phenomenon: Update".

[0108] Regarding programs that operate computers: It is possible to record a program that enables a computer to implement some or all of the processing functions of the information processing device 10 described above on a computer-readable medium, such as an optical disk, magnetic disk, or semiconductor memory, which is a tangible, non-temporary information storage medium, and to provide the program through this information storage medium.

[0109] Alternatively, instead of providing programs by storing them on tangible, non-temporary computer-readable media, it is also possible to provide program signals as a download service using telecommunication lines such as the Internet.

[0110] Furthermore, some or all of the processing functions of the information processing device 10 may be implemented by cloud computing, and it is also possible to provide them as a SaaS (Software as a Service) service.

[0111] Regarding the hardware configuration of each processing unit: The hardware structure of the processing unit that executes various processes in the information processing device 10, such as the text acquisition unit 12, the descriptive information classification unit 14, the text analysis unit 16, the analysis result output unit 18, the term extraction unit 30, the relationship acquisition unit 32, and the display control unit 40, is, for example, various processors as shown below.

[0112] Various types of processors include CPUs, which are general-purpose processors that execute programs and function as various processing units; GPUs, which are processors specialized for image processing; Programmable Logic Devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing; and Dedicated Electrical Circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to perform particular processing.

[0113] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type. For example, a single processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU. Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is typical of computers such as clients and servers. Secondly, a configuration where a processor is used that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is typical of System-on-a-Chip (SoC) systems. Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0114] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit composed of circuit elements such as semiconductor devices.

[0115] Advantages of the Embodiment According to the information processing device 10 of this embodiment, analysis processing such as term extraction is performed for each sentence belonging to the same classification, so that the analysis of the sentence can be performed with high accuracy and the correct relationships between terms in the sentence can be obtained. As a result, highly accurate structured information can be obtained. The technology of this disclosure is particularly effective when analyzing a sentence that contains multiple sentences describing multiple things of different types (attributes).

[0116] "others" This disclosure is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the technical idea of ​​this disclosure. [Explanation of symbols]

[0117] 10 Information Processing Devices 12 Text acquisition part 14 Descriptive information classification section 16 Text analysis department 18 Analysis Result Output Unit 20 Data Storage Section 22 Input devices 24 Display device 30 Term extraction part 32 Relationship Acquisition Unit 40 Display Control Unit 42 Communications Department 50 Classification result storage section 52 Analysis result storage section 102 processors 104 Computer-readable media 106 Communication Interface 108 Input / Output Interfaces 110 Bus 122 memory 124 storage 140 Descriptive Information Classification Program 160 Text Analysis Programs 162 Term Extraction Program 164 Relationship Acquisition Program 180 Display Control Program 182 Communication Control Program 200 Medical Information Systems 202 Electronic Medical Record System 203 Inspection Order System 204 CT device 206 MRI machine 210 Image storage server 220 Report Server 230 Terminal devices 232 Input device 234 Display device 240 communication lines 300 Term Extraction Models 302 encoder 304 Decoder 320 Model 322 BERT model 324 pooling layers 326 Connected Layer 328 Linear Layer 410 Machine Learning Models 412 Term Extraction Models 414 Relationship Acquisition Model 420 Machine Learning Models 422 Term Extraction Models 424 Relationship Acquisition Model 430 Model Selection Section 450 Machine Learning Models 452 Term Extraction Models 454 Relationship Acquisition Model ST1-ST6 Steps in the Information Processing Method

Claims

1. one or more processors; one or more memories for storing instructions to be executed by the one or more processors; Equipped with The one or more processors: Get the text, classifying attributes of information described in the sentences into certain units of the sentences; Based on the results of the classification, the sentences are analyzed for each classification; outputting the analyzed results; Information processing device.

2. The predetermined unit is any one of a sentence unit, a phrase unit, a word unit, and a character unit. The information processing device according to claim 1 .

3. The text is a medical text.

3. The information processing device according to claim 1 or 2.

4. The classification items of the information include one or more of a body part, an organ, a type of disease, a type of medical procedure, and the presence or absence of a disease; The information processing device according to claim 1 .

5. The type of medical process includes at least one of a finding, a diagnosis, and a past comparison. The information processing device according to claim 4 .

6. The process of analyzing the sentences for each of the same classifications includes a process of extracting terms. The information processing device according to claim 1 .

7. The process of analyzing the sentences for each classification includes a process of extracting terms and a process of obtaining relationships between terms. The information processing device according to claim 1 .

8. The term extraction includes obtaining a term expression and determining a term type.

8. The information processing device according to claim 6 or 7.

9. The term extraction process is performed using a prediction model that has been machine-learned in advance. The information processing device according to claim 6 .

10. At least one of the term extraction process and the process of obtaining the relationships between the terms is performed using a prediction model that has been machine-learned in advance. The information processing device according to claim 7 .

11. Different prediction models are used depending on the results of the classification. The information processing device according to claim 10.

12. The sentences for each of the same categories and classification information for the same category are used as inputs to the prediction model. The information processing device according to claim 9 .

13. an input device that accepts input of the sentence; and a display device that displays the results of the analysis. The information processing device according to claim 1 .

14. The one or more processors: storing the results of said classification; When displaying the acquired sentences, a process is performed to display the classification results in an identifiable manner. The information processing device according to claim 1 .

15. 1. An information processing method executed by one or more processors, comprising: One or more processors Obtaining text and classifying attributes of information described in the sentences in a certain unit of the sentences; Analyzing the sentences for each classification based on the results of the classification; outputting the analyzed results; An information processing method including:

16. On the computer, The function to acquire text, A function of classifying attributes of information described in the sentences in a certain unit of the sentences; a function of analyzing the sentences for each classification based on the results of the classification; A function of outputting the analyzed results; A program to make this happen.

Images