Document creation device, document creation method, and program

Abstract

To provide a document creation apparatus, a document creation method, and a program, for creating a high-quality document regarding an image which has a large quantity of information to be described in the document.SOLUTION: A document creation apparatus includes at least one processor and at least one memory for storing an instruction to be executed by the at least one processor. The at least one processor acquires analyzes an input image to obtain content, specifies a base sentence including at least a part of the content from a database that stores a plurality of documents, corrects the at least a part of the base sentence, and creates a first document, which is related to the input image, with no excess or deficiency regarding the content.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a document creation device, a document creation method, and a program, and more particularly to a technique for creating documents relating to images. [Background technology]

[0002] Medical images are analyzed using CAD (Computer-Aided Diagnosis) processing, which uses a classifier trained by deep learning, etc., to extract the area, location, and volume of lesions contained in the medical image, and obtain these as the analysis results. The radiologist interprets the medical image on his or her own radiology terminal, referring to the medical image and the analysis results, and creates a radiology report.

[0003] Furthermore, Patent Documents 1 and 2 disclose methods for reducing the burden on an operator when creating a medical document related to a medical image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-149005 [Patent Document 2] Japanese Patent Application Laid-Open No. 2009-070201 Summary of the Invention [Problem to be solved by the invention]

[0005] As such, instead of the method of radiologists manually inputting all findings in radiology reports, in recent years there has been a growing need for methods to automatically extract candidate findings, such as automatic input of findings using templates or methods to extract candidate findings using machine learning.

[0006] However, with conventional technology, when it is desired to include a sentence with a large amount of information about a lesion in an image in an interpretation report, the automatically generated sentences are of poor quality, and it is not possible to provide a sentence that can be used.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a document creation device, a document creation method, and a program that can create high-quality documents related to images, even if the images have a large amount of information to be included in the document. [Means for solving the problem]

[0008] One aspect of a document creation device for achieving the above object is a document creation device comprising at least one processor and at least one memory for storing instructions to be executed by the at least one processor, wherein the at least one processor analyzes an input image to acquire content, identifies a base sentence including at least a portion of the content from a database in which a plurality of documents are stored, and modifies at least a portion of the base sentence to create a first document related to the input image, the first document being neither excessive nor deficient in content. According to this aspect, the first sentence related to the input image can be created based on a sentence including the content of the input image, so that a high-quality document related to the input image can be created even for an input image with a large amount of information to be included in the document.

[0009] Preferably, the at least one processor identifies a plurality of base sentences and combines and modifies the plurality of base sentences to create a content-complete first document.

[0010] Preferably, the at least one processor determines the extent of the base sentence correction based on the amount of the base sentence that is insufficient or excessive relative to the content. Also, the at least one processor preferably determines the extent of the base sentence correction on a sentence-by-sentence basis. This allows the extent of the base sentence correction to be appropriately determined.

[0011] It is preferable that the database stores sentences contained in multiple documents in a structured manner, so that the base sentence can be appropriately identified.

[0012] Preferably, the input image is a medical image of a first subject, and the first document is an interpretation report. This aspect is suitable for creating an interpretation report on a medical image.

[0013] The plurality of documents preferably includes radiology reports for medical images of a second subject different from the first subject, thereby enabling base sentences of different subjects containing similar content to be identified.

[0014] The content preferably includes at least one of the anatomical location, nature, size, and factuality of the lesion, which allows a first document to be created about the lesion.

[0015] Preferably, the at least one processor utilizes the organ recognition results and / or lesion recognition results of the input image to identify a base sentence, thereby generating a first document related to the organ and / or lesion.

[0016] The modifications preferably include at least one of additions, changes, and deletions, which allows the first document to be created from the base text.

[0017] Preferably, the at least one processor performs at least one of adding and modifying using at least one of a pre-prepared template and a trained model, thereby enabling at least one of adding and modifying to be performed appropriately.

[0018] Preferably, the at least one processor determines the location of the addition as the range of correction of the base sentence based on the type of content, thereby making it possible to appropriately determine the location of the addition.

[0019] Preferably, the at least one processor displays the correction range of the base sentence on the display, distinguishing it from the non-correction range, thereby allowing the user to recognize the correction range.

[0020] Preferably, the at least one processor allows a user to specify a region of interest in the input image and retrieves content based on the region of interest, thereby creating a first document about the user's desired region of interest.

[0021] Preferably, the at least one processor calculates a degree of match between the content and a plurality of documents and identifies a base sentence based on the degree of match, thereby enabling a base sentence with a relatively high degree of match to be identified.

[0022] The processor preferably uses all of the acquired content information and deletes unnecessary information from the base sentence to create a first document that is neither too much nor too little with the content. This allows the processor to appropriately create a first document that is neither too much nor too little with the content.

[0023] One aspect of a document creation method for achieving the above object is a document creation method comprising: an acquisition step of acquiring an input image; an analysis step of analyzing the input image to acquire content; an identification step of identifying a base sentence including at least a portion of the content from a database in which a plurality of documents are stored; and a creation step of modifying at least a portion of the base sentence to create a first document related to the input image, the first document being neither too much nor too little with respect to the content. According to this aspect, the first sentence related to the input image can be created based on a sentence including the content of the input image, so that a high-quality document related to the input image can be created even if the input image has a large amount of information to be included in the document.

[0024] One aspect of a program for achieving the above object is a program that causes a computer to execute the document creation method described above. This aspect may also include a computer-readable non-transitory storage medium on which this program is recorded. According to this aspect, a first sentence related to an input image can be created based on a sentence containing the content of the input image, so that a high-quality document related to the input image can be created even if the input image has a large amount of information to be included in the document. [Effects of the Invention]

[0025] According to the present invention, even if the image has a large amount of information to be written in the document, a high-quality document related to the image can be created. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a document creation system. [Figure 2] FIG. 2 is a block diagram showing the configuration of the document creation device. [Figure 3] FIG. 3 is a flowchart showing a document creation method using the document creation system. [Figure 4] FIG. 4 is a diagram showing an image to be interpreted and its findings. [Figure 5] FIG. 5 is a diagram showing an example of an input observation sentence and the structuring result thereof. [Figure 6] FIG. 6 is a diagram showing an image to be interpreted and its findings. [Figure 7] FIG. 7 is a diagram showing an example of the characteristics of a pulmonary nodule analyzed by the trained model. [Figure 8] FIG. 8 shows the results of characterization of a certain lesion and the findings. [Figure 9] Figure 9 shows an example of a finding statement from a past radiology report and its structuring results. [Figure 10] Figure 10 shows an example of a finding statement from a past radiology report and its structuring results. DETAILED DESCRIPTION OF THE INVENTION

[0027] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] <Conventional issues> The following describes the conventional problems that arise when creating an image interpretation report by including text with a large amount of information. An example of text with a large amount of information is shown below. "A nodule measuring 3.2 cm in diameter was found in S1 of the right lung. It was solid with a clear border. The border was slightly irregular, with some spicules present. Inside, a thin-walled cavity with an irregular border was found. No calcification or bronchial radiolucency was observed. Pleural indentation was also present." When providing such information-rich text using a template, the system cannot provide it unless there is a template corresponding to the text in its own collection. Also, when creating a template based on a past radiology report, the system cannot provide a template unless there is an identical template available, i.e., one that can be reused exactly as is.

[0029] For example, if the only templates available were "L, M, and N not found" and "L and M found," it was not possible to create the sentence "L, M, and N found." Similarly, if the only template available was "Cavities and calcifications found inside," it was not possible to create the sentence "Cavities, calcifications, and bronchial radiographs found inside."

[0030] On the other hand, when providing text with a large amount of information using a trained model method such as a neural network, it is difficult to generate text that mentions a large number of types of characteristic information, and there is a risk that the findings desired by the radiologist will not be generated, and instead findings that the radiologist does not want will be provided. In other words, when providing text using a trained model method, appropriateness and fluency become issues. Here, appropriateness refers to the absence of excessive or omissions, and fluency refers to the absence of unnatural wording as a natural language expression.

[0031] Thus, it has been difficult to provide text with a large amount of information in the past.

[0032] <Document creation system> The document creation system according to this embodiment is a system that captures medical images of a subject (patient) and creates an interpretation report on the captured medical images.

[0033] Fig. 1 is an overall configuration diagram of a document creation system 10. As shown in Fig. 1, the document creation system 10 is configured to include medical image inspection equipment 12, a medical image database 14, a document creation device 16, an image interpretation report database 18, and a user terminal 20.

[0034] The medical image inspection equipment 12, medical image database 14, document creation device 16, radiology report database 18, and user terminal 20 are connected to each other via a network 22 so as to be able to send and receive data. The network 22 includes a wired or wireless LAN (Local Area Network) that connects various devices within a medical institution. The network 22 may also include a WAN (Wide Area Network) that connects the LANs of multiple medical institutions.

[0035] The medical image inspection equipment 12 is an imaging device that captures an image of an inspection target part of a subject and generates a medical image. Examples of the medical image inspection equipment 12 include an X-ray imaging device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasound device, and a CR (Computed Radiography) device using a flat X-ray detector.

[0036] The medical image database 14 is a database that manages medical images captured by the medical image inspection equipment 12. The medical image database 14 is implemented by a computer equipped with a large-capacity storage device for saving medical images. Software that provides the functions of a database management system is installed in the computer.

[0037] The format of medical images can be in accordance with the Dicom (Digital Imaging and Communications in Medicine) standard. Supplementary information (Dicom tag information) defined in the Dicom standard may be added to medical images. Note that the term "image" in this specification includes not only the image itself, such as a photograph, but also image data, which is a signal representing an image.

[0038] The document creation device 16 analyzes an input image to acquire content, identifies a base sentence containing at least a portion of the content from a database storing multiple documents, and modifies at least a portion of the base sentence to create a first document related to the input image that is neither excessive nor insufficient compared to the content. Here, the input image is a current image of a subject acquired from a medical image inspection device 12 or a medical image database 14. The database storing multiple documents is an image interpretation report database 18, and the first document is an image interpretation report related to the current image of the subject.

[0039] A personal computer or a workstation (an example of a "computer") can be applied to the document creation device 16. Fig. 2 is a block diagram showing the configuration of the document creation device 16. As shown in Fig. 2, the document creation device 16 includes a processor 16A, a memory 16B, and a communication interface 16C.

[0040] The processor 16A executes instructions stored in the memory 16B. The hardware structure of the processor 16A is composed of various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various functional units, a GPU (Graphics Processing Unit), which is a processor specialized for image processing, a PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for executing specific processing.

[0041] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Also, multiple functional units may be configured with a single processor. Examples of multiple functional units configured with a single processor include, first, a configuration in which a single processor is configured with a combination of one or more CPUs and software, as typified by a client or server computer, and this processor operates as multiple functional units. Second, a configuration in which a processor is used to realize the functions of an entire system including multiple functional units on a single IC (Integrated Circuit) chip, as typified by an SoC (System On Chip). In this way, the various functional units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0042] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0043] The memory 16B stores instructions to be executed by the processor 16A. The memory 16B includes a RAM (Random Access Memory) and a ROM (Read Only Memory), both not shown. The processor 16A uses the RAM as a work area, executes software using various programs and parameters, including a document creation program (described later) stored in the ROM, and also executes various processes of the document creation device 16 by using the parameters stored in the ROM, etc.

[0044] The communication interface 16C controls communication with the medical image inspection equipment 12, the medical image database 14, the image interpretation report database 18, and the user terminal 20 via the network 22 in accordance with a predetermined protocol.

[0045] The document creation device 16 may be a cloud server that can be accessed from multiple medical institutions via the Internet. The process performed by the document creation device 16 may be a cloud service that is charged or fixed fee.

[0046] Returning to the explanation of FIG. 1, the radiology report database 18 is a database that manages past radiology reports generated by users such as radiologists (radiologists) on the user terminal 20. The radiology report includes a finding sentence. The finding sentence is not limited to a sentence separated by a period or a punctuation mark, and may be a group of words. The finding sentence may be a single sentence or a sentence made up of multiple sentences. The radiology report database 18 may store the finding sentences of past radiology reports in a structured manner, or may store the finding sentences without structuring them.

[0047] The image interpretation report database 18 is implemented by a computer equipped with a large-capacity storage device for storing image interpretation reports. Software that provides the functions of a database management system is installed in the computer. The medical image database 14 and the image interpretation report database 18 may be configured on a single computer.

[0048] The user terminal 20 is a terminal device used by the radiologist to view and edit the radiology report. The user terminal 20 is, for example, a personal computer. The user terminal 20 may be a workstation or a tablet terminal. The user terminal 20 includes an input device 20A and a display 20B. The radiologist uses the input device 20A to input instructions to the document creation system 10. The user terminal 20 also displays the medical image and the radiology report on the display 20B. The radiologist then interprets the medical image displayed on the display 20B and inputs the findings, which are the interpretation results, using the input device 20A.

[0049] <Document Creation Method> 3 is a flowchart showing a document creation method using the document creation system 10. The document creation method is realized by the processor 16A executing a document creation program stored in the memory 16B. The document creation program may be provided by a computer-readable non-transitory storage medium or via the Internet.

[0050] Steps ST1 and ST2 are processes performed in advance to store the image interpretation report including the findings in the image interpretation report database 18.

[0051] In step ST1, the radiologist specifies an image to be interpreted using the input device 20A of the user terminal 20. The document creation device 16 reads out the image to be interpreted specified by the radiologist from among the medical images captured by the medical image inspection equipment 12 and stored in the medical image database 14, and displays it on the display 20B of the user terminal 20. Figure 4 is a diagram showing the image to be interpreted and its findings. F4A shown in Figure 4 indicates the image to be interpreted I1 specified in step ST1.

[0052] Next, document creation device 16 extracts a lesion area (an example of a "region of interest") from the image to be interpreted displayed on display 20B. The lesion area may be extracted manually by the radiologist using input device 20A of user terminal 20. Here, it is assumed that lesion area L1 of image to be interpreted I1 shown in F4A has been extracted.

[0053] In step ST2, the radiologist uses the input device 20A to input a finding for the lesion area L1 of the image I1 to be interpreted, which was displayed on the display 20B in step ST1. Here, it is assumed that the finding FS1 shown in F4B has been saved. The finding FS1 is written as follows: "There is a 3 cm mass in S3 of the left lung. The margins are unclear and the shape is irregular. Spicules are observed in some areas, and it is lobulated. A cavity is observed inside, and the cavity wall is thickened. It is in contact with the pleura." Thereafter, the document creation device 16 stores the image interpretation report including the inputted findings in the image interpretation report database 18.

[0054] By repeating the processes of steps ST1 and ST2, a plurality of radiology reports including the findings are stored in the radiology report database 18. It is preferable that the processes of steps ST11 and ST2 are repeated for different subjects.

[0055] In this embodiment, the document creation device 16 structures the finding text and stores the structured finding text in the radiology report database 18. FIG. 5 is a diagram showing an example of an input finding text and the structuring result thereof. F5A shown in FIG. 5 is the input finding text. Here, the finding text states, "A 3 cm solid mass was observed in S4 of the right lung. The boundary was unclear, the margin was lobulated, and it exhibited spicules. A pleural indentation was also observed. No calcification, cavities, or bronchial radiolucency were observed internally."

[0056] F5B in Figure 5 is the structuring result of the findings in F5A using natural language processing. As shown in F5B, the structuring result shows that the findings written in free text are structured as information for each lesion.

[0057] Here, the "organ" category is "lung," the "location" category is "right lung" and "S4," the "lesion" category is "mass," the "factuality of lesion" category is "present," the "disease name" and "factuality of disease name" categories are left blank, the "size" category is "longest diameter 3 cm," and the "characteristics" category is "solid," "unclear border," "lobulated (+)," "pleural indentation (+)," "calcification (-)," "cavitation (-)," and "bronchial radiography (-)" are classified with a specific relationship. Note that factuality means the possibility of existence.

[0058] In this way, a plurality of finding statements are structured and stored in the image interpretation report database 18.

[0059] Step ST3 and subsequent steps are a process of automatically generating a new finding statement based on the finding statement stored in the image interpretation report database 18. In step ST3 (an example of an "acquisition process"), the image interpretation physician uses the input device 20A of the user terminal 20 to specify an image of the subject to be interpreted from among the medical images captured by the medical image inspection equipment 12 and stored in the medical image database 14.

[0060] Step ST3 is a scene of interpretation of a medical image of a subject (an example of a "first subject") different from the subject (an example of a "second subject") of step ST1, and the image to be interpreted designated in step ST3 is an image different from the image to be interpreted I1 designated in step ST1. FIG. 6 is a diagram showing the image to be interpreted and its findings. F6A shown in FIG. 6 indicates the image to be interpreted I2 designated in step ST3.

[0061] The document creation device 16 reads the image I2 to be interpreted from the medical image database 14 and displays it on the display 20B of the user terminal 20. The document creation device 16 also analyzes the image I2 to be interpreted and extracts a lesion area from the image I2 to be interpreted. The lesion area may be extracted manually by the radiologist using the input device 20A of the user terminal 20. The document creation device 16 also acquires the anatomical location and size of the lesion area. Here, it is assumed that the lesion area L2 shown in F5A has been extracted from the image I2 to be interpreted.

[0062] In step ST4 (an example of an "analysis step"), the document creation device 16 performs a characteristic analysis on the image I2 to be interpreted, designated in step ST3, by CAD processing (an example of "analysis"), and acquires a characteristic analysis result (image feature). The CAD processing uses a trained model to which a convolutional neural network (CNN) is applied. Here, the document creation device 16 is capable of CAD processing to detect pulmonary nodules and CAD processing to detect liver tumors. It is assumed that the lesion area L2 designated in step ST3 is a pulmonary nodule.

[0063] In step ST5 (an example of a "specifying step"), the document creation device 16 searches the structured data of multiple findings stored in the radiology report database 18 based on the content of the image I2 to be interpreted, and extracts past findings that serve as a base (an example of a "base sentence"). The base findings are extracted from the findings stored in the radiology report database 18 that have a relatively high degree of match with the content of the image I2 to be interpreted. The content of the image I2 to be interpreted includes at least one of the anatomical location, characteristics, size, and factuality of the lesion. Here, the content of the image I2 to be interpreted is the anatomical location and size of the lesion area acquired in step ST3 and the result of the characteristic analysis in step ST4. The document creation device 16 may extract findings by performing a brute force search of all pre-structured findings stored in the radiology report database 18.

[0064] The base finding sentence may be extracted by using the organ recognition result for the image to be interpreted. For example, if the lung is the object of interpretation, the base finding sentence may be searched for in the interpretation report for the lung. The finding sentence may be extracted by using the lesion recognition result for the image to be interpreted. For example, if the object of interpretation is a nodule, inflammation, pleural effusion, etc., the base finding sentence may be searched for in the corresponding interpretation report.

[0065] Here, since it has been determined in step ST4 that the lesion area L2 is a pulmonary nodule, the range in which the document creation device 16 compares the structured data of a plurality of finding sentences is limited to finding sentences related to pulmonary nodules.

[0066] The radiologist may manually extract the base finding sentences. In this case, the document creation device 16 displays a list of multiple finding sentences on the display 20B, and the radiologist can select the finding sentence with the highest degree of match from the list using the input device 20A.

[0067] Here, it is assumed that the observation sentence FS1 shown in F4B of FIG. 4 is extracted as a base.

[0068] In step ST6, the document creation device 16 calculates the degree of agreement and the degree of difference between the content of the image to be interpreted I2 and the finding sentence FS1 extracted as the base. Here, it is assumed that the following parts A1, A2, A3, and A5 of the finding sentence FS1 are calculated to agree with the content of the image to be interpreted I2, and part A4 is calculated to differ from the content of the image to be interpreted I2.

[0069] A1: "There is a 3cm mass in the S3 region of the left lung." A2 "The margins are unclear and the shape is irregular." A3: "Some spicules are observed, and the lesion appears lobulated." A4: "There is a cavity inside, and the cavity wall is thickened." A5 "It is in contact with the pleura."

[0070] That is, the document creation device 16 has found the following differences (examples of "excess or deficiency with respect to the content"). Difference: "Inside: No cavity, Bronchial radiography: Present" The degree of agreement and the degree of difference between the content of the image I2 to be interpreted and the findings extracted as the base may be manually extracted by the radiologist.

[0071] In step ST7 (an example of a "creation process"), the document creation device 16 generates a finding statement regarding the lesion area L2 of the image I2 to be interpreted based on the analysis results of step ST4 and the degrees of similarity and difference of step ST6, and displays it on the display 20B. F6B shown in FIG. 6 is the finding statement FS2 generated and displayed in step ST7. The finding statement FS2 is written as follows: "There is a 3 cm mass in S3 of the left lung. The margins are unclear and the shape is irregular. Spicules are observed in some areas and it is lobulated. There is a cavity inside. It is △△. It is in contact with the pleura."

[0072] Here, in the observation statement FS2, the following B1, B2, B3, and B5 do not require correction, and B4 does require correction.

[0073] B1: "There is a 3cm mass in the S3 region of the left lung." B2: The margins are unclear and the shape is irregular. B3 "Some spicules are observed, and the lesion appears lobulated." B4 "There is a cavity inside that is ○○. It is △△." B5: "It's touching the pleura."

[0074] That is, the portions B1, B2, B3, and B5 of the finding statement FS2 are common to the portions A1, A2, A3, and A5 of the finding statement FS1. The document creation device 16 may display on the display 20B portions that require correction (an example of the "scope of correction"), distinguishing them from portions that do not require correction (an example of the "outside the scope of correction"). The document creation device 16 may automatically correct the portions that require correction, or may display at least one or more candidates for correction.

[0075] The correction includes at least one of adding, changing, and deleting. The correction range may be determined based on whether the content of the image to be interpreted is excessive or insufficient. The correction range may also be determined on a sentence-by-sentence basis.

[0076] The additions and changes may be made using sentences created using a prepared template, or may be made using sentences using a trained model. The sentences to be added or changed are only a part of the findings and contain little information, so they can be corrected using the template method or the trained model method.

[0077] Here, the radiologist uses the input device 20A to select the "〇〇, △△" part of B4 and input the appropriate term and sentence, thereby completing the finding sentence for the image I2 to be interpreted.

[0078] The document creation device 16 may automatically perform a process of adding the following content to the difference extracted in step ST6, thereby creating a new finding statement by correcting the finding statement FS1.

[0079] Additional information: "No cavity was observed within the above nodule, and bronchial radiolucency was observed."

[0080] The findings thus prepared are as follows:

[0081] "There is a 3cm mass in the S3 region of the left lung. The margins are unclear and the shape is irregular. Some spicules are present, and the mass is lobulated. No cavity is found inside, and bronchial radiolucency is present. The mass is in contact with the pleura."

[0082] In other words, this finding statement has the same parts A1, A2, A3, and A5 as the finding statement FS1, but part A4 has been changed to "No cavity was observed inside, and a bronchial radiolucency image was observed."

[0083] In this way, the document creation device 16 uses all the content acquired from the image I2 to be interpreted and deletes unnecessary information from the finding statement FS1 to create a finding statement that is neither excessive nor insufficient with respect to the content. Here, the finding statements FS1 of different subjects with the same disease are effectively utilized.

[0084] In step ST7, multiple finding sentences that require correction may be displayed on display 20B, allowing the radiologist to select one. When multiple finding sentences that require correction are displayed, the range of correction is displayed differently for each. The following is an example in which the parts that require correction are displayed differently for candidates C1, C2, and C3. The parts that require correction in candidate C1 are XX, in candidate C2 are XX and △△, and in candidate C3 are XX, △△, and □□.

[0085] Candidate C1: "A 32mm solid nodule is present in the left lower lobe, S6 / 10. The margin is XX, but lobulated areas are present." Candidate C2: "I have a 32mm ○○ in the left lower lobe S6 / 10. The margins are △△, but there are lobulated areas." Candidate C3: "I see a 32mm ○○ in the left lower lobe S6 / 10. The margin is △△, but I see the □□ part."

[0086] The radiologist can select one of the candidates C1, C2, and C3 and correct any parts that need to be corrected to complete the findings.

[0087] As described above, the document creation method according to the first embodiment makes it possible to create a first document relating to an input image that contains neither too much nor too little content acquired from the input image. Therefore, even if the image has a large amount of information to be written in the radiology report, a high-quality document relating to the image can be created.

[0088] If the text to be corrected was created based on the radiology report database 18 in which structured findings are stored, the text may be corrected using the structured findings and the natural language findings that were actually created.

[0089] On the other hand, if the sentence to be corrected was created before the introduction of the radiology report database 18 and is not stored in the radiology report database 18, the sentence to be corrected may be searched for on a word or synonym basis, and the inconsistencies may be extracted and displayed on the display 20B.

[0090] <Second embodiment> Figure 7 shows an example of the properties (an example of "content") of a pulmonary nodule analyzed in CAD processing using a trained model. As shown in Figure 7, the properties include six items: "absorption value," "boundary," "shape," "marginal properties," "internal properties," and "relationship with surrounding tissue."

[0091] "Absorption values" are classified as either "solid," "partially solid," or "ground glass."

[0092] The "boundary" is classified into one of "clear," "slightly clear," "neither clear nor unclear," "slightly unclear," and "clear."

[0093] The "shape" is classified into either "near circular" or "irregular".

[0094] The "marginal characteristics" are analyzed for each of "smoothness," "lobulated," "serrated," "spicules," and "linearity." In addition, "spicules" are also analyzed for "circumferential presence" and "partial presence."

[0095] The "internal characteristics" are analyzed for each of "bronchial radiography," "calcification," "cavity," and "fat." Note that "calcification" is also analyzed for size, location, and aggregation, and "cavity" is also analyzed for specularity, thickness, and smoothness.

[0096] The "relationship with surrounding tissues" is analyzed for "pleural contact," "pleural indentation," "pleural thickening," "vascular / bronchial compression," and "vascular / bronchial convergence." If "pleural contact" is present, it is classified as either "extensive" or "only partial."

[0097] Figure 8 shows the characterization results for a certain lesion and the accompanying findings. F8A in Figure 8 shows the characterization results in bold. As shown in F8A, the "absorption value" is "solid type," the "boundary" is "clear," and the "shape" is "near-circular."

[0098] Furthermore, the "marginal characteristics" are "slightly irregular," "lobulated-," "serrated-," "spicules+" and "present in some areas," and "straight-." The "internal characteristics" are "bronchial radiolucency-," "calcification-," "cavity+" and "air-floor image-," "neither (thickness)," "irregular," and "fat-." Furthermore, the "relationship with surrounding tissues" are "pleural contact-," "pleural indentation+," "pleural thickening-," "neither (compression)," and "neither (convergence)."

[0099] F8B in Figure 8 is a finding statement FS3 that should be prepared by the radiologist in response to the characteristic analysis results shown in F8A. The finding statement FS3 is written as follows: "A nodule with a long diameter of 3.2 cm is observed in S1 of the right lung. It is solid with a clear border. The border is slightly irregular, with some spicules. A cavity with an irregular border is observed inside. No calcification or bronchial radiolucency is observed. Pleural indentation is observed."

[0100] In this way, the finding statement FS3 does not include all of the results of the property analysis shown in F8A. F8C shown in Figure 8 indicates in bold the properties that should be included in the finding statement from the property analysis results shown in F8A.

[0101] As shown in F8C, the characteristics to be described in the findings statement include "solid" in "absorption value," "clear" in "boundary," and "near-circular" in "shape." In addition, the characteristics to be described in the findings statement include "slightly irregular," "spicules+," and "present in some areas" in "marginal characteristics," "calcification-," "cavities+," and "irregularity" in "internal characteristics," and "pleural indentation+" in "relationship with surrounding tissues."

[0102] The characteristics to be described in the findings may be determined using a trained model, may be prioritized based on rules, or may be determined based on past statistics.

[0103] In this way, the document creation device 16 uses all of the information from the characteristic analysis results that is to be described in the finding statement, and deletes unnecessary information from the finding statement, thereby creating a finding statement that is neither excessive nor insufficient compared to the characteristic analysis results.

[0104] Figure 9 shows an example of a finding statement from a past radiology report stored in the radiology report database 18, and the structured result thereof. F9A shown in Figure 9 is a finding statement FS4, and F9B shown in Figure 9 is the structured result of finding statement FS4. Finding statement FS4 states, "A 3.2 cm nodule was observed in S1 of the right lung. It is solid with a clear border. The border is slightly irregular, and spicules are observed all around. A cavity with an irregular border was observed inside. No calcification or bronchial radiolucency was observed. Pleural indentation was observed."

[0105] FIG. 10 shows an example of a finding statement from a past radiology report stored in the radiology report database 18, and the structuring result thereof. F10A shown in FIG. 10 is a finding statement FS5, and F10B shown in FIG. 10 is the structuring result of finding statement FS5. Finding statement FS5 states, "A solid nodule measuring 2 cm in major axis was observed in S8 of the left lung. The boundary is somewhat unclear. The margin is somewhat irregular, and spicules are observed in some areas."

[0106] Comparing the characteristics to be described as finding statement FS3 with finding statement FS4 and finding statement FS5, it is found to be quite similar to finding statement FS4. In finding statement FS4, "spicules" are described as "all around," so it is necessary to find a finding statement that describes that "spicules" are "present in part," and finding statement FS5 corresponds to this.

[0107] Therefore, by using the finding sentence FS4 as a base and modifying only the sentence "spicules," that is, by rewriting it using the finding sentence FS5, it is possible to describe all the desired characteristics.

[0108] In this way, multiple base finding sentences may be identified and corrections may be made by combining the identified multiple base finding sentences (an example of "multiple base sentences"). A finding sentence containing content that is missing from the first base finding sentence may be identified as the second base finding sentence. The desired finding sentence can be generated by deleting the excess parts of the first finding sentence and extracting the missing parts from the second finding sentence.

[0109] This example shows how to modify some of the text, but in the case of deletion, it is sufficient to simply delete the unnecessary text. When adding information, the position at which to add it is important. The position to add it may be determined based on the category of the content to be added (absorption value, boundary, shape, marginal characteristics, internal characteristics, relationship with surrounding tissue). For example, in the case of "pleural indentation," it is customary to write "relationship with surrounding tissue" last, so it should be added at the end of the findings.

[0110] <Other> The document creation according to this embodiment can be applied to documents other than medical documents and documents related to medical images. For example, when creating a diagnostic document for social infrastructure facilities such as transportation, electricity, gas, and water, an input image can be analyzed to acquire content, a base sentence containing at least a portion of the content can be identified from a database storing multiple documents, and at least a portion of the base sentence can be modified to create a diagnostic document related to the input image that is neither too much nor too little in terms of content.

[0111] The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention. [Explanation of symbols]

[0112] 10. Document Creation System 12...Medical imaging inspection equipment 14...Medical image database 16...Document creation device 16A...Processor 16B...Memory 16C...Communication interface 18...Radiography report database 20...User terminal 20A...input device 20B…Multiple displays 20B...Display 22…Network FS1...Observations FS2…Observations FS3...Observations FS4…Observations FS5…Observations I1: Image to be interpreted I2: Image to be interpreted L1: Lesion area L2: Lesion area ST1~ST7...Steps for document creation

Claims

1. at least one processor; at least one memory storing instructions for execution by said at least one processor; Equipped with The at least one processor Analyze the input image to obtain the content, Identifying a base document that includes at least a portion of the content from a database in which a plurality of documents are stored; modifying at least a portion of the base text to create a first document related to the input image, the first document being equivalent to or incomplete with the content; determining a range of correction of the base sentence based on an excess or deficiency of the base sentence relative to the content; Document creation device.

2. The at least one processor Identifying a plurality of said base sentences; combining and modifying the plurality of base sentences; The document creation device according to claim 1 .

3. The at least one processor determining a range of correction of the base sentence on a sentence-by-sentence basis; 3. The document creation device according to claim 1.

4. The database stores sentences contained in the plurality of documents in a structured manner. The document creation device according to any one of claims 1 to 3.

5. the input image is a medical image of a first subject, the first document is an image interpretation report; The document creation device according to any one of claims 1 to 4.

6. the plurality of documents include an interpretation report for a medical image of a second subject different from the first subject; The document creation device according to claim 5.

7. the content includes at least one of an anatomical location, a nature, a size, and a factuality of the lesion; 7. The document creation device according to claim 5 or 6.

8. The at least one processor identifying the base sentence using at least one of an organ recognition result and a lesion recognition result of the input image; The document creation device according to any one of claims 5 to 7.

9. The modification includes at least one of addition, modification, and deletion. The document creation device according to any one of claims 1 to 8.

10. The at least one processor Adding and / or modifying the data using at least one of a pre-prepared template and a trained model; The document creation device according to claim 9.

11. The at least one processor determining a position to add the content based on the type of content as a range to be corrected in the base sentence; 11. The document creation device according to claim 9 or 10.

12. The at least one processor displaying the correction range of the base sentence on a display, distinguishing it from the range other than the correction range; The document creation device according to any one of claims 1 to 11.

13. The at least one processor prompting a user to specify a region of interest in the input image; obtaining the content based on the region of interest; The document creation device according to any one of claims 1 to 12.

14. The at least one processor Calculating a degree of match between the content and the plurality of documents; identifying the base sentence based on the degree of match; The document creation device according to any one of claims 1 to 13.

15. the processor uses all of the information of the acquired content and deletes unnecessary information from the base sentence to create the first document so that it is neither too much nor too little compared to the content; The document creation device according to any one of claims 1 to 14.

16. At least one processor: an acquisition step of acquiring an input image; an analysis step of analyzing the input image to obtain content; an identifying step of identifying a base document including at least a portion of the content from a database in which a plurality of documents are stored; a creating step of modifying at least a portion of the base text to create a first document related to the input image, the first document being just the same as the content; Run The creating step determines a range of correction of the base sentence based on an excess or deficiency of the base sentence relative to the content. How to create documentation.

17. A program that causes a computer to execute the document creation method according to claim 16.

Images