Document creation support device, document creation support method, and document creation support program

Abstract

This document creation assistance device extracts a region having one or more predetermined physical features from a medical image, and generates finding text using a disease name associated with a physical feature present in at least one region of the extracted regions.

Description

Technical Field

[0001] The present disclosure relates to a document creation support device, a document creation support method, and a document creation support program.

Background Art

[0002] International Publication No. 2020 / 209382 discloses a technique for detecting a plurality of findings representing features related to abnormal shadows included in medical images, identifying at least one finding to be used for creating a radiology report from the detected findings, and creating a radiology report using the identified findings.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In the technique described in International Publication No. 2020 / 209382, various processes are executed, such as a process of detecting abnormal shadows from medical images, a process of detecting a plurality of findings representing features related to the abnormal shadows, and a process of identifying at least one finding to be used for generating a radiology report from the plurality of findings. That is, the technique described in International Publication No. 2020 / 209382 cannot easily support the generation of a radiology report.

[0004] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a document creation support device, a document creation support method, and a document creation support program that can easily support the generation of a radiology report.

Means for Solving the Problems

[0005] The document creation support device of the present disclosure is a document creation support device including at least one processor, and the processor extracts an area having one or more preset physical features from a medical image, and generates a finding sentence using a disease name associated with the physical features of at least one of the extracted areas.

[0006] Furthermore, the document creation support device described herein may have a processor that controls the display of information representing the extracted region, accepts information representing a selected region from among the extracted regions, and generates a findings statement using a disease name associated with the physical characteristics of the selected region.

[0007] Furthermore, the document creation support device of this disclosure may have a processor that references data in which physical characteristics and disease names are associated, and generates a report using the disease names associated with the physical characteristics.

[0008] Furthermore, the document creation support device of this disclosure may have a processor that extracts from a medical image an area within a range of pixel values ​​set as an area having physical characteristics.

[0009] Furthermore, the document creation support device of this disclosure may have a processor that extracts regions with a shape having set characteristics from a medical image as regions having physical characteristics.

[0010] Furthermore, the document creation support device described herein may have physical characteristics set for each organ.

[0011] Furthermore, the document creation support device described herein may have multiple disease names associated with its physical characteristics.

[0012] Furthermore, the document creation support device of this disclosure may have a processor that performs control to highlight the extracted region.

[0013] Furthermore, the document creation support device of this disclosure may also control the processor to display the generated observation text.

[0014] Furthermore, the document creation support device of this disclosure may have a processor that generates multiple findings statements for a single disease name.

[0015] Furthermore, the document creation support device of this disclosure may have a processor that generates multiple findings statements using multiple disease names.

[0016] Furthermore, the document creation support device of this disclosure may have a processor that controls the display of multiple generated observation sentences and accept observation sentences selected by the user.

[0017] Furthermore, the document creation support device of this disclosure may have a processor that uses medical images to infer disease names and, based on the inference results, controls the display manner of the generated multiple findings statements.

[0018] Furthermore, the document creation support device of this disclosure may have a processor that infers the name of a disease based on the similarity between a medical image and images prepared in advance for each disease.

[0019] Furthermore, the document creation support device of this disclosure may have a processor that infers the name of a disease based on statistical values ​​of pixel values ​​within a region extracted from a medical image.

[0020] Furthermore, the document creation support device of this disclosure may have a processor that infers the disease name based on a medical image and a pre-trained model that has been trained in advance using training data including training medical images and disease names of diseases contained in the training medical images.

[0021] Furthermore, the document creation support device of this disclosure may have a processor that displays, in an identifiable manner, which of several statuses related to the user's medical document creation work the extracted area is in.

[0022] Furthermore, the document creation support device of this disclosure may include two or more statuses, such as a status where the user has not yet confirmed the area, a status where the user has specified that the area be included in the creation process but the creation process is not yet complete, a status where the user has specified that the area be excluded from the creation process, and a status where the creation process for the area has been completed.

[0023] Further, in the document creation support apparatus of the present disclosure, when the user designates that the status is a status in which an area is to be the target of the creation work and the creation work is incomplete, the processor may perform control to visibly display the status by adding a predetermined mark to the area.

[0024] Further, in the document creation support apparatus of the present disclosure, the processor may further perform control to list and display information regarding each of a plurality of areas.

[0025] Further, in the document creation support apparatus of the present disclosure, the processor may perform control to list and display information regarding each of a plurality of areas for each status.

[0026] Further, the document creation support method of the present disclosure is one in which a processor included in a document creation support apparatus executes a process of extracting an area having one or more preset physical features from a medical image and generating a finding sentence using a disease name associated with the physical features of at least one of the extracted areas.

[0027] Further, the document creation support program of the present disclosure is for causing a processor included in a document creation support apparatus to execute a process of extracting an area having one or more preset physical features from a medical image and generating a finding sentence using a disease name associated with the physical features of at least one of the extracted areas.

Advantages of the Invention

[0028] According to the present disclosure, it is possible to easily support the generation of a radiology report.

Brief Description of the Drawings

[0029] [Figure 1] It is a block diagram showing a schematic configuration of a medical information system. [Figure 2] It is a block diagram showing an example of the hardware configuration of a document creation support apparatus. [Figure 3]This figure shows an example of a disease name table according to the first embodiment. [Figure 4] This block diagram shows an example of the functional configuration of a document creation support device according to the first embodiment. [Figure 5] This figure shows an example of the extraction results for regions with physical characteristics. [Figure 6] This is a flowchart showing an example of document creation support processing according to the first embodiment. [Figure 7] This is a block diagram showing an example of the functional configuration of a document creation support device according to the second embodiment. [Figure 8] This figure shows examples of how multiple observation statements are displayed. [Figure 9] This flowchart shows an example of document creation support processing according to the second embodiment. [Figure 10] This block diagram shows an example of the functional configuration of a document creation support device according to the third embodiment. [Figure 11] This figure shows an example of a screen displaying the findings. [Figure 12] This figure shows an example of a screen displaying the findings. [Figure 13] This figure shows an example of a status display screen. [Figure 14] This is a flowchart showing an example of document creation support processing according to the third embodiment. [Figure 15] This figure shows an example of a list view screen. [Figure 16] This figure shows an example of a list view screen. [Modes for carrying out the invention]

[0030] Hereinafter, with reference to the drawings, examples of embodiments for carrying out the technology of this disclosure will be described in detail.

[0031] [First Embodiment] First, with reference to Figure 1, the configuration of Medical Information System 1, to which the document creation support device related to the disclosure technology is applied, will be explained. Medical Information System 1 is a system for taking photographs of the diagnostic target area of ​​a subject and storing the medical images obtained from the photographs, based on examination orders from physicians in clinical departments using a known ordering system. Medical Information System 1 is also a system for radiologists to interpret medical images and create interpretation reports, and for physicians in the requesting clinical departments to view interpretation reports and perform detailed observations of the medical images being interpreted.

[0032] As shown in Figure 1, the medical information system 1 according to this embodiment includes multiple imaging devices 2, multiple image interpretation workstations (WS) 3 which are image interpretation terminals, clinical department WS4, image server 5, image database (DataBase: DB) 6, image interpretation report server 7, and image interpretation report DB8. The imaging devices 2, image interpretation WS3, clinical department WS4, image server 5, and image interpretation report server 7 are connected to each other via a wired or wireless network 9, enabling them to communicate with one another. In addition, image DB6 is connected to image server 5, and image interpretation report DB8 is connected to image interpretation report server 7.

[0033] The imaging device 2 is a device that generates a medical image representing the diagnostic target area of ​​a subject by imaging that area. The imaging device 2 may be, for example, a simple X-ray imaging device, an endoscope, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or a PET (Positron Emission Tomography) device. The medical image generated by the imaging device 2 is transmitted to and stored in the image server 5.

[0034] The Department WS4 is a computer used by physicians in a clinical department for detailed observation of medical images, viewing of image interpretation reports, and creation of electronic medical records. In the Department WS4, the creation of patient electronic medical records, requests for image viewing from the image server 5, and the display of medical images received from the image server 5 are performed by executing software programs for each process. In addition, the Department WS4 performs processes such as automatic detection or highlighting of disease-prone areas in medical images, requests for viewing of image interpretation reports from the image interpretation report server 7, and the display of image interpretation reports received from the image interpretation report server 7, by executing software programs for each process.

[0035] Image server 5 incorporates a software program that provides database management system (DBMS) functionality to a general-purpose computer. When image server 5 receives a request to register a medical image from imaging device 2, it formats the medical image into a database format and registers it in image DB6.

[0036] Image DB6 stores image data representing medical images acquired by imaging device 2, along with associated information. This associated information includes, for example, an image ID (identification) to identify individual medical images, a patient ID to identify the patient being studied, an examination ID to identify the examination content, and a unique ID (UID: unique identification) assigned to each medical image. It also includes information such as the examination date and time the medical image was generated, the type of imaging device used to acquire the image, patient information (e.g., patient's name, age, and gender), examination site (i.e., imaging site), imaging information (e.g., imaging protocol, imaging sequence, imaging method, imaging conditions, and whether contrast agent was used), and series number or acquisition number when multiple medical images are acquired in a single examination. Furthermore, when image server 5 receives a viewing request from image interpretation WS3 via network 9, it searches for medical images registered in image DB6 and sends the retrieved medical images to the requesting image interpretation WS3.

[0037] The image interpretation report server 7 incorporates a software program that provides DBMS functionality to a general-purpose computer. When the image interpretation report server 7 receives a registration request for an image interpretation report from the image interpretation WS3, it formats the image interpretation report into a database format and registers it in the image interpretation report database 8. Furthermore, when it receives a search request for an image interpretation report, it searches for that report in the image interpretation report database 8.

[0038] The image interpretation report DB8 stores image interpretation reports that include information such as an image ID to identify the medical image being interpreted, a radiologist ID to identify the radiologist who performed the interpretation, the name of the lesion, the location of the lesion, findings, and the confidence level of the findings.

[0039] Network 9 is a wired or wireless local area network that connects various devices within the hospital. If the image interpretation WS3 is installed in another hospital or clinic, Network 9 may be configured to connect the local area networks of each hospital via the Internet or a dedicated line. In either case, it is preferable that Network 9 be configured to enable high-speed transfer of medical images, such as through an optical network.

[0040] The image interpretation WS3 performs the following: requests to view medical images from the image server 5, various image processing on medical images received from the image server 5, display of medical images, analysis processing of medical images, highlighting of medical images based on the analysis results, and creation of image interpretation reports based on the analysis results. In addition, the image interpretation WS3 assists in the creation of image interpretation reports, requests registration and viewing of image interpretation reports from the image interpretation report server 7, and displays image interpretation reports received from the image interpretation report server 7. The image interpretation WS3 performs each of the above processes by executing software programs for each process. The image interpretation WS3 incorporates a document creation support device 10, which will be described later, and since the processes other than those performed by the document creation support device 10 are performed by well-known software programs, a detailed explanation is omitted here. Alternatively, the image interpretation WS3 may not perform the processes other than those performed by the document creation support device 10, and a separate computer that performs those processes may be connected to the network 9, and the computer may perform the requested processing in response to processing requests from the image interpretation WS3. The document creation support device 10 included in the image interpretation WS3 will be described in detail below.

[0041] Next, with reference to Figure 2, the hardware configuration of the document creation support device 10 according to this embodiment will be described. As shown in Figure 2, the document creation support device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The document creation support device 10 also includes a display 23 such as a liquid crystal display, input devices 24 such as a keyboard and mouse, and a network I / F (Interface) 25 connected to the network 9. The CPU 20, memory 21, storage unit 22, display 23, input devices 24, and network I / F 25 are connected to the bus 27.

[0042] The storage unit 22 is implemented by an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, etc. The document creation support program 30 is stored in the storage unit 22 as a storage medium. The CPU 20 reads the document creation support program 30 from the storage unit 22, expands it into memory 21, and executes the expanded document creation support program 30.

[0043] Furthermore, the memory unit 22 stores a disease name table 32. Figure 3 shows an example of the disease name table 32. As shown in Figure 3, the disease name table 32 contains multiple records that associate combinations of organs and physical features included in medical images with disease names. The disease name table 32 is an example of data that associates physical features with disease names related to the disclosed technology.

[0044] Next, with reference to Figure 4, the functional configuration of the document creation support device 10 according to this embodiment will be described. As shown in Figure 4, the document creation support device 10 includes an acquisition unit 40, an extraction unit 42, a display control unit 44, a reception unit 46, and a generation unit 48. The CPU 20 executes the document creation support program 30, thereby enabling the acquisition unit 40, the extraction unit 42, the display control unit 44, the reception unit 46, and the generation unit 48 to function.

[0045] The acquisition unit 40 acquires the medical images to be diagnosed (hereinafter referred to as "diagnostic images") from the image server 5 via the network interface 25.

[0046] The extraction unit 42 extracts regions having one or more pre-set physical features from the diagnostic target image acquired by the acquisition unit 40. In this embodiment, the extraction unit 42 extracts regions having physical features from the diagnostic target image using a trained model M1 for extracting regions having physical features from the diagnostic target image.

[0047] The trained model M1 is constructed using a Convolutional Neural Network (CNN), for example, which takes medical images as input and outputs regions containing physical features within those medical images. The trained model M1 is a model trained through machine learning using a large number of medical images with known regions containing physical features as training data.

[0048] Examples of regions with physical characteristics include regions where the pixel values ​​are within a predetermined range. More specifically, examples of regions where the pixel values ​​are within a predetermined range include regions where the pixel values ​​are relatively close to the predetermined value compared to the surrounding area. More specifically, examples of regions where the pixel values ​​are relatively close to the predetermined value compared to the surrounding area include regions that are relatively white compared to the surrounding area, and regions that are relatively black compared to the surrounding area.

[0049] Furthermore, areas with physical characteristics include, for example, areas with a shape that has pre-defined characteristics. Specifically, examples of areas with a shape that has pre-defined characteristics include areas with a raised shape and areas with an uneven, irregular edge.

[0050] The physical characteristics described above are useful for evaluating diseases. Some of these useful characteristics are specific to certain organs. For example, in a CT image, a relatively white mass in the brain compared to the surrounding area suggests a cerebral hemorrhage. Similarly, in a CT image, a relatively dark mass in the brain compared to the surrounding area suggests a cerebral infarction. In an endoscopic image, a raised area in the large intestine suggests a colon polyp. In an endoscopic image, an area with irregular, uneven edges in the liver suggests cirrhosis.

[0051] Therefore, in this embodiment, a pre-trained model M1 is prepared for each combination of organ and physical features. In other words, the physical features to be extracted are pre-set for each organ. The extraction unit 42 inputs the image to be diagnosed into the pre-trained model M1 prepared for the organs included in the image to extract regions having one or more physical features from the image to be diagnosed.

[0052] Figure 5 shows an example of the extraction results by the extraction unit 42. In Figure 5, an example is shown in which a region that is relatively white compared to the surrounding area is extracted from a CT image of the brain. In the example in Figure 5, the area filled with diagonal lines indicates a region that is relatively white compared to the surrounding area. In this way, by using the trained model M1, it is possible to extract regions that have physical features that are difficult to perceive with the human eye.

[0053] The display control unit 44 controls the display 23 to display information representing the region extracted by the extraction unit 42. Specifically, the display control unit 44 controls the display to highlight the region extracted by the extraction unit 42 in the diagnostic target image by filling that region with a preset color. As a result of this control, for example, the diagnostic target image in Figure 5, in which the region filled with diagonal lines is filled with light blue, is displayed on the display 23. When multiple regions with different physical characteristics are extracted by the extraction unit 42 during this control, the display control unit 44 may make the physical characteristics distinguishable by using different colors for each physical characteristic. The display control unit 44 may also control the display to highlight the region extracted by the extraction unit 42 by drawing the outer edge of that region with a line of a preset color. Alternatively, the display control unit 44 may also control the display to highlight the region extracted by the extraction unit 42 by surrounding it with a bounding box.

[0054] Furthermore, in a CT image containing multiple slice images, if there is a slice image in which the extraction unit 42 has extracted only one region, the display control unit 44 may control the display 23 to display information representing the region extracted by the extraction unit 42 for that slice image. In this case, if there are multiple slice images in which the region is extracted only once, the display control unit 44 may control the display 23 to display information representing the region extracted by the extraction unit 42 for the slice image in which the area of ​​that single region is the largest.

[0055] Furthermore, the display control unit 44 controls the display 23 to show the observation text generated by the generation unit 48, which will be described later.

[0056] A user, such as a physician, selects the area to be included in the image interpretation report from among the areas with physical characteristics displayed on the display 23, via the input device 24. The reception unit 46 receives information representing the area selected by the user from among the areas extracted by the extraction unit 42.

[0057] The generation unit 48 refers to the disease name table 32 and generates a findings statement using disease names associated with the physical characteristics of the region received by the reception unit 46, that is, the region selected by the user. Specifically, the generation unit 48 refers to the disease name table 32 and obtains disease names associated with the combination of organs included in the diagnostic image and the physical characteristics of the region received by the reception unit 46. Then, the generation unit 48 generates a findings statement using the obtained disease names. Examples of findings statements related to the brain include "Intracranial hemorrhage is observed," "Subarachnoid hemorrhage is observed," and "Cerebral infarction is observed." For example, the generation unit 48 generates a findings statement by inputting the disease name into a recurrent neural network that has been trained to generate text from input words. The user creates an image interpretation report based on the findings statement generated by the generation unit 48 and displayed on the display 23 under the control of the display control unit 44.

[0058] Furthermore, the generation unit 48 may generate multiple findings for a single disease name. In this case, for example, for a lung included in the diagnostic image, the generation unit 48 may generate multiple findings for a single disease name, "tumor," corresponding to a region selected by the user, with different combinations of characteristic items, such as "A tumor is observed in the left upper lobe" and "A 4.2 cm tumor with pleural indentation is observed in the left upper lobe." Also, in this case, for example, the generation unit 48 may generate multiple findings for a single disease name, "tumor," with the same meaning but different wording, such as "A partially solid tumor is observed in the left upper lobe" and "A tumor with a solid center and a ground-glass-like periphery is observed in the left upper lobe."

[0059] Next, the operation of the document creation support device 10 according to this embodiment will be explained with reference to Figure 6. The CPU 20 executes the document creation support program 30, thereby executing the document creation support process shown in Figure 6. The document creation support process shown in Figure 6 is executed, for example, when a user inputs an instruction to start execution.

[0060] In step S10 of Figure 6, the acquisition unit 40 acquires the diagnostic target image from the image server 5 via the network interface 25. In step S12, the extraction unit 42, as described above, uses the trained model M1 to extract regions having one or more pre-defined physical features from the diagnostic target image acquired in step S10. In step S14, the display control unit 44, as described above, controls the display 23 to display information representing the regions extracted in step S12.

[0061] In step S16, the reception unit 46 receives information representing the region selected by the user from the regions extracted in step S12. In step S18, the generation unit 48, as described above, refers to the disease name table 32 and generates a report using the disease name associated with the physical characteristics of the region received in step S16. In step S20, the display control unit 44 controls the display 23 to show the report generated in step S18. When the processing in step S20 is completed, the document creation support process ends.

[0062] As described above, this embodiment makes it possible to easily support the generation of image interpretation reports.

[0063] [Second Embodiment] A second embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the document creation support device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0064] In the disease name table 32 according to this embodiment, multiple disease names are associated with a single physical characteristic. Specifically, for example, the organ "brain" and the physical characteristic "a mass that is relatively white compared to its surroundings" are associated with two disease names, "cerebral hemorrhage" and "brain tumor."

[0065] Referring to Figure 7, the functional configuration of the document creation support device 10 according to this embodiment will be described. Functional parts having the same functions as the document creation support device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 7, the document creation support device 10 includes an acquisition unit 40, an extraction unit 42, a display control unit 44A, a reception unit 46A, a generation unit 48A, and an estimation unit 50. The CPU 20 executes the document creation support program 30, thereby enabling the acquisition unit 40, the extraction unit 42, the display control unit 44A, the reception unit 46A, the generation unit 48A, and the estimation unit 50 to function.

[0066] The display control unit 44A, similar to the display control unit 44 in the first embodiment, controls the display 23 to display information representing the region extracted by the extraction unit 42.

[0067] Furthermore, the display control unit 44A controls the display of the multiple findings generated by the generation unit 48A, which will be described later, on the display 23. During this control, the display control unit 44A controls the display manner of the multiple findings generated by the generation unit 48A based on the disease name prediction result from the prediction unit 50, which will be described later.

[0068] Specifically, the display control unit 44A controls the display on the display 23 to display, with a higher priority, the finding sentences that include the disease name predicted by the prediction unit 50 from among the multiple finding sentences generated by the generation unit 48A, than the finding sentences that do not include the disease name predicted by the prediction unit 50. As a specific example, let's consider the case where the two finding sentences generated by the generation unit 48A are "cerebral hemorrhage is observed" and "brain tumor is observed," and the disease name predicted by the prediction unit 50 is "brain tumor." In this case, as shown in Figure 8 as an example, the display control unit 44A increases the priority of the finding sentences that include "brain tumor" by displaying them above the finding sentences that do not include "brain tumor." The display control unit 44A may, for example, make the font size of the finding sentences that include the disease name predicted by the prediction unit 50 larger than the font size of the finding sentences that do not include the disease name predicted by the prediction unit 50. Furthermore, for example, the display control unit 44A may control the display 23 to display multiple findings sentences generated by the generation unit 48A in predetermined colors according to priority. Alternatively, for example, the display control unit 44A may control the display 23 to display only the findings sentences that include the disease name predicted by the prediction unit 50 from among the multiple findings sentences generated by the generation unit 48A.

[0069] The reception unit 46A, similar to the reception unit 46 in the first embodiment, receives information representing the region selected by the user from among the regions extracted by the extraction unit 42. The reception unit 46A also receives the observation statement selected by the user from among the multiple observation statements displayed on the display 23 under the control of the display control unit 44A. This received observation statement is used to create the image interpretation report.

[0070] The generation unit 48A refers to the disease name table 32 and generates multiple findings sentences using multiple disease names associated with the physical characteristics of the region received by the reception unit 46A, i.e., the region selected by the user. Specifically, the generation unit 48A refers to the disease name table 32 and obtains multiple disease names associated with combinations of organs included in the diagnostic image and the physical characteristics of the region received by the reception unit 46A. Then, the generation unit 48A generates multiple findings sentences using the obtained multiple disease names. For example, the generation unit 48A generates multiple findings sentences by inputting each disease name into a recurrent neural network that has been trained to generate text from input words.

[0071] The generation unit 48A may also derive a recommendation score for each of the multiple observation statements it has generated. In this case, for example, the generation unit 48A derives the recommendation score for the observation statement based on the image of the region received by the reception unit 46A in the image to be diagnosed, the multiple observation statements it has generated, and a pre-trained model M3. The pre-trained model M3 is, for example, a machine learning model pre-trained using training data that includes the image of the region received by the reception unit 46A in the image to be diagnosed, the multiple observation statements, and the recommendation score for each of those multiple observation statements. When the pre-trained model M3 receives the image of the region received by the reception unit 46A in the image to be diagnosed and the multiple observation statements generated by the generation unit 48A as input, it outputs the recommendation score for each of the multiple observation statements. This pre-trained model M3 is, for example, configured to include a CNN. In this embodiment, the display control unit 44A may perform control to display the recommendation level derived for each of the multiple observation sentences, along with the multiple observation sentences generated by the generation unit 48A.

[0072] The inference unit 50 infers the disease name using the image to be diagnosed. Specifically, the inference unit 50 infers the disease name based on the image of the region in the image to be diagnosed received by the reception unit 46A and a pre-trained model M2 that has been trained using training data that includes images of the region having physical features in training medical images and the disease names of the diseases contained in the images of that region. When the image of the region in the image to be diagnosed received by the reception unit 46A is input to this pre-trained model M2, the disease name is output. This pre-trained model M2 is composed of, for example, a CNN.

[0073] The prediction unit 50 may also predict the disease name based on the similarity between the image to be diagnosed and images prepared in advance for each disease. In this case, the similarity of the images can be determined by, for example, the distance between feature vectors obtained by vectorizing multiple features extracted from the image. In this case, the prediction unit 50 predicts the disease name of the image with the highest similarity to the image of the region portion received by the image reception unit 46A of the image to be diagnosed.

[0074] Furthermore, in the case of liver cysts, the CT values ​​of the cystic area are often uniformly between 10 and 40. In the case of liver tumors, the CT values ​​of the tumor area often show a large variance. Therefore, the estimation unit 50 may estimate the disease name based on the statistical values ​​of the pixel values ​​within the region extracted from the image to be diagnosed. Examples of statistical values ​​in this case include at least one of the mean, standard deviation, variance, maximum brightness value, and minimum brightness value.

[0075] Next, the operation of the document creation support device 10 according to this embodiment will be explained with reference to Figure 9. The document creation support process shown in Figure 9 is executed when the CPU 20 executes the document creation support program 30. The document creation support process shown in Figure 9 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 9 that perform the same process as in Figure 6 are given the same step numbers and their explanation is omitted.

[0076] In step S18A of Figure 9, the generation unit 48A, as described above, refers to the disease name table 32 and generates multiple findings statements using multiple disease names associated with the physical characteristics of the region received in step S16. In step S19A, the prediction unit 50, as described above, predicts the disease name using the diagnostic target image acquired in step S10.

[0077] In step S20A, the display control unit 44A controls the display of the multiple findings generated in step S18A on the display 23. During this control, as described above, the display control unit 44A controls the display manner of the multiple findings generated in step S18A based on the disease name prediction result in step S19A. In step S22A, the reception unit 46A receives the findings selected by the user from among the multiple findings displayed on the display 23 in step S20A. These received findings are used to create the image interpretation report. When the processing in step S22A is completed, the document creation support process ends.

[0078] As described above, this embodiment can achieve the same effects as the first embodiment.

[0079] [Third Embodiment] A third embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the document creation support device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described. Also, the disease name table 32 according to this embodiment is the same as that of the second embodiment, and therefore will not be described.

[0080] Referring to Figure 10, the functional configuration of the document creation support device 10 according to this embodiment will be described. Functional parts having the same functions as the document creation support device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 10, the document creation support device 10 includes an acquisition unit 40, an extraction unit 42, a display control unit 44B, a reception unit 46B, and a generation unit 48B. The CPU 20 executes the document creation support program 30, thereby enabling the acquisition unit 40, the extraction unit 42, the display control unit 44B, the reception unit 46B, and the generation unit 48B to function.

[0081] The display control unit 44B controls the display 23 to show information representing the region extracted by the extraction unit 42. The display control unit 44B also controls the display 23 to show multiple observation statements generated by the generation unit 48B, which will be described later.

[0082] Furthermore, the display control unit 44B controls the display of the area extracted by the extraction unit 42 in a way that allows identification of which of the multiple statuses related to the user's medical document creation work it is. Examples of medical documents include image interpretation reports. In this embodiment, an example is described in which the following four statuses are applied as multiple statuses related to the medical document creation work.

[0083] The first status is that the user has not yet confirmed the area extracted by the extraction unit 42. The second status is that the user has specified that the area extracted by the extraction unit 42 should be included in the medical document creation process, but that process is not yet complete. The third status is that the user has specified that the area extracted by the extraction unit 42 should not be included in the medical document creation process. The fourth status is that the medical document creation process for the area extracted by the extraction unit 42 has been completed. Note that there may be two or three of these four statuses.

[0084] When the display control unit 44B first displays information representing the region extracted by the extraction unit 42 on the display 23, it performs control to display the first status in a way that makes it identifiable. Specifically, for example, the display control unit 44B performs control to display the image to be diagnosed on the display 23 with the region extracted by the extraction unit 42 in the image to be diagnosed filled with a preset color. In this control, if multiple regions with different physical characteristics are extracted by the extraction unit 42, the display control unit 44B may make the physical characteristics identifiable by using a different color for each physical characteristic.

[0085] The display control unit 44B, when a user specifies an area and performs an operation to change the status of the specified area to a second status, performs control to display the second status in an identifiable manner by adding a predetermined mark to that area. This makes areas that have been specified by the user as targets for medical document creation but where medical document creation is incomplete more visible, thus preventing omissions in medical document creation.

[0086] When a user specifies an area and performs an operation to instruct the display of observation texts, the display control unit 44B controls the display 23 to display multiple observation texts generated by the generation unit 48B, which will be described later, for that area. An example of the observation text display screen displayed on the display 23 by this control is shown in Figure 11. As shown in Figure 11, the observation text display screen displays multiple observation texts, a button that the user specifies when selecting each observation text, and a button that the user specifies when they determine there are no observation texts. When the user performs an operation to select one observation text from the multiple observation texts on the observation text display screen, the display control unit 44B controls the display to make the fourth status identifiable by defilling the area and drawing the outer edge of the area with a predetermined colored line.

[0087] Figure 12 shows another example of the findings display screen. Figure 11 is an example where a region with physical characteristics is extracted within the liver, and Figure 12 is an example where the shape of the liver itself has physical characteristics.

[0088] The display control unit 44B performs control to display the third status in an identifiable manner by graying out the area when the user has specified an area and performed an operation to change the status of the specified area to a third status. An example of this operation is when the user selects the "No findings" button on the findings text display screen shown in Figure 11.

[0089] Figure 13 shows an example of the status display screen shown on the display 23 by the control of the display control unit 44B described above. Figure 13 shows an example in which a CT image of the liver is applied as the image to be diagnosed. In Figure 13, the status of regions R1 and R2 is the first status, and the status of region R3 is the second status. A check mark C is added to region R3 as a predetermined mark. Also in Figure 13, the status of region R4 is the third status, and the status of region R5 is the fourth status.

[0090] The method for displaying the first to fourth statuses in an identifiable manner is not limited to the examples above. For example, the first to fourth statuses may be displayed in an identifiable manner by changing the line type or thickness of the area's outline, the transparency or pattern of the area's fill, blinking the area, displaying an animation of the area, or adding different marks. Furthermore, the marks are not limited to check marks; they may also be arrows or symbols such as "+".

[0091] The reception unit 46B receives information representing the region selected by the user from among the regions extracted by the extraction unit 42. The reception unit 46B also receives an operation indicating which of the four statuses described above should be assigned to the selected region. Furthermore, the reception unit 46B receives the observation statement selected by the user from among the multiple observation statements displayed on the display 23 under the control of the display control unit 44B. This received observation statement is used to create the image interpretation report.

[0092] The generation unit 48B, similar to the generation unit 48A in the second embodiment, refers to the disease name table 32 and generates multiple findings statements using multiple disease names associated with the physical characteristics of the region received by the reception unit 46B. Alternatively, the generation unit 48B may generate one findings statement using one disease name associated with the physical characteristics of the region received by the reception unit 46B, similar to the generation unit 48 in the first embodiment.

[0093] Next, the operation of the document creation support device 10 according to this embodiment will be explained with reference to Figure 14. The document creation support process shown in Figure 14 is executed when the CPU 20 executes the document creation support program 30. The document creation support process shown in Figure 14 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 14 that perform the same process as in Figure 9 are given the same step numbers and their explanation is omitted.

[0094] In step S30 of Figure 14, the display control unit 44B controls the display 23 to show information representing the region extracted in step S12. During this control, the display control unit 44B controls the display so that it can be identified that the status of each region is the first status, as described above. In step S32, the reception unit 46B accepts an operation from the user. If the operation accepted by the reception unit 46B in step S32 is an operation in which the user specifies a region and sets the status of the specified region to the second status, the process proceeds to step S34.

[0095] In step S34, the display control unit 44B performs control to display the second status in an identifiable manner by adding a predetermined mark to an area specified by the user. When the processing in step S34 is completed, the process returns to step S32.

[0096] If the operation received by the reception unit 46B in step S32 is an operation in which a region is specified by the user and an instruction to display a finding statement, the process proceeds to step S36. In step S36, the generation unit 48B, as described above, refers to the disease name table 32 and generates multiple finding statements using multiple disease names associated with the physical characteristics of the region specified by the user.

[0097] In step S38, the display control unit 44B controls the display 23 to display the multiple observation statements generated in step S36 for the area specified by the user. In step S40, the reception unit 46B determines whether it has received the observation statement selected by the user from the multiple observation statements displayed on the display 23 in step S38. If this determination is affirmative, the process proceeds to step S42. In step S42, the display control unit 44B controls the display to display the fourth status in an identifiable manner by clearing the fill of the area specified by the user and drawing the outer edge of that area with a predetermined colored line. When the process in step S42 is completed, the process returns to step S32.

[0098] If the operation received by the reception unit 46B in step S40 is an operation in which the user specifies an area and sets the status of the specified area to a third status, the determination in step S40 is a negative determination, and the process proceeds to step S44. In step S44, the display control unit 44B performs control to display the third status in an identifiable manner by graying out the area specified by the user. When the processing in step S44 is completed, the process returns to step S32.

[0099] If the operation received by the reception unit 46B in step S32 is an operation to terminate the display of the screen, the document creation support process is terminated.

[0100] As described above, this embodiment can achieve the same effects as the first embodiment. Furthermore, this embodiment allows the user to easily grasp the progress of the work.

[0101] In each of the above embodiments, as an example, as shown in Figure 15, the display control units 44, 44A, and 44B may perform control to display a list of information for each of the multiple regions extracted by the extraction unit 42 when the user performs an operation to instruct the display to show a list. In the third embodiment, when the above list display control is performed, the display control unit 44B may perform control to display a list of information for each of the multiple regions extracted by the extraction unit 42 for each status. The user may also select the region to be used for creating the image interpretation report from among the multiple regions displayed in the list on the display 23 via the input device 24.

[0102] Furthermore, in the above embodiments, the generation units 48, 48A, and 48B were described as generating observation statements for regions selected by the user from among the regions extracted by the extraction unit 42, but the system is not limited to this. The generation units 48, 48A, and 48B may also be configured to generate observation statements for all regions extracted by the extraction unit 42.

[0103] Furthermore, in the embodiments described above, the extraction unit 42 extracts regions having physical features from the diagnostic image using a trained model M1, but it is not limited to this. For example, the extraction unit 42 may extract regions in the diagnostic image that satisfy pre-set conditions as regions having physical features. Examples of regions that satisfy these conditions include regions where the CT value is above a first threshold and the area is above a second threshold. Examples of regions that satisfy these conditions include regions where the CT value is below a third threshold and the area is above a second threshold. Examples of regions that satisfy these conditions include regions where the CT value is within a certain proportion of the top frequencies in the histogram and the area is above a second threshold. In addition, the conditions in this case may be set for each organ, for example.

[0104] In this case, as an example, as shown in Figure 16, the display control units 44, 44A, and 44B may also control the display 23 to display information indicating the conditions under which each of the multiple regions extracted by the extraction unit 42 was extracted. Figure 16 shows an example in which each region is extracted according to the conditions that it is low brightness in the liver and has an area greater than or equal to a certain size, that is, a region in the liver where the CT value is less than the third threshold and the area is greater than or equal to the second threshold.

[0105] Furthermore, in each of the above embodiments, if the user modifies the disease name for a region extracted by the extraction unit 42, the document creation support device 10 may add a record to the disease name table 32 that associates the physical characteristics of that region with the disease name modified by the user.

[0106] Furthermore, in each of the above embodiments, the hardware structure of the processing unit that performs various processes, such as the various functional units of the document creation support device 10, can be the following types of processors. As mentioned above, these types of processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to perform specific processes.

[0107] A single processing unit may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor.

[0108] Examples of configuring multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software combine to form a single processor, which then functions as multiple processing units, as exemplified by client and server computers. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as exemplified by System on Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned processors.

[0109] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits, which are combinations of circuit elements such as semiconductor devices.

[0110] Furthermore, although the above embodiment describes a configuration in which the document creation support program 30 is pre-stored (installed) in the storage unit 22, the invention is not limited to this configuration. The document creation support program 30 may be provided in the form of a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the document creation support program 30 may be provided in the form of a download from an external device via a network.

[0111] The disclosures of Japanese Patent Application No. 2021-068674, filed on 14 April 2021, and Japanese Patent Application No. 2021-208523, filed on 22 December 2021, are incorporated herein by reference in their entirety. Furthermore, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated as being incorporated by reference.

Claims

1. A document creation support device comprising at least one processor, The aforementioned processor, Extract regions from medical images that have one or more predefined physical features, Multiple disease names are generated using the physical characteristics of at least one of the extracted regions. Using the aforementioned medical images, the name of the disease is inferred. Based on the inference results, control is implemented to display multiple generated observation statements according to their priority. In the above control, the priority of the findings statement containing the predicted disease name is set higher than the priority of the findings statement that does not contain the predicted disease name. Document creation support device.

2. The aforementioned processor, Control the display of information representing the extracted region. The system accepts information representing the selected region from the extracted regions. The findings statement is generated using the disease name associated with the physical characteristics of the selected region. The document creation support device according to claim 1.

3. The aforementioned processor, The system references data that associates the aforementioned physical characteristics with disease names, and generates the aforementioned findings statement using the disease names associated with the physical characteristics. A document creation support device according to claim 1 or claim 2.

4. The aforementioned processor, From the aforementioned medical image, a region having the aforementioned physical characteristics is extracted, within a range of pixel values ​​that has been set. A document creation support device according to any one of claims 1 to 3.

5. The aforementioned processor, From the aforementioned medical image, regions with the defined physical characteristics are extracted as regions with a defined shape. A document creation support device according to any one of claims 1 to 4.

6. The aforementioned physical characteristics are set for each organ. A document creation support device according to any one of claims 1 to 5.

7. The aforementioned processor, Control the highlighting of the extracted region. A document creation support device according to any one of claims 1 to 6.

8. The aforementioned processor, Control the display of multiple generated observation statements. Accepts user-selected observations. A document creation support device according to any one of claims 1 to 7.

9. The aforementioned processor, The disease name is inferred based on the similarity between the aforementioned medical image and pre-prepared images for each disease. A document creation support device according to any one of claims 1 to 8.

10. The aforementioned processor, The disease name is inferred based on the statistical values ​​of the pixel values ​​within the region extracted from the aforementioned medical image. A document creation support device according to any one of claims 1 to 8.

11. The aforementioned processor, The disease name is inferred based on the aforementioned medical image and a pre-trained model that has been trained using training data including training medical images and disease names of diseases contained in the training medical images. A document creation support device according to any one of claims 1 to 8.

12. The aforementioned processor, The system provides control to clearly display which of several statuses the extracted area represents in relation to the user's medical document creation process. A document creation support device according to any one of claims 1 to 11.

13. The aforementioned multiple statuses include two or more of the following: a status in which the user has not yet confirmed the area; a status in which the user has designated the area to be the subject of the creation work and the creation work is not yet completed; a status in which the user has designated the area to be excluded from the creation work; and a status in which the creation work for the area has been completed. The document creation support device according to claim 12.

14. The aforementioned processor, If the status is such that the user has specified that the area be the target of the creation work, and the creation work is incomplete, the system will display the status in an identifiable manner by adding a predetermined mark to the area. The document creation support device according to claim 13.

15. The aforementioned processor, Furthermore, control is performed to display a list of information related to each of the multiple aforementioned regions. A document creation support device according to any one of claims 12 to 14.

16. The aforementioned processor, For each of the aforementioned statuses, control is performed to display a list of information related to each of the multiple aforementioned areas. The document creation support device according to claim 15.

17. Extract regions from medical images that have one or more predefined physical features, Multiple disease names are generated using the physical characteristics of at least one of the extracted regions. Using the aforementioned medical images, the name of the disease is inferred. Based on the inference results, control is implemented to display multiple generated observation statements according to their priority. In the above control, the priority of the findings statement containing the predicted disease name is set higher than the priority of the findings statement that does not contain the predicted disease name. A document creation support method in which the processing is performed by a processor in a document creation support device.

18. Extract regions from medical images that have one or more predefined physical features, Multiple disease names are generated using the physical characteristics of at least one of the extracted regions. Using the aforementioned medical images, the name of the disease is inferred. Based on the inference results, control is implemented to display multiple generated observation statements according to their priority. In the above control, the priority of the findings statement containing the predicted disease name is set higher than the priority of the findings statement that does not contain the predicted disease name. A document creation support program that causes the processor of a document creation support device to perform the processing.

Images