Medical image analysis device, medical image analysis method, and medical image analysis program

Abstract

This medical image analysis device performs first analysis processing related to multiple areas of interest contained in a medical image, receives input relating to a first area of interest from the multiple areas of interest, and, on the basis of this input, performs second analysis processing relating to a second area of interest that is related to the first area of interest, and outputs the analysis results of the second area of interest.

Description

Technical Field

[0001] The present disclosure relates to a medical image analysis device, a medical image analysis method, and a medical image analysis program.

Background Art

[0002] International Publication No. 2020 / 209382 discloses a technique for detecting a plurality of findings representing features related to abnormal shadows included in a medical image, identifying at least one finding to be used for creating a radiology report from the detected findings, and creating a radiology report using the identified findings.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, the technique described in International Publication No. 2020 / 209382 has room for improvement from the viewpoint of improving the accuracy of the analysis results of the region of interest.

[0004] The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a medical image analysis device, a medical image analysis method, and a medical image analysis program capable of improving the accuracy of the analysis results of a region of interest.

Means for Solving the Problems

[0005] The medical image analysis device of the present disclosure is a medical image analysis device including at least one processor. The processor performs a first analysis process on a plurality of regions of interest included in a medical image, receives an input regarding a first region of interest among the plurality of regions of interest, and based on the input, performs a second analysis process on a second region of interest related to the first region of interest, and outputs an analysis result of the second region of interest.

[0006] Furthermore, the medical image analysis device of this disclosure may have a processor that accepts, as input, a selection of one analysis result from among multiple analysis results of a first analysis process relating to a first region of interest, and performs a second analysis process relating to a second region of interest from which analysis results related to the selected analysis result have been obtained.

[0007] Furthermore, the medical image analysis device of this disclosure may have a processor that receives a statement of findings regarding a first region of interest as input, selects one analysis result from among a plurality of analysis results of a first analysis process regarding the first region of interest based on the statement of findings, and performs a second analysis process regarding a second region of interest from which an analysis result related to the one analysis result has been obtained.

[0008] Furthermore, the medical image analysis device of this disclosure may have a processor that, as a second analysis process, modifies the confidence level of an analysis result related to one of the multiple analysis results obtained by the first analysis process relating to a second region of interest.

[0009] Furthermore, the medical image analysis device of this disclosure may have a processor that, as a second analysis process, performs an analysis process on the second region of interest based on a partial image containing the second region of interest of the medical image, one analysis result, and a pre-trained model that has been trained in advance using training partial images and training data including the analysis results of the regions of interest contained in the training partial images.

[0010] Furthermore, in the medical image analysis device of this disclosure, if there are multiple analysis results related to one analysis result, the processor may select one analysis result from among the multiple analysis results related to one analysis result based on the co-occurrence probability between the one analysis result and each of the multiple analysis results related to that one analysis result.

[0011] Furthermore, the medical image analysis device of this disclosure may have a processor that, as a second analysis process, modifies the analysis parameters for the analysis results related to the analysis results of the first region of interest and performs the first analysis process.

[0012] Furthermore, the medical image analysis device disclosed herein may provide analysis results that include the name of the region of interest, findings, findings statements, or a diagnosis.

[0013] Furthermore, the medical image analysis device of this disclosure may have a processor that notifies the system that the result of the second analysis process for a second region of interest differs from the result of the first analysis process.

[0014] Furthermore, the medical image analysis device of this disclosure may have a processor that determines a second region of interest related to a first region of interest based on the input, and performs a second analysis process relating to the determined second region of interest.

[0015] Furthermore, the medical image analysis device of this disclosure may have a processor that, before receiving input, determines a second region of interest related to each of the multiple regions of interest, selects a second region of interest related to the first region of interest from among the second regions of interest determined for each of the multiple regions of interest based on the input, and performs a second analysis process on the selected second region of interest.

[0016] Furthermore, the medical image analysis method of this disclosure involves a processor in a medical image analysis device performing a first analysis process on multiple regions of interest included in a medical image, receiving input for the first region of interest among the multiple regions of interest, performing a second analysis process on a second region of interest related to the first region of interest based on the input, and outputting the analysis results for the second region of interest.

[0017] Furthermore, the medical image analysis program of this disclosure causes a processor in a medical image analysis device to perform a first analysis process on multiple regions of interest contained in a medical image, accept input for the first region of interest among the multiple regions of interest, perform a second analysis process on a second region of interest related to the first region of interest based on the input, and output the analysis result of the second region of interest.

[0018] Furthermore, the medical image analysis device of this disclosure is a medical image analysis device comprising at least one processor, the processor performing analysis processing on a region of interest included in a medical image, receiving input for correction to the analysis results of the region of interest, and, based on the input, changing the analysis parameters for the analysis results related to the corrected analysis results and performing analysis processing.

[0019] Furthermore, the medical image analysis method of this disclosure involves a processor in a medical image analysis device that performs analysis processing on a region of interest contained in a medical image, accepts input for corrections to the analysis results of the region of interest, and performs analysis processing by changing the analysis parameters for the analysis results related to the corrected analysis results based on the input.

[0020] Furthermore, the medical image analysis program of this disclosure performs analysis processing on regions of interest included in medical images, accepts input for corrections to the analysis results of the regions of interest, and causes a processor in a medical image analysis device to perform analysis processing by changing the analysis parameters for the analysis results related to the corrected analysis results based on the input. [Effects of the Invention]

[0021] According to this disclosure, the accuracy of the analysis results for the region of interest can be improved. [Brief explanation of the drawing]

[0022] [Figure 1] This is a block diagram illustrating the schematic configuration of a medical information system. [Figure 2] This block diagram shows an example of the hardware configuration of a medical image analysis device. [Figure 3] This figure shows an example of a related diagnostic name table. [Figure 4] This block diagram shows an example of the functional configuration of a medical image analysis device according to the first embodiment. [Figure 5] This figure illustrates the first analysis process using the trained models according to the first to third embodiments. [Figure 6] It is a diagram showing an example of a first analysis result display screen according to the first and second embodiments. [Figure 7] It is a diagram showing an example of a second analysis result display screen according to the first and second embodiments. [Figure 8] It is a flowchart showing an example of medical image analysis processing according to the first embodiment. [Figure 9] It is a diagram for explaining a second analysis process using a learned model. [Figure 10] It is a diagram showing an example of a related diagnosis name table according to a modification example. [Figure 11] It is a diagram showing an example of a first analysis result display screen according to a modification example. [Figure 12] It is a diagram showing an example of a second analysis result display screen according to a modification example. [Figure 13] It is a block diagram showing an example of a functional configuration of a medical image analysis device according to the second embodiment. [Figure 14] It is a diagram for explaining the input of findings by a user. [Figure 15] It is a flowchart showing an example of medical image analysis processing according to the second embodiment. [Figure 16] It is a block diagram showing an example of a functional configuration of a medical image analysis device according to the third embodiment. [Figure 17] It is a diagram showing an example of a first analysis result display screen according to the third embodiment. [Figure 18] It is a diagram showing an example of a second analysis result display screen according to the third embodiment. [Figure 19] It is a flowchart showing an example of medical image analysis processing according to the third embodiment. [Figure 20] It is a diagram showing an example of a related findings table. [Figure 21] It is a block diagram showing an example of a functional configuration of a medical image analysis device according to the fourth embodiment. [Figure 22] It is a diagram for explaining an analysis process using a learned model according to the fourth embodiment. [Figure 23]This is a diagram to explain how findings can be revised. [Figure 24] This is a flowchart showing an example of medical image analysis processing according to the fourth embodiment. [Figure 25] This figure shows an example of the second analysis result display screen related to a modified example. [Figure 26] This flowchart shows an example of medical image analysis processing related to a modified image. [Modes for carrying out the invention]

[0023] Hereinafter, with reference to the drawings, examples of embodiments for carrying out the technology of this disclosure will be described in detail.

[0024] [First Embodiment] First, with reference to Figure 1, the configuration of Medical Information System 1, to which the medical image analysis device relating to the disclosed technology is applied, will be explained. Medical Information System 1 is a system for taking images of the diagnostic target area of ​​a subject and storing the medical images obtained from the images, based on examination orders from physicians in clinical departments using a known ordering system. Medical Information System 1 is also a system for radiologists to interpret medical images and create interpretation reports, and for physicians in the requesting clinical departments to view interpretation reports and perform detailed observations of the medical images being interpreted.

[0025] As shown in Figure 1, the medical information system 1 according to this embodiment includes multiple imaging devices 2, multiple image interpretation workstations (WS) 3 which are image interpretation terminals, clinical department WS4, and The system includes an image server 5, an image database (DataBase: DB) 6, a radiographic report server 7, and a radiographic report DB 8. The imaging device 2, radiographic workstation 3, clinical department workstation 4, image server 5, and radiographic report server 7 are connected to each other via a wired or wireless network 9, enabling them to communicate with one another. In addition, image DB 6 is connected to image server 5, and radiographic report DB 8 is connected to radiographic report server 7.

[0026] The imaging device 2 is a device that generates a medical image representing the diagnostic target area by imaging the diagnostic target area of ​​the subject. The imaging device 2 is, for example, a simple X-ray imaging device, an endoscope device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, and P An ET (Positron Emission Tomography) device may also be used. The medical images generated by the imaging device 2 are transmitted to and stored in the image server 5.

[0027] The Department WS4 is a computer used by physicians in a clinical department for detailed observation of medical images, viewing of image interpretation reports, and creation of electronic medical records. In the Department WS4, the creation of patient electronic medical records, requests for image viewing from the image server 5, and the display of medical images received from the image server 5 are performed by executing software programs for each process. In addition, the Department WS4 performs processes such as automatic detection or highlighting of disease-prone areas in medical images, requests for viewing of image interpretation reports from the image interpretation report server 7, and the display of image interpretation reports received from the image interpretation report server 7, by executing software programs for each process.

[0028] Image server 5 incorporates a software program that provides database management system (DBMS) functionality to a general-purpose computer. When image server 5 receives a request to register a medical image from imaging device 2, it formats the medical image into a database format and registers it in image DB6.

[0029] Image DB6 registers image data representing medical images acquired by imaging device 2, and associated information attached to the image data. The associated information includes, for example, an image ID (identification) to identify individual medical images, a patient ID to identify the patient being photographed, an examination ID to identify the examination content, and a unique ID (UID: unique identification) assigned to each medical image. Furthermore, the associated information also includes information about the medical image. The generated data includes the date and time of the examination, the type of imaging device used to acquire the medical image, patient information (e.g., patient's name, age, and sex), the examination site (i.e., the imaging site), imaging information (e.g., imaging protocol, imaging sequence, imaging method, imaging conditions, and whether or not contrast agent was used), and information such as the series number or acquisition number when multiple medical images are acquired in a single examination. Furthermore, when the image server 5 receives a viewing request from the image interpretation WS3 via the network 9, it searches the image database 6 for medical images and sends the retrieved medical images to the requesting image interpretation WS3.

[0030] The image interpretation report server 7 incorporates a software program that provides DBMS functionality to a general-purpose computer. When the image interpretation report server 7 receives a registration request for an image interpretation report from the image interpretation WS3, it formats the image interpretation report into a database format and registers it in the image interpretation report database 8. Furthermore, when it receives a search request for an image interpretation report, it searches for that report in the image interpretation report database 8.

[0031] The image interpretation report DB8 stores image interpretation reports that include information such as an image ID to identify the medical image being interpreted, a radiologist ID to identify the radiologist who performed the interpretation, the name of the lesion, the location of the lesion, findings, and the confidence level of the findings.

[0032] Network 9 is a wired or wireless local area network that connects various devices within the hospital. If the image interpretation WS3 is installed in another hospital or clinic, Network 9 may be configured to connect the local area networks of each hospital via the Internet or a dedicated line. In either case, it is preferable that Network 9 be configured to enable high-speed transfer of medical images, such as through an optical network.

[0033] The image interpretation WS3 performs the following: requests to view medical images from the image server 5, various image processing on medical images received from the image server 5, display of medical images, analysis processing of medical images, highlighting of medical images based on the analysis results, and creation of image interpretation reports based on the analysis results. In addition, the image interpretation WS3 assists in the creation of image interpretation reports, requests registration and viewing of image interpretation reports from the image interpretation report server 7, and displays image interpretation reports received from the image interpretation report server 7. The image interpretation WS3 performs each of the above processes by executing software programs for each process. The image interpretation WS3 incorporates a medical image analysis device 10, which will be described later, and processes other than those performed by the medical image analysis device 10 are performed by well-known software programs, so a detailed explanation is omitted here. Alternatively, instead of performing processes other than those performed by the medical image analysis device 10 in the image interpretation WS3, a separate computer that performs such processes may be connected to the network 9, and that computer may perform the requested processing in response to processing requests from the image interpretation WS3. The following provides a detailed explanation of the medical image analysis device 10 included in the image interpretation WS3.

[0034] Next, with reference to Figure 2, the hardware configuration of the medical image analysis device 10 according to this embodiment will be described. As shown in Figure 2, the medical image analysis device 10 includes a CPU (Central Processing Unit) 20, a memory 21 as a temporary storage area, and a non-volatile storage unit 22. The medical image analysis device 10 includes a display 23 such as an LCD display, input devices 24 such as a keyboard and mouse, and a network I / F (Interface) connected to the network 9. )25 is included. The CPU 20, memory 21, storage unit 22, display 23, input device 24, and network I / F 25 are connected to the bus 27.

[0035] The storage unit 22 is implemented by an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, etc. The medical image analysis program 30 is stored in the storage unit 22 as a storage medium. The CPU 20 reads the medical image analysis program 30 from the storage unit 22, expands it into memory 21, and executes the expanded medical image analysis program 30.

[0036] Furthermore, the memory unit 22 stores a related diagnostic name table 32. Figure 3 shows an example of the related diagnostic name table 32. As shown in Figure 3, the related diagnostic name table 32 associates a diagnosis with a diagnosis related to that diagnosis. For example, primary lung cancer of the lung field is associated with lymphoma of the lung hilum. This is because primary lung cancer of the lung field and lymphoma of the lung hilum are related, and if one develops, the other may develop.

[0037] Next, with reference to Figure 4, the functional configuration of the medical image analysis device 10 according to this embodiment will be described. As shown in Figure 4, the medical image analysis device 10 includes an acquisition unit 40, a first analysis unit 42, a first output unit 44, a reception unit 46, a second analysis unit 48, and a second output unit 50. The CPU 20 executes the medical image analysis program 30, thereby enabling the acquisition unit 40, the first analysis unit 42, the first output unit 44, the reception unit 46, the second analysis unit 48, and the second output unit 50 to function.

[0038] The acquisition unit 40 acquires the medical image to be diagnosed (hereinafter referred to as the "diagnostic image") from the image server 5 via the network interface 25. In the following explanation, the case in which the diagnostic image is a chest CT image will be used as an example.

[0039] The first analysis unit 42 performs a first analysis process on multiple regions of interest included in the diagnostic image acquired by the acquisition unit 40. Specifically, as the first analysis process, the first analysis unit 42 performs a process to detect abnormal shadows using a trained model M1 for detecting abnormal shadows as an example of a region of interest from the diagnostic image.

[0040] The trained model M1 is constructed using a Convolutional Neural Network (CNN), for example, which takes a medical image as input and outputs information about abnormal shadows contained in that medical image. The trained model M1 is a model trained using machine learning, for example, by using a large number of combinations of medical images containing abnormal shadows, information identifying the region where the abnormal shadow exists in the medical image, and the diagnostic name of that abnormal shadow as training data.

[0041] As an example, as shown in Figure 5, the first analysis unit 42 inputs the image to be diagnosed into the trained model M1. The trained model M1 outputs information representing the region containing abnormal shadows in the input image to be diagnosed, as well as the diagnostic name and confidence level of that abnormal shadow. In this case, the trained model M1 outputs information representing the region containing abnormal shadows whose confidence level of the diagnostic name is equal to or greater than a predetermined threshold TH1 (for example, 0.5). In the example in Figure 5, the dashed rectangle indicates the region containing abnormal shadows, and the diagnostic name and confidence level are shown in the callout. As the analysis result of the first analysis process, the first analysis unit 42 obtains information representing the region containing abnormal shadows in the image to be diagnosed, as well as the diagnostic name and confidence level of that abnormal shadow.

[0042] The first output unit 44 controls the display 23 by outputting the analysis results from the first analysis unit 42 to the display 23. At this time, the first output unit 44 controls the display 23 to display the diagnostic names of the abnormal shadows in descending order of confidence. Figure 6 shows an example of the first analysis result display screen displayed on the display 23 by the control of the first output unit 44. As shown in Figure 6, the first analysis result display screen displays information representing the area in which the abnormal shadow exists and the diagnostic name of that abnormal shadow. In the example in Figure 6, the area in which the abnormal shadow exists is indicated by a dashed rectangle. Also, Figure 6 shows an example where the confidence level for benign tumors is higher than that for primary lung cancer, and the confidence level for cystic lesions is higher than that for lymphoma.

[0043] A user such as a physician selects one diagnosis from among multiple diagnosis names, which are the results of the first analysis process for the first abnormal shadow among the multiple abnormal shadows displayed on the first analysis result display screen. The reception unit 46 accepts the one diagnosis name selected by the user from among multiple diagnosis names, which are the results of the first analysis process for the first abnormal shadow, as input for the first abnormal shadow among the multiple abnormal shadows.

[0044] The second analysis unit 48 performs a second analysis process on the second abnormal shadow related to the first abnormal shadow. Specifically, first, the second analysis unit 48 refers to the related diagnosis name table 32 and obtains a diagnosis name related to one diagnosis name received by the reception unit 46. Then, the second analysis unit 48 performs a second analysis process on the second abnormal shadow obtained as an analysis result by the first analysis unit 42, where the obtained diagnosis name is the diagnosis name. In this embodiment, as the second analysis process, the second analysis unit 48 performs a process to correct the confidence level of the diagnosis name that matches the obtained diagnosis name among the multiple diagnosis names obtained by the first analysis process on the second abnormal shadow. At this time, the second analysis unit 48 corrects the confidence level by a predetermined percentage (for example, 20%).

[0045] The second output unit 50 controls the display on the display 23 by outputting the analysis results of the second abnormal shadow by the second analysis unit 48 to the display 23. At this time, the second output unit 50 controls the display on the display 23 to display the diagnostic names of the second abnormal shadow in descending order of confidence. Figure 7 shows an example of the second analysis result display screen displayed on the display 23 by the control of the second output unit 50. Figure 7 shows an example in which the user selected primary lung cancer from among the multiple diagnostic names of the first abnormal shadow on the left side in the example in Figure 6. Also in Figure 7, the confidence level of lymphoma in the second abnormal shadow on the right side, which is related to primary lung cancer, has been increased by the second analysis unit 48, resulting in a higher confidence level than that of cystic lesion. In this way, the display order of the diagnostic names of the second abnormal shadow is changed according to the user's selection of a diagnostic name for the first abnormal shadow. The user refers to the second display screen and creates medical documents such as a radiology report.

[0046] Furthermore, the second output unit 50 may notify the user that the result of the second analysis process for the second abnormal shadow differs from the result of the first analysis process if the result of the second analysis process for the second abnormal shadow differs from the result of the first analysis process. Specifically, in this case, as shown in Figure 25 as an example, the second output unit 50 controls the display 23 to show that the result of the second analysis process for the second abnormal shadow has changed from the result of the first analysis process because the user has selected one of the diagnostic names for the first abnormal shadow. Figure 25 shows an example in which the user is notified that the display order of the candidate diagnostic names for the second abnormal shadow has changed because the user has selected primary lung cancer as the diagnostic name for the first abnormal shadow. This notification function may also be switchable on and off by the user.

[0047] Next, the operation of the medical image analysis device 10 according to this embodiment will be explained with reference to Figure 8. The medical image analysis process shown in Figure 8 is executed when the CPU 20 executes the medical image analysis program 30. The medical image analysis process shown in Figure 8 is executed, for example, when a user inputs an instruction to start execution.

[0048] In step S10 of Figure 8, the acquisition unit 40 acquires the diagnostic target image from the image server 5 via the network interface 25. In step S12, the first analysis unit 42 performs a first analysis process on multiple regions of interest included in the diagnostic target image acquired in step S10, as described above. In step S14, the first output unit 44 controls the display of the analysis results from step S12 by outputting them to the display 23, as described above.

[0049] In step S16, the reception unit 46 receives one diagnosis name selected by the user from among multiple diagnosis names for the first abnormal shadow among the multiple abnormal shadows displayed in step S14. In step S18, the second analysis unit 48, as described above, refers to the related diagnosis name table 32 and obtains a diagnosis name related to the one diagnosis name received in step S16. Then, as described above, the second analysis unit 48 performs a second analysis process on the second abnormal shadow obtained as an analysis result in step S12, with the obtained diagnosis name being the second analysis processing.

[0050] In step S20, the second output unit 50 controls the display 23 by outputting the analysis result of the second abnormal shadow from step S18 to the display 23, as described above. When the processing in step S20 is completed, the medical image analysis process is completed.

[0051] As described above, this embodiment makes it possible to improve the accuracy of the analysis results of the region of interest.

[0052] The second analysis unit 48 may, as a second analysis process, perform analysis on the second abnormal shadow based on a partial image containing the second abnormal shadow of the image to be diagnosed, one diagnosis name received by the reception unit 46, and a pre-trained model M2 that has been trained in advance using training partial images and training data including the diagnosis names of the abnormal shadows contained in the training partial images. In this case, as an example shown in Figure 9, the second analysis unit 48 inputs the one diagnosis name received by the reception unit 46 and the partial image containing the second abnormal shadow of the image to be diagnosed into the pre-trained model M1. The pre-trained model M2 outputs the diagnosis name and confidence level of the diagnosis name of the second abnormal shadow contained in the input partial image. The second analysis unit 48 obtains the diagnosis name and confidence level of the diagnosis name of the second abnormal shadow as the analysis result of the second analysis process.

[0053] Furthermore, if there are multiple diagnostic names related to a single diagnostic name received by the reception unit 46 in the related diagnostic name table 32, the second analysis unit 48 may select one diagnostic name from among the multiple diagnostic names related to a single diagnostic name based on the co-occurrence probability between that single diagnostic name and each of the multiple diagnostic names related to that single diagnostic name. In this case, for example, the second analysis unit 48 selects the diagnostic name with the highest co-occurrence probability among the multiple diagnostic names related to the single diagnostic name. In this example configuration, the co-occurrence probabilities may also be stored in the related diagnostic name table 32.

[0054] Furthermore, not only diagnostic names but also combinations of diagnostic names and findings may be associated with diagnostic names. An example of the associated diagnostic name table 32 in this case is shown in Figure 10. In this case, as shown in Figure 11 as an example, the diagnostic names and findings are displayed on the first analysis results display screen. Then, as shown in Figure 12 as an example, on the second analysis results display screen, the confidence level of the diagnostic name corresponding to the combination of diagnostic name and findings selected by the user is modified, and that diagnostic name is displayed according to the confidence level.

[0055] Furthermore, the second analysis unit 48 may determine a second abnormal shadow associated with each of the multiple abnormal shadows before the reception unit 46 receives input from the user regarding the first abnormal shadow. In this case, the second analysis unit 48 selects a second abnormal shadow related to the first abnormal shadow from among the second abnormal shadows determined for each of the multiple abnormal shadows, based on the input received by the reception unit 46. Then, the second analysis unit 48 performs a second analysis process on the selected second abnormal shadow.

[0056] Specifically, the second analysis unit 48 refers to the related diagnostic name table 32 and obtains a diagnostic name related to the diagnostic name obtained as an analysis result by the first analysis unit 42 for each of the multiple abnormal shadows. The second analysis unit 48 also determines that the abnormal shadow included in the analysis result by the first analysis unit 42, for each of the multiple abnormal shadows, is the related second abnormal shadow. Next, on the first analysis result display screen, when the user selects a diagnostic name for the first abnormal shadow, the second analysis unit 48 selects the second abnormal shadow from among the second abnormal shadows determined for each of the multiple abnormal shadows, the second abnormal shadow whose selected diagnostic name is included in the analysis result by the first analysis unit 42. Then, the second analysis unit 48 performs a second analysis process on the selected second abnormal shadow.

[0057] Figure 26 shows an example of the flow of medical image analysis processing in this configuration. Steps in Figure 26 that perform the same processing as in Figure 8 are given the same step numbers and their explanations are omitted.

[0058] In step S15 of Figure 26, the second analysis unit 48 refers to the related diagnostic name table 32 and obtains a diagnostic name related to the diagnostic name obtained as an analysis result by the first analysis unit 42 for each of the multiple abnormal shadows. The second analysis unit 48 also determines that the diagnostic name obtained for each of the multiple abnormal shadows is the abnormal shadow included in the analysis result by the first analysis unit 42 as the related second abnormal shadow. Note that the processing in step S15 may be performed after step S12 and before step S14. Also, the processing in step S15 may be performed in parallel with the processing in step S14.

[0059] In step S18D, the second analysis unit 48 selects a second abnormal shadow from among the second abnormal shadows determined in step S15 for each of the multiple abnormal shadows, the second abnormal shadow whose diagnosis name received in step S16 is included in the analysis results from step S12. The second analysis unit 48 then performs a second analysis process on the selected second abnormal shadow.

[0060] [Second Embodiment] A second embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the medical image analysis device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0061] Referring to Figure 13, the functional configuration of the medical image analysis device 10 according to this embodiment will be described. Functional parts having the same functions as the medical image analysis device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 13, the medical image analysis device 10 includes an acquisition unit 40, a first analysis unit 42, a first output unit 44, a reception unit 46A, a second analysis unit 48A, a second output unit 50A, and an extraction unit 52. The CPU 20 executes the medical image analysis program 30, thereby enabling the acquisition unit 40, the first analysis unit 42, the first output unit 44, the reception unit 46A, the second analysis unit 48A, the second output unit 50A, and the extraction unit 52 to function.

[0062] In this embodiment, the user, on the first analysis result display screen shown in Figure 6, selects a first abnormal shadow from among several abnormal shadows, as shown in Figure 14 as an example, and inputs a statement of observations regarding the selected first abnormal shadow. The reception unit 46A receives the statement of observations regarding the first abnormal shadow, which was input by the user, as input for the first abnormal shadow.

[0063] The extraction unit 52 extracts the diagnostic name of the first abnormal shadow from the findings statement received by the reception unit 46A. For this extraction, known techniques such as natural language processing using a recurrent neural network or matching with a pre-prepared word dictionary of diagnostic names can be used.

[0064] The second analysis unit 48A selects a diagnostic name from among multiple diagnostic names, which are the results of the first analysis process performed by the first analysis unit 42 regarding the first abnormal shadow, that matches the diagnostic name extracted by the extraction unit 52. Next, the second analysis unit 48A refers to the related diagnostic name table 32 and obtains diagnostic names related to the selected diagnostic name. Then, the second analysis unit 48 performs a second analysis process on the second abnormal shadow obtained as an analysis result by the first analysis unit 42, with the obtained diagnostic name being the same process. The second analysis process is the same as in the first embodiment, so its explanation is omitted.

[0065] The second output unit 50A, similar to the second output unit 50 in the first embodiment, controls the display on the display 23 by outputting the analysis result of the second abnormal shadow by the second analysis unit 48A to the display 23. That is, in this embodiment, in the example of Figure 6, when the user inputs the findings statement shown in Figure 14 for the first abnormal shadow on the left, the display order of the diagnostic names of the second abnormal shadow is changed based on that findings statement, as shown in Figure 7.

[0066] Next, the operation of the medical image analysis device 10 according to this embodiment will be explained with reference to Figure 15. The medical image analysis process shown in Figure 15 is executed when the CPU 20 executes the medical image analysis program 30. The medical image analysis process shown in Figure 15 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 15 that perform the same process as in Figure 8 are given the same step numbers and their explanations are omitted.

[0067] In step S16A of Figure 15, the reception unit 46A receives the findings statement entered by the user for the first abnormal shadow among the multiple abnormal shadows displayed in step S14. In step S17, the extraction unit 52 extracts the diagnostic name of the first abnormal shadow from the findings statement received in step S16A.

[0068] In step S18A, the second analysis unit 48A selects a diagnosis name that matches the diagnosis name extracted in step S17 from among multiple diagnosis names, which are the multiple analysis results of the first analysis process performed in step S12 regarding the first abnormal shadow. Next, the second analysis unit 48A refers to the related diagnosis name table 32 and obtains a diagnosis name related to the selected diagnosis name. Then, the second analysis unit 48 performs a second analysis process on the second abnormal shadow obtained as an analysis result in step S12, with the obtained diagnosis name being the diagnosis name.

[0069] In step S20A, the second output unit 50A controls the display 23 by outputting the analysis result of the second abnormal shadow from step S18A to the display 23. When the processing in step S20A is completed, the medical image analysis process is completed.

[0070] As described above, this embodiment can achieve the same effects as the first embodiment.

[0071] [Third Embodiment] A third embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the medical image analysis device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0072] Referring to Figure 16, the functional configuration of the medical image analysis device 10 according to this embodiment will be described. Functional parts having the same functions as the medical image analysis device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 16, the medical image analysis device 10 includes an acquisition unit 40, a first analysis unit 42, a first output unit 44B, a reception unit 46B, a second analysis unit 48B, a second output unit 50B, and an extraction unit 52B. The CPU 20 executes the medical image analysis program 30, thereby enabling the acquisition unit 40, the first analysis unit 42, the first output unit 44B, the reception unit 46B, the second analysis unit 48B, the second output unit 50B, and the extraction unit 52B to function.

[0073] The first output unit 44B controls the display 23 by outputting the analysis results from the first analysis unit 42 to the display 23. In this embodiment, the first output unit 44B controls the display 23 to show information representing abnormal shadows. Figure 17 shows an example of the first analysis result display screen shown on the display 23 by the control of the first output unit 44B. As shown in Figure 17, in the first analysis result display screen according to this embodiment, the diagnostic name of the abnormal shadow is not displayed on the display 23. However, the first output unit 44B may also control the display 23 to show the diagnostic name of the abnormal shadow, similar to the first output unit 44.

[0074] The user specifies a first abnormal shadow on the first analysis result display screen shown in Figure 17 and inputs a statement of observations regarding the specified first abnormal shadow. The reception unit 46B, similar to the reception unit 46A in the second embodiment, receives the statement of observations regarding the first abnormal shadow entered by the user as input for the first abnormal shadow.

[0075] The extraction unit 52B, similar to the extraction unit 52 in the second embodiment, extracts the diagnostic name of the first abnormal shadow from the findings document received by the reception unit 46B.

[0076] The second analysis unit 48B performs the first analysis process by changing the analysis parameters for diagnostic names related to the diagnostic names extracted by the extraction unit 52B as the second analysis process. Specifically, the second analysis unit 48B refers to the related diagnostic name table 32 and obtains diagnostic names related to the diagnostic names extracted by the extraction unit 52B. Then, the second analysis unit 48B performs the first analysis process by changing the analysis parameters for the obtained diagnostic names. In this embodiment, the second analysis unit 48B performs the first analysis process by changing the threshold TH1 used for comparison with the confidence level of the obtained diagnostic names as an analysis parameter to a smaller value than that used in the analysis by the first analysis unit 42.

[0077] In other words, the second analysis unit 48B changes the threshold TH1 used for comparison with the confidence level of the acquired diagnostic name to a smaller value than that used in the analysis process by the first analysis unit 42, and then inputs the image to be diagnosed into the trained model M1. The trained model M1 outputs information representing the region in which abnormal shadows exist in the input image to be diagnosed, the diagnostic name of the abnormal shadow, and the confidence level of the diagnostic name. At this time, the trained model M1 outputs information representing the region in which abnormal shadows exist whose confidence level of the diagnostic name is greater than or equal to the threshold TH1. Therefore, compared to the analysis process by the first analysis unit 42, the detection sensitivity of diagnostic names related to the diagnostic name extracted by the extraction unit 52B is increased.

[0078] For example, if the second analysis unit 48B wants to lower the detection sensitivity of diagnostic names with low co-occurrence probability, it may change the threshold TH1 to a larger value than that used during the analysis process by the first analysis unit 42.

[0079] The second output unit 50B, similar to the first output unit 44B, controls the display 23 by outputting the analysis results from the second analysis unit 48B to the display 23. Figure 18 shows an example of the second analysis result display screen shown on the display 23 by the control of the second output unit 50B. In Figure 18, an abnormal shadow enclosed by the dashed rectangle on the right, which was not detected in the example of Figure 17, is newly detected by the analysis processing of the second analysis unit 48B.

[0080] Next, the operation of the medical image analysis device 10 according to this embodiment will be explained with reference to Figure 19. The medical image analysis process shown in Figure 19 is executed when the CPU 20 executes the medical image analysis program 30. The medical image analysis process shown in Figure 19 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 19 that perform the same process as in Figure 8 are given the same step numbers and their explanations are omitted.

[0081] In step S14B of Figure 19, the first output unit 44B controls the display 23 by outputting the analysis results from step S12 to the display 23. At this time, the first output unit 44B controls the display 23 to display information representing the abnormal shadow. In step S16B, the reception unit 46B receives the findings statement entered by the user for the first abnormal shadow displayed in step S14B. In step S17B, the extraction unit 52B extracts the diagnostic name of the first abnormal shadow from the findings statement received in step S16B.

[0082] In step S18B, the second analysis unit 48B, as described above, performs the first analysis process by changing the analysis parameters for the diagnostic names related to the diagnostic names extracted in step S17B as the second analysis process. In step S20B, the second output unit 50B controls the display on the display 23 by outputting the analysis results from step S18B to the display 23. When the processing in step S20B is completed, the medical image analysis process is completed.

[0083] As described above, this embodiment can achieve the same effects as the first embodiment.

[0084] [Fourth Embodiment] A fourth embodiment of the disclosed technology will now be described. Note that the configuration of the medical information system 1 and the hardware configuration of the medical image analysis device 10 according to this embodiment are the same as those of the first embodiment, and therefore will not be described.

[0085] In this embodiment, a related findings table 34 is stored in the storage unit 22 instead of the related diagnostic name table 32. Figure 20 shows an example of the related findings table 34. As shown in Figure 20, the related findings table 34 associates findings with findings related to those findings. For example, spicula+ is associated with lobulated+. This is because spicula+ and lobulated+ have a relatively high probability of co-occurrence.

[0086] Referring to Figure 21, the functional configuration of the medical image analysis device 10 according to this embodiment will be described. Functional parts having the same functions as the medical image analysis device 10 according to the first embodiment are denoted by the same reference numerals and their description is omitted. As shown in Figure 21, the medical image analysis device 10 includes an acquisition unit 40, a first analysis unit 42C, a first output unit 44C, a reception unit 46C, a second analysis unit 48C, and a second output unit 50C. The CPU 20 executes the medical image analysis program 30, thereby enabling the acquisition unit 40, the first analysis unit 42C, the first output unit 44C, the reception unit 46C, the second analysis unit 48C, and the second output unit 50C to function.

[0087] The first analysis unit 42C performs analysis processing on the region of interest included in the diagnostic target image acquired by the acquisition unit 40. Specifically, first, the first analysis unit 42C, similar to the first analysis unit 42 in the first embodiment, uses the trained model M1 to detect abnormal shadows as an example of a region of interest included in the diagnostic target image.

[0088] Next, the first analysis unit 42C performs an analysis process to derive findings using a pre-trained model M3 for deriving findings from images containing abnormal shadows. The pre-trained model M3 is composed of a CNN that takes an image containing abnormal shadows as input and outputs findings related to the abnormal shadows contained in that image. The pre-trained model M3 is a model trained by machine learning using, for example, a large number of combinations of images containing abnormal shadows and findings of the abnormal shadows in those images as training data.

[0089] As an example, as shown in Figure 22, the first analysis unit 42C inputs a partial image of the region containing the abnormal shadow in the image to be diagnosed into the trained model M3. The trained model M3 outputs findings of the abnormal shadow contained in the input partial image. In this case, the trained model M3 outputs findings with a confidence level of TH2 (for example, 0.5) or higher. Figure 22 shows an example in which five findings are output from the trained model M3.

[0090] The first output unit 44C controls the display 23 by outputting the findings derived by the first analysis unit 42C to the display 23 as the analysis result of the first analysis unit 42C.

[0091] The user modifies the findings displayed on the display 23 as needed. The reception unit 46C receives input from the user regarding the modifications to the findings.

[0092] The second analysis unit 48C performs analysis processing by changing the analysis parameters for findings related to the corrected findings based on the input received by the reception unit 46C. Specifically, the second analysis unit 48C refers to the related findings table 34 and obtains findings related to the corrected findings. Then, the second analysis unit 48C performs analysis processing by changing the analysis parameters for the obtained findings. In this embodiment, the second analysis unit 48C performs analysis processing by changing the threshold TH2 used for comparison with the confidence level of the obtained findings as an analysis parameter to a smaller value than that used in the analysis by the first analysis unit 42C.

[0093] Specifically, the second analysis unit 48C changes the threshold TH2 used for comparison with the confidence level of the acquired findings to a smaller value than that used in the analysis process by the first analysis unit 42C, and then inputs a partial image of the region containing the abnormal shadow in the image to be diagnosed into the trained model M3. The trained model M3 outputs findings of the abnormal shadow contained in the input partial image. At this time, the trained model M3 outputs information representing findings whose confidence level is equal to or greater than the threshold TH2. Therefore, compared to the analysis process by the first analysis unit 42C, the detection sensitivity of findings related to the corrected findings is increased.

[0094] The second output unit 50C, similar to the first output unit 44C, controls the display 23 by outputting the findings derived by the second analysis unit 48C to the display 23 as the analysis result of the second analysis unit 48C.

[0095] As an example, let's assume that five findings are derived by the first analysis unit 42C, as shown in Figure 23. Next, let's assume that one of the five findings is modified by the user. Figure 23 shows an example where "unclear margin" is modified to "spicule+". In this case, the second analysis unit 48C increases the detection sensitivity of "lobular+", which is a finding related to the modified finding "spicule+", and performs the findings derivation process. Figure 23 shows an example where "lobular+", which was not derived by the analysis process of the first analysis unit 42C, is derived by the analysis process of the second analysis unit 48C.

[0096] Next, the operation of the medical image analysis device 10 according to this embodiment will be explained with reference to Figure 24. The medical image analysis process shown in Figure 24 is executed when the CPU 20 executes the medical image analysis program 30. The medical image analysis process shown in Figure 24 is executed, for example, when a user inputs an instruction to start execution. Steps in Figure 24 that perform the same process as in Figure 8 are given the same step numbers and their explanation is omitted.

[0097] In step S12C of Figure 24, the first analysis unit 42C, as described above, uses the trained model M1 to detect abnormal shadows in the diagnostic image acquired in step S10. Then, as described above, the first analysis unit 42C performs a processing to derive findings using the trained model M3 for deriving findings from the image containing the detected abnormal shadows as part of the analysis process.

[0098] In step S14C, the first output unit 44C controls the display 23 by outputting the findings derived in step S12C to the display 23 as the analysis result of step S12C. In step S16C, the reception unit 46C receives input from the user for corrections to the findings displayed in step S14C.

[0099] In step S18C, the second analysis unit 48C, as described above, modifies the analysis parameters for findings related to the modified findings based on the input received in step S16C and performs analysis processing. In step S20C, the second output unit 50C controls the display 23 by outputting the findings derived in step S18C to the display 23 as the analysis result of step S18C. When the processing in step S20C is completed, the medical image analysis processing is completed.

[0100] As described above, this embodiment can achieve the same effects as the first embodiment.

[0101] In the first to third embodiments described above, the case in which a diagnostic name for the abnormal shadow is obtained as an analysis result by the first analysis unit 42 has been explained, but the invention is not limited to this. For example, the name of the abnormal shadow, the findings of the abnormal shadow, or a description of the findings of the abnormal shadow may be applied as an analysis result by the first analysis unit 42. Examples of findings in this case include the location, size, transmittance (e.g., solid or ground-glass), presence or absence of spicules, presence or absence of calcification, presence or absence of marginal irregularity, presence or absence of pleural invagination, or presence or absence of chest wall contact in the abnormal shadow. An example of a description of findings is a sentence obtained by inputting multiple findings into a recurrent neural network.

[0102] Furthermore, while the above embodiments described the case where the region of interest is the region of abnormal shadow, the model is not limited to this. The region of interest may also be the region of an organ or the region of an anatomical structure.

[0103] Furthermore, in each of the above embodiments, the hardware structure of the processing unit that performs various processes, such as the various functional parts of the medical image analysis device 10, is as follows: The following types of processors can be used. As mentioned above, in addition to the CPU, a general-purpose processor that executes software (programs) and functions as various processing units, FPGAs (Field Programmable Gate Arrays) are also available. Programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacturing, and ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations specifically designed to perform particular processes. This includes dedicated electrical circuits, etc.

[0104] A single processing unit may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor.

[0105] Examples of configuring multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software are combined to form a single processor, as exemplified by client and server computers, and this processor functions as multiple processing units. Secondly, a configuration where multiple processing units are configured with a single processor, as exemplified by a System on Chip (SoC), etc. One configuration uses a processor that implements the functions of the entire system, including the processing units, on a single IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above-mentioned processors as hardware structures.

[0106] Furthermore, the hardware structure of these various processors can, more specifically, utilize electrical circuits that combine circuit elements such as semiconductor devices. .

[0107] Furthermore, although the above embodiment describes a configuration in which the medical image analysis program 30 is pre-stored (installed) in the storage unit 22, the invention is not limited to this configuration. The medical image analysis program 30 may be provided in the form of a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Alternatively, the medical image analysis program 30 may be provided in the form of a download from an external device via a network.

[0108] The disclosures of Japanese Patent Application No. 2021-064397, filed on April 5, 2021, and Japanese Patent Application No. 2021-208524, filed on December 22, 2021, are incorporated herein by reference in their entirety. Furthermore, all documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually indicated as being incorporated by reference.

Claims

1. A medical image analysis device comprising at least one processor, The aforementioned processor, By performing a first analysis process on multiple regions of interest contained in the medical image, analysis results including a diagnostic name corresponding to each of the multiple regions of interest are obtained. Among the aforementioned multiple areas of interest, input is received for the first area of ​​interest. Based on the input and the analysis results of the first analysis process, a first diagnostic name corresponding to the first area of ​​interest is identified. In relation to the first diagnosis, a second diagnosis is identified in the pre-stored related diagnosis information as a diagnosis in which the other may develop if one of them develops. Based on the input, among the plurality of areas of interest, the area of ​​interest that includes the second diagnostic name in the analysis result of the first analysis process is determined to be the second area of ​​interest related to the first area of ​​interest. A second analysis process is performed on the second region of interest determined based on the aforementioned input. Output the analysis results for the second area of ​​interest. Medical image analysis device.

2. The aforementioned processor, As input, the system accepts the selection of one analysis result from among multiple analysis results of the first analysis process relating to the first region of interest. As an analysis result related to the aforementioned one analysis result, the analysis result that includes the second diagnostic name corresponding to the first diagnostic name included in the aforementioned one analysis result is subjected to a second analysis process relating to the second area of ​​interest obtained by the first analysis process. The medical image analysis apparatus according to claim 1.

3. The aforementioned processor, As the input, the following is accepted: Based on the aforementioned findings, one analysis result is selected from among the multiple analysis results of the first analysis process relating to the first area of ​​interest. As an analysis result related to the aforementioned one analysis result, the analysis result that includes the second diagnostic name corresponding to the first diagnostic name included in the aforementioned one analysis result is subjected to a second analysis process relating to the second area of ​​interest obtained by the first analysis process. The medical image analysis apparatus according to claim 1.

4. The aforementioned processor, As the second analysis process, the confidence level of an analysis result related to one of the multiple analysis results obtained by the first analysis process concerning the second region of interest is modified. A medical image analysis device according to claim 2 or claim 3.

5. The aforementioned processor, As the second analysis process, the analysis process concerning the second region of interest is performed based on a partial image of the medical image including the second region of interest, the first analysis result, and a pre-trained model that has been trained using training data including a training partial image and the analysis result of the region of interest contained in the training partial image. A medical image analysis device according to claim 2 or claim 3.

6. The aforementioned processor, If there are multiple analysis results related to the aforementioned single analysis result, one analysis result is selected from among the multiple analysis results related to the aforementioned single analysis result based on the co-occurrence probability between the first diagnostic name included in the aforementioned single analysis result and each of the multiple diagnostic names included in the multiple analysis results related to the aforementioned single analysis result. A medical image analysis device according to any one of claims 2 to 5.

7. The aforementioned processor, As the second analysis process, the first analysis process is performed by changing the analysis parameters for the analysis results related to the analysis results of the first region of interest. The medical image analysis apparatus according to claim 1.

8. The aforementioned processor, If the result of the second analysis process with respect to the second region of interest differs from the result of the first analysis process, the system will notify that the result of the second analysis process differs from the result of the first analysis process. A medical image analysis device according to any one of claims 1 to 7.

9. The aforementioned processor, Before receiving the aforementioned input, determine the second region of interest associated with each of the multiple regions of interest, Based on the input, a second region of interest related to the first region of interest is selected from among the second regions of interest determined for each of the plurality of regions of interest. Perform the second analysis process on the selected second region of interest. A medical image analysis device according to any one of claims 1 to 8.

10. By performing a first analysis process on multiple regions of interest contained in the medical image, analysis results including a diagnostic name corresponding to each of the multiple regions of interest are obtained. Among the aforementioned multiple areas of interest, input is received for the first area of ​​interest. Based on the input and the analysis results of the first analysis process, a first diagnostic name corresponding to the first area of ​​interest is identified. In relation to the first diagnosis, a second diagnosis is identified in the pre-stored related diagnosis information as a diagnosis in which the other may develop if one of them develops. Based on the input, among the plurality of areas of interest, the area of ​​interest that includes the second diagnostic name in the analysis result of the first analysis process is determined to be the second area of ​​interest related to the first area of ​​interest. A second analysis process is performed on the second region of interest determined based on the aforementioned input. Output the analysis results for the second area of ​​interest. A medical image analysis method in which the processing is performed by a processor installed in a medical image analysis device.

11. By performing a first analysis process on multiple regions of interest contained in the medical image, analysis results including a diagnostic name corresponding to each of the multiple regions of interest are obtained. Among the aforementioned multiple areas of interest, input is received for the first area of ​​interest. Based on the input and the analysis results of the first analysis process, a first diagnostic name corresponding to the first area of ​​interest is identified. In relation to the first diagnosis, a second diagnosis is identified in the pre-stored related diagnosis information as a diagnosis in which the other may develop if one of them develops. Based on the input, among the plurality of areas of interest, the area of ​​interest that includes the second diagnostic name in the analysis result of the first analysis process is determined to be the second area of ​​interest related to the first area of ​​interest. A second analysis process is performed on the second region of interest determined based on the aforementioned input. Output the analysis results for the second area of ​​interest. A medical image analysis program that causes the processor in a medical image analysis device to perform the processing.

Images