Image processing device, method and program

Abstract

In the present invention, a processor: sets, in a medical image, a first region including the entirety of a target organ; sets, within the first region, a plurality of small regions including the target organ; derives a first evaluation value suggesting the presence / absence of an abnormality in the first region; derives a second evaluation value suggesting the presence / absence of an abnormality in each of the plurality of small regions; and derives, from the first evaluation value and the second evaluation value, a third evaluation value suggesting the presence / absence of an abnormality in the medical image.

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, a method, and a program. [Background technology]

[0002] In recent years, advances in medical equipment such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) have made it possible to perform image diagnosis using higher quality, higher resolution medical images. In addition, computer-aided diagnosis (CAD) has been put into practical use, which analyzes medical images to derive information such as the probability of the presence and location of lesions and presents this information to doctors, including radiologists.

[0003] However, even in medical images of patients with lesions, only a small amount of abnormality may be detected. For such cases, methods have been proposed to divide the target organ into small regions and use the evaluation results of those regions to consider localized information, such as the characteristics and shape changes localized around the lesion. For example, in "Liu, Kao-Lang, et al. "Deep learning to distinguish pancreatic cancer tissue from non-cancerous pancreatic tissue: a retrospective study with cross-racial external validation," The Lancet Digital Health 2.6 (2020): e303-e313," a method is proposed in which the target organ is divided into small regions, an evaluation score is calculated for each small region indicating the presence or absence of a lesion, and the evaluation scores are integrated to determine whether the entire target organ is normal or abnormal. In addition, Japanese Patent Application Laid-Open No. 2016-007270 proposes a method of dividing a lesion detected by CAD into multiple small regions, extracting features of each small region, and integrating the features of each small region to extract features of the lesion. Summary of the Invention [Problem to be solved by the invention]

[0004] However, changes in the shape and properties of a target organ that suggest a lesion are not necessarily localized to a small region, and therefore, simply integrating the evaluation values ​​or feature quantities of small regions, as in the method described in the literature by Liu et al. and JP 2016-007270 A, is not sufficient to accurately determine abnormalities in the target organ.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to enable accurate evaluation of abnormalities in target organs. [Means for solving the problem]

[0006] An image processing device according to the present disclosure includes at least one processor, The processor sets a first region including the entire target organ in the medical image; A plurality of small regions including a target organ are set within the first region; deriving a first evaluation value indicating the presence or absence of an abnormality in the first region; deriving a second evaluation value indicating the presence or absence of an abnormality in each of the plurality of small regions; A third evaluation value that suggests the presence or absence of an abnormality in the medical image is derived from the first evaluation value and the second evaluation value.

[0007] In the image processing device according to the present disclosure, the first evaluation value includes at least one of the probability of an abnormality being present in the first region, the position of the abnormality, the shape characteristics of the abnormality, and the property characteristics of the abnormality; the second evaluation value includes at least one of the probability of an anomaly existing in each small region, the position of the anomaly, the shape characteristic of the anomaly, and the property characteristic of the anomaly; The third evaluation value may include at least one of the probability of an abnormality existing in the medical image, the position of the abnormality, the shape characteristics of the abnormality, and the property characteristics of the abnormality.

[0008] In the image processing device according to the present disclosure, the processor may set a plurality of small regions by dividing the first region based on an anatomical structure.

[0009] In the image processing device according to the present disclosure, the processor may set a plurality of small regions based on indirect findings on the target organ.

[0010] In the image processing device according to the present disclosure, the indirect findings may include at least one of atrophy, swelling, stenosis, and dilation occurring in the target organ.

[0011] In addition, in the image processing device according to the present disclosure, the processor sets an axis passing through the target organ, A small region may be set in the target organ along the axis.

[0012] In the image processing device according to the present disclosure, the processor may display an evaluation result based on at least one of the first evaluation value, the second evaluation value, and the third evaluation value on a display.

[0013] In the image processing device according to the present disclosure, the medical image is a tomographic image of the abdomen including the pancreas, The target organ may be the pancreas.

[0014] In the image processing device according to the present disclosure, the processor may set small regions by dividing the pancreas into a head, a body, and a tail.

[0015] The image processing method according to the present disclosure includes setting a first region including the entire target organ in a medical image; A plurality of small regions including a target organ are set within the first region; deriving a first evaluation value indicating the presence or absence of an abnormality in the first region; deriving a second evaluation value indicating the presence or absence of an abnormality in each of the plurality of small regions; A third evaluation value that suggests the presence or absence of an abnormality in the medical image is derived from the first evaluation value and the second evaluation value.

[0016] The image processing method according to the present disclosure may be provided as a program for causing a computer to execute the method. [Effects of the Invention]

[0017] According to the present disclosure, abnormalities in target organs can be evaluated with high accuracy. [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a diagnosis support system to which an image processing device according to an embodiment of the present disclosure is applied; [Figure 2] FIG. 1 is a diagram showing the hardware configuration of an image processing apparatus according to an embodiment of the present invention; [Figure 3] Functional configuration diagram of an image processing apparatus according to this embodiment [Figure 4] FIG. 1 is a diagram illustrating setting of a first region; [Figure 5] FIG. 1 is a diagram illustrating setting of a first region; [Figure 6] A diagram for explaining the setting of small areas [Figure 7] A diagram for explaining the setting of small areas [Figure 8] A diagram for explaining the setting of small areas [Figure 9] A diagram for explaining the setting of small areas [Figure 10] A diagram for explaining the setting of small areas [Figure 11] A diagram for explaining the setting of small areas [Figure 12] A diagram for explaining the setting of small areas [Figure 13] FIG. 10 is a diagram schematically illustrating a derivation model in a first evaluation value derivation unit; [Figure 14] FIG. 10 is a diagram schematically illustrating a derivation model in a third evaluation value derivation unit. [Figure 15] FIG. 10 is a diagram showing the flow of processing performed in this embodiment. [Figure 16] A diagram showing the evaluation result display screen [Figure 17] A diagram showing the evaluation result display screen [Figure 18] A diagram showing the evaluation result display screen [Figure 19] A diagram showing the evaluation result display screen [Figure 20] A flowchart showing the processing performed in this embodiment DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. First, the configuration of a medical information system to which an image processing device according to this embodiment is applied will be described. FIG. 1 is a diagram showing a schematic configuration of the medical information system. In the medical information system shown in FIG. 1, a computer 1 incorporating an image processing device according to this embodiment, an imaging device 2, and an image storage server 3 are connected in a communicable state via a network 4.

[0020] The computer 1 includes an image processing device according to this embodiment and has the image processing program of this embodiment installed. The computer 1 may be a workstation or personal computer operated directly by a doctor making a diagnosis, or may be a server computer connected to either of these via a network. The image processing program is stored in a storage device of the server computer connected to the network or in network storage in an externally accessible state, and is downloaded and installed into the computer 1 used by the doctor upon request. Alternatively, the program may be recorded on a recording medium such as a DVD (Digital Versatile Disc) or CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed into the computer 1 from the recording medium.

[0021] The imaging device 2 is a device that captures an image of a region of a subject to be diagnosed and generates a three-dimensional image representing the region, and specifically includes a CT device, an MRI device, a PET (Positron Emission Tomography) device, etc. The three-dimensional image composed of multiple tomographic images generated by the imaging device 2 is transmitted to and stored in the image storage server 3. In this embodiment, the imaging device 2 is a CT device, and generates a three-dimensional CT image of the subject's chest and abdomen. The acquired CT image may be a contrast-enhanced CT image or a non-contrast-enhanced CT image.

[0022] The image storage server 3 is a computer that stores and manages various data, and is equipped with a large-capacity external storage device and database management software. The image storage server 3 communicates with other devices via a wired or wireless network 4, sending and receiving image data and the like. Specifically, it acquires various data, including image data of 3D images generated by the imaging device 2, via the network, and stores and manages them on a recording medium such as a large-capacity external storage device. The storage format of the image data and communication between devices via the network 4 are based on protocols such as DICOM (Digital Imaging and Communication in Medicine).

[0023] Next, an image processing device according to this embodiment will be described. Fig. 2 is a diagram showing the hardware configuration of the image processing device according to this embodiment. As shown in Fig. 2, the image processing device 20 includes a CPU (Central Processing Unit) 11, non-volatile storage 13, and memory 16 as a temporary storage area. The image processing device 20 also includes a display 14 such as a liquid crystal display, input devices 15 such as a keyboard and a mouse, and a network I / F (Interface) 17 connected to a network 4. The CPU 11, storage 13, display 14, input devices 15, memory 16, and network I / F 17 are connected to a bus 18. The CPU 11 is an example of a processor in the present disclosure.

[0024] The storage 13 is realized by a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 13 as a storage medium stores an image processing program 12. The CPU 11 reads the image processing program 12 from the storage 13, loads it into the memory 16, and executes the loaded image processing program 12.

[0025] Next, the functional configuration of the image processing device according to this embodiment will be described. Fig. 3 is a diagram showing the functional configuration of the image processing device according to this embodiment. As shown in Fig. 3, the image processing device 20 includes an image acquisition unit 21, a first region setting unit 22, a second region setting unit 23, a first evaluation value derivation unit 24, a second evaluation value derivation unit 25, a third evaluation value derivation unit 26, and a display control unit 27. When the CPU 11 executes the image processing program 12, the CPU 11 functions as the image acquisition unit 21, the first region setting unit 22, the second region setting unit 23, the first evaluation value derivation unit 24, the second evaluation value derivation unit 25, the third evaluation value derivation unit 26, and the display control unit 27.

[0026] The image acquisition unit 21 acquires a target image G0 to be processed from the image storage server 3 in response to an instruction from an operator via the input device 15. In this embodiment, the target image G0 is a CT image consisting of multiple tomographic images including the chest and abdomen of a human body, as described above. The target image G0 is an example of a medical image of the present disclosure.

[0027] The first region setting unit 22 sets a first region including the entire target organ in the target image G0. In this embodiment, the target organ is the pancreas. Therefore, the first region setting unit 22 sets a first region including the entire pancreas in the target image G0. Specifically, the first region setting unit 22 may set the entire region of the target image G0 as the first region. Alternatively, the first region setting unit 22 may set the region in the target image G0 where the subject is present as the first region. Alternatively, as shown in FIG. 4, the first region A1 may be set to include the pancreas 30 and its surrounding region. Alternatively, as shown in FIG. 5, only the region of the pancreas 30 may be set as the first region A1. Note that FIGS. 4 and 5 show the setting of the first region A1 for one tomographic image D0 included in the target image G0.

[0028] In order to identify the region of the pancreas in the target image G0, the first region setting unit 22 extracts the pancreas, which is the target organ, from the target image G0. To this end, the first region setting unit 22 has a semantic segmentation model (hereinafter referred to as an SS (Semantic Segmentation) model) that has been machine-learned to extract the pancreas from the target image G0. As is well known, the SS model is a machine-learning model that outputs an output image in which a label representing an extracted object (class) is assigned to each pixel of an input image. In this embodiment, the input image is a tomographic image that constitutes the target image G0, the extracted object is the pancreas, and the output image is an image in which the region of the pancreas is labeled. The SS model is constructed using a convolutional neural network (CNN) such as Residual Networks (ResNet) or U-shaped Networks (U-Net).

[0029] The extraction of the target organ is not limited to using the SS model, and any method for extracting the target organ from the target image G0, such as template matching or threshold processing of the CT value, can be applied.

[0030] The second region setting unit 23 sets a plurality of small regions including the target organ within a first region A1 including the entire target organ (i.e., the pancreas) set in the target image G0 by the first region setting unit 22. For example, when the first region A1 is the entire region of the target image G0 or a region including a subject included in the target image G0, the second region setting unit 23 sets individual organs included in the first region A1, such as the pancreas, liver, spleen, and kidneys, as small regions. When the first region A1 is a region including the pancreas 30 as shown in FIG. 4, the second region setting unit 23 may set individual organs included in the first region A1, such as the pancreas, liver, spleen, and kidneys, as small regions. Alternatively, as shown in FIG. 6, the small regions may be set by dividing the first region A1 into tiles. Even when the first region A1 is the entire region of the target image G0 or a subject region included in the target image G0, the first region A1 may be divided into tiles.

[0031] 5, when the first region A1 is the region of the pancreas 30, the second region setting unit 23 may set a plurality of small regions in the target organ (i.e., the pancreas). For example, the second region setting unit 23 may divide the region of the pancreas, which is the first region A1, into a head, a body, and a tail, and set each of the head, the body, and the tail as a small region.

[0032] FIG. 7 is a diagram illustrating the division of the pancreas into the head, body, and tail. FIG. 7 shows the pancreas as viewed from the front of the human body. In the following description, up, down, left, and right refer to a front view of a standing human body. As shown in FIG. 7 , when viewed from the front, a vein 31 and an artery 32 run parallel to each other in the vertical direction behind the pancreas 30. Anatomically, the pancreas 30 is divided into the head on the left side of the vein 31, the body between the vein 31 and the artery 32, and the tail on the right side of the artery 32. Therefore, in this embodiment, the second region setting unit 23 divides the pancreas 30 into three small regions: the head 33, the body 34, and the tail 35, based on the vein 31 and the artery 32. The boundaries of the head 33, the body 34, and the tail 35 are based on the boundary definitions described in "Pancreatic Cancer Treatment Guidelines, 7th Edition, Supplementary Edition, Edited by the Japan Pancreas Society, p. 12, September 2020." Specifically, the left edge of vein 31 (the right edge of vein 31 when viewed from the front) is the boundary between body 33 and body 34, and the left edge of artery 32 (the right edge of artery 32 when viewed from the front) is the boundary between body 34 and tail 35.

[0033] For segmentation, the second region setting unit 23 extracts veins 31 and arteries 32 near the pancreas 30 in the target image G0. The second region setting unit 23 extracts vascular regions and their centerlines (i.e., central axes) from the region near the pancreas 30 in the target image G0, for example, using the methods described in Japanese Patent Application Laid-Open Nos. 2010-200925 and 2010-220732. This method first calculates the positions and major axis directions of multiple candidate points constituting the vascular centerlines based on the values ​​of voxel data constituting the target image G0. Alternatively, the Hessian matrix is ​​calculated for the target image G0, and the eigenvalues ​​of the calculated Hessian matrix are analyzed to calculate the position information and major axis directions of multiple candidate points constituting the vascular centerlines. Then, feature values ​​representing the likelihood of blood vessels are calculated for voxel data around the candidate points, and whether or not the voxel data represents a blood vessel is determined based on the calculated feature values. This allows the vascular region and its centerline to be extracted from the target image G0. The second region setting unit 23 divides the pancreas 30 into a head 33, a body 34, and a tail 35, using the left edge of the extracted vein 31 and artery 32 (the right edge when the human body is viewed from the front) as a reference.

[0034] The division of the pancreas 30 into the head 33, body 34, and tail 35 is not limited to the above method. For example, the pancreas 30 may be divided into the head 33, body 34, and tail 35 by using a segmentation model that has been machine-learned to extract the head 33, body 34, and tail 35 from the pancreas 30. In this case, the segmentation model may be trained by preparing multiple sets of training data each consisting of a training image including the pancreas and a mask image obtained by dividing the pancreas into the head, body, and tail based on the boundary definition described above.

[0035] Furthermore, setting of small regions for the pancreas 30 is not limited to division into the head 33, body 34, and tail 35. FIG. 8 is a diagram for explaining another example of setting of small regions. Note that FIG. 8 is a diagram of the pancreas 30 viewed from the head side of the human body. In another example, the second region setting unit 23 extracts a central axis 36 extending in the longitudinal direction of the pancreas 30. The central axis 36 can be extracted by a method similar to the method for extracting the center lines of the vein 31 and the artery 32 described above. The second region setting unit 23 may then set small regions for the pancreas 30 by dividing the pancreas 30 into a plurality of small regions at equal intervals along the central axis 36.

[0036] 9, small regions 37A to 37C may be set so as to overlap each other within the pancreas 30, or small regions spaced apart such as small regions 37D and 37E may be set. In this case, the small regions may be set along the central axis 36 of the pancreas 30, or may be set at any position.

[0037] 10, within the pancreas 30, the main pancreatic duct 30A exists along the central axis 36 of the pancreas 30. In a CT image, the main pancreatic duct 30A and the pancreatic parenchyma 30B have different CT values, so the pancreas 30 can be divided into a region of the main pancreatic duct 30A and a region of the pancreatic parenchyma 30B. Therefore, by dividing the pancreas 30 into the main pancreatic duct 30A and the pancreatic parenchyma 30B, the main pancreatic duct 30A and the pancreatic parenchyma 30B may be set as small regions, respectively.

[0038] Alternatively, small regions may be set by dividing only one of the main pancreatic duct 30A and the pancreatic parenchyma 30B along the central axis 36 of the pancreas 30 into a plurality of regions, and the second evaluation value may be derived for each small region.

[0039] 11, further small regions may be set in each of the head 33, body 34, and tail 35 of the pancreas 30. In this case, the sizes of the small regions may be different in the head 33, body 34, and tail 35. In FIG. 11, the sizes of the set small regions decrease in the order of the head 33, body 34, and tail 35.

[0040] The second region setting unit 23 may also set multiple small regions based on indirect findings of the target organ. In this case, the second region setting unit 23 has a derivation model that derives indirect finding information representing indirect findings contained in the target image G0 by analyzing the target image G0. An indirect finding is a finding that represents at least one of the characteristics of the shape and properties of tissue surrounding a tumor that occurs in the pancreas. The term "indirect" in "indirect finding" is used in contrast to "direct" findings that directly link lesions such as tumors to diseases such as cancer.

[0041] Indirect findings that characterize the shape of the tissue surrounding the tumor include partial atrophy and swelling of the pancreatic tissue, and stricture and dilation of the pancreatic duct. Indirect findings that characterize the nature of the tissue surrounding the tumor include fatty replacement of the pancreatic tissue (pancreatic parenchyma) and calcification of the pancreatic tissue.

[0042] The derivation model is a semantic segmentation model, similar to the model that extracts the pancreas from the target image G0. The input image of the derivation model is the target image G0, and the extracted objects are seven classes in total: each part of the pancreas showing the indirect findings of atrophy, swelling, stenosis, dilation, fatty substitution, and calcification, as well as the entire pancreas. The output is an image in which the seven classes are labeled for each pixel of the target image G0.

[0043] When indirect findings are present in the pancreas based on the derived model, the second region setting unit 23 sets multiple small regions in the pancreas based on the indirect findings. For example, as shown in Fig. 12, when a stenosis of the main pancreatic duct 30A is observed near the boundary between the body 34 and tail 35 of the pancreas 30, a small region smaller than the head 33 and tail 35 is set in the body 34 where the stenosis is expected to exist.

[0044] In addition, the setting of the small areas for the pancreas 30 shown in Figures 6 to 12 may be performed when the first area A1 is the entire area of ​​the target image G0, the area where the subject included in the target image G0 is located, or when the first area A1 is set to include the pancreas 30 and its surrounding area as shown in Figure 4.

[0045] The first evaluation value derivation unit 24 derives a first evaluation value E1 that indicates the presence or absence of an abnormality in the first region A1 that the first region setting unit 22 set in the target image G0. To this end, the first evaluation value derivation unit 24 has a derivation model 24A that derives the first evaluation value E1 from the first region A1. The derivation model 24A is constructed using a convolutional neural network, similar to the model that extracts the pancreas from the target image G0 in the first region A1. The input image of the derivation model 24A is an image within the first region A1, and the output, the first evaluation value E1, is at least one of the probability of an abnormality being present in the first region A1, position information of the abnormality, shape characteristics of the abnormality, and property characteristics of the abnormality.

[0046] FIG. 13 is a diagram schematically illustrating a derivation model in the first evaluation value derivation unit. As shown in FIG. 13, the derivation model 24A has convolutional neural networks (CNNs) CNN1 to CNN4 corresponding to the types of first evaluation values ​​E1 to be output. CNN1 derives the probability of an abnormality. CNN2 derives the position information of the abnormality. CNN3 derives the shape characteristics of the abnormality. CNN4 derives the characteristic features of the abnormality. The probability of an abnormality is derived as a numerical value between 0 and 1. The position information of the abnormality is derived as a mask for the abnormality in the first region A1 or a bounding box surrounding the abnormality. The shape characteristics of the abnormality may be a mask or bounding box with a color corresponding to the type of shape of the abnormality, or may be a numerical value representing the probability of each type of shape of the abnormality. Types of shape characteristics include partial atrophy, swelling, stenosis, dilation, and cross-sectional circularity of pancreatic tissue. Note that circularity indicates the degree of shape irregularity or organ deformation. The abnormality feature may be a mask or a bounding box having a color corresponding to the type of abnormality feature, or a numerical value representing the probability of each type of abnormality feature. Examples of the type of feature feature include fatty substitution and calcification of pancreatic tissue.

[0047] The first area A1 is input to the derived model 24A, and the organ area, the sub-areas within the organ, and the area of ​​indirect findings within the first area A1 may be input as auxiliary information to the derived model 24A. This auxiliary information is a mask of the organ area, the sub-areas within the organ, and the area of ​​indirect findings within the first area A1.

[0048] The organ region is the region of the target organ contained in the first region A1 when the entire target image G0 or the region where the subject is present is the first region A1, or when the region includes the pancreas and its surrounding region as shown in FIG. 4. A subregion within an organ is a region obtained by further classifying the region of the target organ contained in the first region A1. For example, when the target organ is the pancreas, the regions of the head, body, and tail correspond to subregions within the organ. An indirect finding region is a region that exhibits indirect findings. For example, when the tail of the pancreas is atrophied, the tail region is the indirect finding region.

[0049] Here, atrophy, swelling, stenosis, and dilation included in the shape features derived by the CNN 3 of the derived model 24A may be regarded as indirect findings in the target organ. Meanwhile, the auxiliary information input to the derived model 24A may include indirect findings. When indirect findings are input as auxiliary information to the derived model 24A, the indirect findings are known, and therefore the derived model 24A may be constructed so as not to derive shape features related to the indirect findings.

[0050] In FIG. 13, the derived model 24A has four CNN1 to CNN4. Therefore, it may be possible to select in advance which CNN to use using the input device 15. Note that the derived model 24A is not limited to having four CNN1 to CNN4. It is sufficient that the derived model 24A has at least one of the four CNN1 to CNN4. When the first region A1 is input, the derived model 24A outputs a first evaluation value E1 according to the selected CNN1 to CNN4.

[0051] The second evaluation value derivation unit 25 derives a second evaluation value E2 indicating the presence or absence of an abnormality in each of the multiple small regions set by the second region setting unit 23. To this end, the second evaluation value derivation unit 25 has a derivation model 25A that derives the second evaluation value E2 from the small regions. Similar to the derivation model 24A of the first evaluation value derivation unit 24, the derivation model 25A is constructed using a convolutional neural network. The derivation model 25A has the same schematic configuration as the derivation model 24A shown in FIG. 13, including the input of auxiliary information, except that the input image is a small region. When a small region is input, the derivation model 25A outputs a second evaluation value E2 corresponding to the selected CNN1 to CNN4. The second evaluation value E2 is at least one of the probability of an abnormality being present in each small region, position information of the abnormality, shape characteristics of the abnormality, and property characteristics of the abnormality.

[0052] The auxiliary information input to the derived model 25A includes organ regions within small regions, sub-regions within organs, and regions of indirect findings.

[0053] Furthermore, the small region from which the second evaluation value E2 is derived is set to include the target organ within the first region. Therefore, the second evaluation value E2, when compared with the first evaluation value E1, indicates the presence or absence of a local abnormality in the target organ included in the target image G0. On the other hand, the first evaluation value E1, when compared with the second evaluation value E2, indicates the presence or absence of a global abnormality in the target image G0.

[0054] The third evaluation value derivation unit 26 derives a third evaluation value E3 indicating the presence or absence of an abnormality in the target image G0 from the first evaluation value E1 and the second evaluation value E2. To this end, the third evaluation value derivation unit 26 has a derivation model 26A that derives the third evaluation value E3 from the first evaluation value E1 and the second evaluation value E2. Similar to the derivation model 24A of the first evaluation value derivation unit 24, the derivation model 26A is constructed using a convolutional neural network. The inputs to the derivation model 26A are the first evaluation value E1 and the second evaluation values ​​E2 for each of the multiple small regions. The output of the derivation model 26A is the third evaluation value E3, which is at least one of the probability of an abnormality in the target image G0, position information of the abnormality, shape characteristics of the abnormality, and property characteristics of the abnormality. Note that the presence or absence of an abnormality may be used as the third evaluation value E3 instead of the probability of an abnormality.

[0055] FIG. 14 is a diagram schematically illustrating a derivation model in the third evaluation value derivation unit. As shown in FIG. 14, the derivation model 26A has CNNs 31 to 34 corresponding to the type of the third evaluation value E3 to be output. Like CNNs 1 to 4 in the derivation model 24A shown in FIG. 13, CNNs 31 to 34 derive the probability of an abnormality, position information of the abnormality, shape characteristics of the abnormality, and property characteristics of the abnormality, respectively. Note that auxiliary information may be input to the derivation model 26A in the same way as the derivation model 24A. Examples of the auxiliary information input to the derivation model 26A include the target image G0, organ regions within the target image G0, sub-regions within the organ, and regions of indirect findings.

[0056] From the above, the processing flow performed by the first area setting unit 22, the second area setting unit 23, the first evaluation value derivation unit 24, the second evaluation value derivation unit 25 and the third evaluation value derivation unit 26 in this embodiment is as shown in Figure 15.

[0057] The display control unit 27 displays the evaluation result based on at least one of the first evaluation value E1, the second evaluation value E2, and the third evaluation value E3 on the display 14. FIG. 16 is a diagram showing a display screen for the evaluation result. As shown in FIG. 16, an evaluation result display screen 50 displays one tomographic image D0 of the target image G0 and an evaluation result 51. In FIG. 16, the evaluation result 51 is the probability of an abnormality included in the third evaluation value E3. In FIG. 16, 0.9 is displayed as the probability of an abnormality.

[0058] Furthermore, the evaluation result display screen 50 displays the tomographic image D0 with the position of the abnormality distinguished from other regions based on the position information of the abnormality included in the third evaluation value E3. In FIG. 16, a first abnormal region 41 is displayed in the head 33 of the pancreas 30, and a second abnormal region 42 is displayed in the tail 35 of the pancreas 30, distinguished from other regions. Specifically, the first abnormal region 41 and the second abnormal region 42 are highlighted by being colored. In FIG. 16, the coloring is indicated by hatching. Here, the first abnormal region 41 is a region identified based on the first evaluation value E1. The second abnormal region 42 is a region identified based on the second evaluation value E2.

[0059] Furthermore, in this embodiment, the operator can switch between displaying the evaluation result based on the first evaluation value E1 and the evaluation result based on the second evaluation value E2 by operating the input device 15. FIG. 17 is a diagram showing a display screen of the evaluation result based on the first evaluation value E1. As shown in FIG. 17, the evaluation result 51 displays 0.8, which is the probability of abnormality, which is the evaluation result based on the first evaluation value E1. Furthermore, only the first abnormal region 41 is highlighted in the tomographic image D0.

[0060] Fig. 18 is a diagram showing a display screen of the evaluation result based on the second evaluation value E2. As shown in Fig. 18, the evaluation result 51 displays 0.9, which is the probability of abnormality, which is the evaluation result based on the second evaluation value E2. Furthermore, only the second abnormal region 42 is highlighted in the tomographic image D0. Note that the displayed second evaluation value E2 is derived for the small region from which the second abnormal region 42 was extracted.

[0061] On the other hand, as shown in Fig. 19, the first abnormal region 41 and the second abnormal region 42 may be highlighted in different colors in the tomographic image D0. In Fig. 19, the first abnormal region 41 is hatched and the second abnormal region 42 is filled in, thereby indicating that the colors are different. In this case, the evaluation result 51 displays all of the first evaluation value E1, the second evaluation value E2, and the third evaluation value E3. The displayed second evaluation value E2 is derived for the small region from which the second abnormal region 42 is extracted.

[0062] The highlighting in the tomographic image D0 may be switched on and off by an instruction from the input device 15.

[0063] Next, the processing performed in this embodiment will be described. Fig. 20 is a flowchart showing the processing performed in this embodiment. First, the image acquisition unit 21 acquires the target image G0 from the storage 13 (step ST1), and the first region setting unit 22 sets a first region A1 including the entire target organ in the target image G0 (step ST2).

[0064] Next, the second region setting unit 23 sets a plurality of small regions in the pancreas, which is the target organ (step ST3). Then, the first evaluation value derivation unit 24 derives a first evaluation value E1 indicating the presence or absence of an abnormality in the first region (step ST4). Furthermore, the second evaluation value derivation unit 25 derives a second evaluation value E2 indicating the presence or absence of an abnormality in each of the plurality of small regions (step ST5). Furthermore, the third evaluation value derivation unit 26 derives a third evaluation value E3 indicating the presence or absence of an abnormality in the target image G0 from the first evaluation value E1 and the second evaluation value E2 (step ST6). Then, the display control unit 27 displays the evaluation result on the display 14 (step ST7), and the process ends.

[0065] As described above, in this embodiment, a first evaluation value E1 indicating the presence or absence of an abnormality in the first region is derived, a second evaluation value E2 indicating the presence or absence of an abnormality in each of the plurality of small regions is derived, and a third evaluation value E3 indicating the presence or absence of an abnormality in the target image G0 is derived from the first evaluation value E1 and the second evaluation value E2. Therefore, it is possible to thoroughly evaluate both abnormalities present globally throughout the target organ and abnormalities present locally in the target organ. Therefore, it is possible to accurately evaluate abnormalities in the target organ.

[0066] Furthermore, by setting the first to third evaluation values ​​E1 to E3 to at least one of the probability of abnormality, the position of the abnormality, the shape characteristics of the abnormality, and the characteristic features of the abnormality for each of the target image G0, the first area A1, and the small area, it is possible to evaluate at least one of the probability of abnormality, the position of the abnormality, the shape characteristics of the abnormality, and the characteristic features of the abnormality in the medical image.

[0067] Furthermore, by dividing the first region based on anatomical structures to set multiple small regions, the second evaluation value E2 can be derived for each anatomical structure included in the first region, thereby enabling evaluation of abnormalities for each anatomical structure.

[0068] Furthermore, by setting multiple small regions based on indirect findings for the target organ, a second evaluation value E2 can be derived for the small regions that cause the indirect findings, allowing evaluation of abnormalities in the small regions that cause the indirect findings.

[0069] In the above embodiment, the third evaluation value derivation unit 26 has a derivation model 26A including a CNN, but this is not limited to this. When the first evaluation value E1 and the second evaluation value E2 represent the probability of an abnormality, the third evaluation value E3 may be derived based on the relationship between the first evaluation value E1 and the second evaluation value E2. For example, when the first evaluation value E1 is greater than a first threshold value Th1 and the number of small regions from which a second evaluation value E2 greater than a second threshold value Th2 is obtained is equal to or greater than a third threshold value, the third evaluation value E3 may be determined to indicate an abnormality.

[0070] In the above embodiment, CNN is used as the SS model of the first region setting unit 22, the derivation model 24A of the first evaluation value derivation unit 24, and the derivation model 26A of the third evaluation value derivation unit 26, but this is not limitative. Models constructed by machine learning techniques other than CNN can also be used.

[0071] In the above embodiment, the target organ is the pancreas, but is not limited to this. In addition to the pancreas, any other organ such as the brain, heart, lungs, and liver can be used as the target organ.

[0072] In the above embodiment, a CT image is used as the target image G0, but this is not limiting. Any image, such as a three-dimensional image such as an MRI image, or a radiological image obtained by simple imaging, can be used as the target image G0.

[0073] Furthermore, in the above embodiment, the following various processors may be used as the hardware structure of processing units that perform various processes, such as the image acquisition unit 21, the first region setting unit 22, the second region setting unit 23, the first evaluation value derivation unit 24, the second evaluation value derivation unit 25, the third evaluation value derivation unit 26, and the display control unit 27. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD), a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0074] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0075] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0076] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0077] 1. Computer 2. Imaging equipment 3. Image storage server 4 Network 11 CPU 12 Memory 13. Storage 14 Display 15 Input Devices 16 memory 20 Image processing device 21 Image acquisition unit 22 First area setting unit 23 Second area setting section 24 First evaluation value derivation unit 25 Second evaluation value derivation unit 26 Third evaluation value derivation unit 27 Display control unit 30 Pancreas 30A Main pancreatic duct 30B Pancreatic parenchyma 31 Veins 32 arteries 33 Head 34 body part 35 Tail 36 Center axis 37A~37E Small area 41 First Anomaly 42 Second Anomaly 50 Evaluation result display screen 51 Evaluation Results D0 Tomographic image G0 Target image

Claims

1. at least one processor; The processor: A first region including the entire pancreas, which is a target organ, is set in a medical image that is a tomographic image of the abdomen including the pancreas; setting a plurality of small regions including the target organ within the first region; deriving a first evaluation value indicating the presence or absence of an abnormality in the first region; deriving a second evaluation value indicating the presence or absence of an abnormality in each of the plurality of small regions; an image processing device that derives a third evaluation value that suggests the presence or absence of an abnormality in the medical image from the first evaluation value and the second evaluation value;

2. the first evaluation value includes at least one of a probability of an anomaly existing in the first region, a position of the anomaly, a shape characteristic of the anomaly, and an attribute characteristic of the anomaly; the second evaluation value includes at least one of the probability of an anomaly existing in each of the small regions, the position of the anomaly, the shape characteristics of the anomaly, and the property characteristics of the anomaly; The image processing device according to claim 1 , wherein the third evaluation value includes at least one of the probability of an abnormality existing in the medical image, the position of the abnormality, the shape characteristics of the abnormality, and the property characteristics of the abnormality.

3. The image processing device according to claim 1 , wherein the processor sets the plurality of small regions based on indirect findings about the target organ.

4. The image processing device according to claim 3 , wherein the indirect findings include at least one of atrophy, swelling, stenosis, and dilation occurring in the target organ.

5. The processor sets an axis passing through the target organ; The image processing device according to claim 1 or 2, wherein the small region is set in the target organ along the axis.

6. The image processing device according to claim 1 , wherein the processor displays an evaluation result based on at least one of the first evaluation value, the second evaluation value, and the third evaluation value on a display.

7. The image processing device according to claim 1 , wherein the processor sets the small regions by dividing the pancreas into a head, a body, and a tail.

8. A medical image is a cross-sectional image of the abdomen including the pancreas, and a first region including the entire pancreas is set in the medical image, setting a plurality of small regions including the target organ within the first region; deriving a first evaluation value indicating the presence or absence of an abnormality in the first region; deriving a second evaluation value indicating the presence or absence of an abnormality in each of the plurality of small regions; an image processing method for deriving a third evaluation value, which indicates the presence or absence of an abnormality in the medical image, from the first evaluation value and the second evaluation value;

9. A method for detecting a pancreatic cancer in a medical image, the medical image being a cross-sectional image of the abdomen including the pancreas, comprising: a step of setting a plurality of small regions including the target organ within the first region; deriving a first evaluation value indicating the presence or absence of an abnormality in the first region; deriving a second evaluation value indicating the presence or absence of an abnormality in each of the plurality of small regions; and deriving a third evaluation value, which suggests the presence or absence of an abnormality in the medical image, from the first evaluation value and the second evaluation value.

Images