Image processing apparatus, method, and program

Abstract

According to the present invention, a processor extracts a tubular tissue region from a medical image including tubular tissue, extracts a lesion region present around the tubular tissue region from the medical image, and corrects the extraction result of the tubular tissue region, if necessary, on the basis of positional information about the lesion region.

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, method, and program.

Background Art

[0002] In recent years, tubular tissues such as the large intestine, small intestine, bronchi, and blood vessels of a patient have been extracted from three-dimensional images taken by a CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, etc., and the three-dimensional images of the extracted tubular tissues have been used for image diagnosis. However, when fat, plaque, or the like adheres to a tubular tissue and the tubular tissue is blocked, the tubular tissue may be segmented and extracted from the three-dimensional image. For this reason, a technique has been proposed in which the segmented part of the tubular tissue extracted from the three-dimensional image is specified, the path of the segmented tubular tissue is specified, and when displaying the three-dimensional image, the specified path is interpolated and displayed (see, for example, Japanese Patent Application Laid-Open No. 2014-064824 and Japanese Patent Application Laid-Open No. 2014-054359).

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, in the techniques described in Japanese Patent Application Laid-Open No. 2014-064824 and Japanese Patent Application Laid-Open No. 2014-054359, since the tubular tissue with the segmented part interpolated is displayed, the tubular tissue is interpolated and displayed even when the tubular tissue is actually segmented due to the presence of a lesion. For this reason, the techniques described in Japanese Patent Application Laid-Open No. 2014-064824 and Japanese Patent Application Laid-Open No. 2014-054359 have a problem that it is difficult to determine whether the extracted tubular tissue is actually segmented.

[0004] The present disclosure has been made in view of the above circumstances, and an object thereof is to improve the extraction accuracy of a tubular tissue in consideration of whether the tubular tissue is actually segmented.

Means for Solving the Problems

[0005] The image processing apparatus according to this disclosure comprises at least one processor, The processor extracts tubular tissue regions from medical images that include tubular tissue. From the aforementioned medical image, the lesion area surrounding the tubular tissue region is extracted. Based on the location information of the lesion area, the extraction result of the tubular tissue area is corrected as necessary.

[0006] Furthermore, in the image processing apparatus described herein, the processor is: Identify the location of the division in the tubular tissue region, Based on the location information of the lesion region, it is determined whether or not to connect the divided portion of the tubular tissue region. If it is determined that the aforementioned divided portions should be connected, the extraction result of the tubular tissue region may be modified to connect the aforementioned divided portions in the tubular tissue region.

[0007] Furthermore, in the image processing apparatus according to this disclosure, the processor may determine whether or not the lesion region exists at the divided portion based on the location information of the lesion region, and if it is determined that the lesion region does not exist at the divided portion, it may determine to connect the divided portion.

[0008] Furthermore, in the image processing apparatus according to this disclosure, the processor may function as a trained model that, when the medical image, the tubular tissue region, and the positional information of the lesion region are input, outputs the medical image in which the extraction result of the tubular tissue region has been corrected, if necessary.

[0009] Furthermore, in the image processing apparatus according to this disclosure, the processor may display the medical image in a manner that allows for the identification of whether or not modifications have been made to the tubular tissue region.

[0010] Furthermore, in the image processing apparatus according to this disclosure, if the processor corrects the extraction result of the tubular tissue region, it may display the medical image in a way that allows identification of the corrected portion in the tubular tissue region.

[0011] Furthermore, in the image processing apparatus according to this disclosure, the processor may modify the path of the tubular tissue region in response to an instruction to modify the path of the tubular tissue region in the medical image.

[0012] Furthermore, in the image processing apparatus according to this disclosure, the tubular tissue is the main pancreatic duct. The aforementioned lesion area may be a lesion area in the pancreas.

[0013] The image processing method disclosed herein extracts tubular tissue regions from medical images containing tubular tissue, From the aforementioned medical image, the lesion area surrounding the tubular tissue region is extracted. Based on the location information of the lesion area, the extraction result of the tubular tissue area is corrected as necessary.

[0014] Furthermore, the image processing method described herein may be provided as a program for causing a computer to execute it. [Effects of the Invention]

[0015] According to this disclosure, the accuracy of extracting tubular tissue can be improved by taking into account whether or not the tubular tissue is actually fragmented. [Brief explanation of the drawing]

[0016] [Figure 1] This figure shows a schematic configuration of a diagnostic support system using an image processing device according to the first embodiment of this disclosure. [Figure 2] This figure shows the schematic configuration of the image processing device according to the first embodiment. [Figure 3] Functional configuration diagram of the image processing apparatus according to the first embodiment. [Figure 4] Figure showing the extraction results of the tubular tissue region. [Figure 5] Figure for explaining identification of disconnection points [Figure 6] Figure for explaining identification of disconnection points [Figure 7] Figure for explaining determination of whether to connect disconnection points [Figure 8] Figure for explaining determination of whether to connect disconnection points [Figure 9] Figure for explaining connection of disconnection points [Figure 10] Figure showing display screen of target image [Figure 11] Figure showing display screen of target image [Figure 12] Flowchart showing processing performed in this embodiment [Figure 13] Functional configuration diagram of image processing apparatus according to second embodiment [Figure 14] Figure showing teacher data for training learned model

Mode for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. First, the configuration of a medical information system to which an image processing apparatus according to this embodiment is applied will be described. FIG. 1 is a diagram showing a schematic configuration of the medical information system. The medical information system shown in FIG. 1 includes a computer 1 incorporating an image processing apparatus according to this embodiment, an imaging device 2, and an image storage server 3, which are connected in a communicable state via a network 4.

[0018] Computer 1 contains the image processing device according to this embodiment, and the image processing program of this embodiment is installed on it. Computer 1 may be a workstation or personal computer directly operated by the physician performing the diagnosis, or it may be a server computer connected to them via a network. The image processing program is stored in a storage device of the server computer connected to the network, or in network storage, in a state that is accessible from the outside, and is downloaded and installed on Computer 1 used by the physician as needed. Alternatively, it may be recorded on a recording medium such as a DVD (Digital Versatile Disc) or CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed on Computer 1 from that recording medium.

[0019] The imaging device 2 is a device that generates a three-dimensional image representing a part of the subject to be diagnosed by imaging that part of the subject. Specifically, it is a CT scanner, an MRI scanner, or a PET (Positron Emission Tomography) scanner. The three-dimensional image, consisting of multiple tomographic images, generated by this imaging device 2 is transmitted to the image storage server 3 and stored. In this embodiment, the imaging device 2 is a CT scanner, and it generates a three-dimensional image of the subject's chest and abdomen using CT imaging. The acquired CT image may be a contrast-enhanced or non-contrast-enhanced CT image.

[0020] The image storage server 3 is a computer that stores and manages various types of data, and is equipped with a large-capacity external storage device and database management software. The image storage server 3 communicates with other devices via a wired or wireless network 4 to send and receive image data, etc. Specifically, it acquires various types of data, including image data of 3D images generated by the imaging device 2, via the network, and stores and manages them on a recording medium such as a large-capacity external storage device. The storage format of the image data and communication between each device via the network 4 are based on protocols such as DICOM (Digital Imaging and Communication in Medicine).

[0021] Next, an image processing apparatus according to this embodiment will be described. Figure 2 is a diagram showing the hardware configuration of the image processing apparatus according to this embodiment. As shown in Figure 2, the image processing apparatus 20 includes a CPU (Central Processing Unit) 11, non-volatile storage 13, and memory 16 as a temporary storage area. The image processing apparatus 20 also includes a display 14 such as a liquid crystal display, input devices 15 such as a keyboard and mouse, and a network I / F (Interface) 17 connected to a network 4. The CPU 11, storage 13, display 14, input devices 15, memory 16, and network I / F 17 are connected to a bus 18. Note that the CPU 11 is an example of a processor in this disclosure.

[0022] The storage 13 is implemented using an HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. The image processing program 12 is stored in the storage 13 as a storage medium. The CPU 11 reads the image processing program 12 from the storage 13, expands it into memory 16, and executes the expanded image processing program 12.

[0023] Next, the functional configuration of the image processing apparatus according to this embodiment will be described. Figure 3 is a diagram showing the functional configuration of the image processing apparatus according to this embodiment. As shown in Figure 3, the image processing apparatus 20 includes an image acquisition unit 21, a tubular tissue extraction unit 22, a lesion extraction unit 23, a segmentation identification unit 24, a determination unit 25, a correction unit 26, and a display control unit 27. When the CPU 11 executes the image processing program 12, the CPU 11 functions as the image acquisition unit 21, the tubular tissue extraction unit 22, the lesion extraction unit 23, the segmentation identification unit 24, the determination unit 25, the correction unit 26, and the display control unit 27.

[0024] The image acquisition unit 21 acquires the target image G0 to be processed from the image storage server 3 based on instructions from the operator via the input device 15. In this embodiment, as described above, the target image G0 is a CT image consisting of multiple tomographic images including the chest and abdomen of the human body. The target image G0 is an example of a medical image in this disclosure.

[0025] The tubular tissue extraction unit 22 extracts tubular tissue regions from the target image. In this embodiment, the tubular tissue extraction unit 22 extracts the region of the main pancreatic duct of the pancreas contained in the target image G0 as a tubular tissue region. To this end, the tubular tissue extraction unit 22 first extracts the pancreas from the target image G0, and then extracts the main pancreatic duct from the extracted pancreas.

[0026] The tubular tissue extraction unit 22 consists of a semantic segmentation model (hereinafter referred to as the SS (Semantic Segmentation) model) that has been machine-trained to extract the pancreas from the target image G0, and an SS model that has been machine-trained to extract the main pancreatic duct. The former is referred to as the first SS model 22A, and the latter as the second SS model 22B. As is well known, each SS model 22A and 22B is a machine learning model that outputs an output image in which each pixel of the input image is labeled to represent the object (class) to be extracted. In this embodiment, the input image of the first SS model 22A is a tomographic image constituting the target image G0, the object to be extracted is the pancreas, and the output image is an image with the pancreatic region labeled. The input image of the second SS model 22B is an image of the extracted pancreatic region, the object to be extracted is the main pancreatic duct, and the output image is an image with the main pancreatic duct region labeled.

[0027] SS models 22A and 22B are constructed using convolutional neural networks (CNNs), such as ResNet (Residual Networks) and U-Net (U-shaped Networks).

[0028] Although the above description assumes that the tubular tissue extraction unit 22 has two SS models 22A and 22B, it is not limited to this. A single SS model may be used to extract the pancreas and the main pancreatic duct from the target image G0. Alternatively, the main pancreatic duct may be extracted directly from the target image G0 without extracting the pancreas.

[0029] Furthermore, the extraction of target organs is not limited to methods using the SS model. Any method for extracting target organs from the target image G0 can be applied, such as template matching or thresholding of CT values.

[0030] Furthermore, the extraction of the main pancreatic duct is not limited to the SS model. Any method for extracting the pancreas from the target image G0 can be applied, such as template matching or thresholding of CT values. Alternatively, the main pancreatic duct may be extracted using the method described in Japanese Patent Publication No. 2014-064824.

[0031] When using the method described in Japanese Patent Publication No. 2014-064824, the tubular tissue extraction unit 22 searches for linear structures by calculating eigenvalues ​​of a 3x3 Hessian matrix for each local region within the pancreas. In regions containing linear structures, one of the three eigenvalues ​​of the Hessian matrix will be close to 0, while the other two will be relatively large. Furthermore, the eigenvectors corresponding to the eigenvalues ​​close to 0 indicate the principal axis direction of the linear structure. The tubular tissue extraction unit 22 utilizes this relationship to determine the likelihood of a linear structure in each local region based on the eigenvalues ​​of the Hessian matrix, and for local regions where a linear structure is identified, it detects its center point as a candidate point.

[0032] Next, the candidate points detected through the search are connected based on a predetermined algorithm. This constructs a tree structure consisting of candidate points and vascular branches connecting them, and this tree structure is extracted as the main pancreatic duct pathway. Subsequently, for each detected candidate point, the contour of the main pancreatic duct (outer wall of the main pancreatic duct) is identified in a cross-section perpendicular to the main pancreatic duct pathway, based on the values ​​of the surrounding voxels (CT values). Shape identification is performed using a known segmentation method such as Graph-Cuts. Through the above processing, the pathway and the main pancreatic duct can be extracted.

[0033] The lesion extraction unit 23 extracts lesion regions, such as tumors, located around the main pancreatic duct, which is a tubular tissue region, within the pancreatic region. To this end, the lesion extraction unit 23 consists of an SS model 23A that has been machine-learned to extract lesion regions around the main pancreatic duct within the pancreatic region. In this embodiment, the input image of the SS model 23A is an image of the pancreatic region in the target image G0, the object to be extracted is a lesion, and the output image is an image with the lesion region labeled. If the lesion extraction unit 23 is unable to extract a lesion region from the pancreatic region, it outputs information indicating that no lesions exist. In this case, the display control unit 27 notifies the user that no lesions exist, as will be described later.

[0034] Here, the tubular tissue extraction unit 22 extracts the main pancreatic duct in the pancreas as a tubular tissue region, as described above. However, in areas where lesions such as tumors exist, the main pancreatic duct 31 may be severed or narrowed in the pancreas 30, as shown in Figure 4, making it impossible to extract the main pancreatic duct. For this reason, the division identification unit 24 identifies the location of the division in the tubular tissue region extracted by the tubular tissue extraction unit 22. In this embodiment, the division location in the main pancreatic duct 31 within the pancreas 30 extracted by the tubular tissue extraction unit 22 is identified.

[0035] To this end, the segmentation identification unit 24 first detects the endpoint of the main pancreatic duct 31 extracted by the tubular tissue extraction unit 22. Then, the segmentation identification unit 24 determines whether there is another endpoint of the main pancreatic duct within a predetermined distance Th0 centered on the detected endpoint, with the pancreatic parenchyma in between. If another endpoint exists within a predetermined distance Th0 centered on the endpoint, with the pancreatic parenchyma in between, the unit considers the main pancreatic duct to be segmented between the two endpoints and identifies the region between the endpoints as the segmentation point.

[0036] Figures 5 and 6 are diagrams illustrating the identification of the rupture point. As shown in Figure 5, if the main pancreatic duct 31 is not ruptured in the pancreas 30, two endpoints P1 and P2 are detected in the main pancreatic duct 31. Figure 5 shows circles 34 and 35, each with a radius of a predetermined distance Th0 centered on endpoints P1 and P2. As shown in Figure 5, no other endpoints of the main pancreatic duct exist within the predetermined distance Th0 centered on endpoints P1 and P2, via the pancreatic parenchyma. Therefore, in cases like that shown in Figure 5, the rupture identification unit 24 determines that there is no rupture point.

[0037] On the other hand, as shown in Figure 6, if the main pancreatic duct 31 is severed in the pancreas 30, four endpoints P1, P2, P3, and P4 are detected in the main pancreatic duct 31. In Figure 6, only a circle 36 with a radius of a predetermined distance Th0 centered on endpoint P3 is shown. As shown in Figure 6, within the predetermined distance Th0 centered on endpoint P3, there is another endpoint P4 of the main pancreatic duct via the pancreatic parenchyma. Therefore, in the case shown in Figure 6, the rupture identification unit 24 identifies the region between endpoint P3 and endpoint P4 as the rupture point.

[0038] If the main pancreatic duct 31 is not severed, the rupture identification unit 24 outputs information to that effect, and the processing of the determination unit 25 and the correction unit 26, described later, is not performed. In this case, the display control unit 27 may notify that the main pancreatic duct 31 is not severed, but it does not have to notify anything.

[0039] The determination unit 25 and the correction unit 26 correct the extraction result of the main pancreatic duct 31 as necessary, based on the location information of the lesion area. First, the determination unit 25 will be explained. The determination unit 25 determines whether or not to connect the divided portion of the main pancreatic duct 31 based on the location information of the lesion area extracted by the lesion extraction unit 23. Specifically, the determination unit 25 determines whether or not a lesion area exists at the divided portion, and if it determines that no lesion area exists at the divided portion, it determines to connect the divided portion.

[0040] In this embodiment, the positional information of the lesion region refers to the coordinates of each pixel within the lesion region. Figure 7 is a diagram illustrating the determination of whether or not to connect the divided sections. For the purpose of determination, the determination unit 25 sets a straight line 37 connecting the endpoints of the main pancreatic duct 31 at the divided section, as shown in Figure 7. The determination unit 25 then determines whether or not the lesion region 38 intersects with the straight line 37. As shown in Figure 7, if the lesion region 38 intersects with the straight line 37, it can be considered that the main pancreatic duct 31 is divided due to the presence of the lesion. On the other hand, as shown in Figure 8, if the lesion region 38 does not intersect with the straight line 37, the main pancreatic duct 31 must exist at the divided section, either due to an error in extracting the main pancreatic duct 31 or because the main pancreatic duct 31 was narrowed and could not be extracted.

[0041] Therefore, the determination unit 25 determines that a lesion exists at the dividing point if the lesion area 38 intersects with the straight line 37, and determines that there is no lesion at the dividing point if the lesion area does not intersect with the straight line 37. If the determination unit 25 determines that there is no lesion at the dividing point, it determines that the dividing point should be connected. If the determination unit 25 determines that a lesion exists at the dividing point, it determines that the dividing point should not be connected.

[0042] Alternatively, as shown in Figure 7, instead of setting a straight line 37 connecting the endpoints P3 and P4 of the main pancreatic duct 31, multiple coordinate values ​​on the centerline of the extracted main pancreatic duct 31 may be used to set a line segment connecting the endpoints P3 and P4 so as to smoothly connect them with the extracted main pancreatic duct 31 by using higher-order interpolation such as spline interpolation. In this case, the determination unit 25 determines whether or not a lesion region exists at the point of division based on whether or not the lesion region 38 intersects with the set line segment.

[0043] The modification unit 26 modifies the extraction result of the main pancreatic duct 31 according to the determination result of the determination unit 25. Specifically, if the determination unit 25 determines that the divided portion should be connected, the modification unit 26 modifies the extraction result of the main pancreatic duct 31 to connect the divided portion in the main pancreatic duct 31. On the other hand, if the determination unit 25 determines that the divided portion should not be connected, the modification unit 26 does not modify the extraction result of the main pancreatic duct 31.

[0044] Figure 9 is a diagram illustrating the connection of the divided sections. The modification unit 26 uses multiple coordinate values ​​on the centerline of the extracted main pancreatic duct 31 to derive a line segment 40 that smoothly connects the endpoints P3 and P4 of the main pancreatic duct 31 to the extracted main pancreatic duct 31 by higher-order interpolation such as spline interpolation. Then, the endpoints P3 and P4 are connected along the derived line segment 40. Alternatively, instead of the line segment 40 obtained by higher-order interpolation, the endpoints P3 and P4 may be connected along a straight line connecting the endpoints P3 and P4.

[0045] The display control unit 27 displays the target image G0 on the display 14 so as to identify whether or not modifications have been made to the main pancreatic duct 31. Specifically, if the extraction result of the main pancreatic duct 31 is modified, the target image G0 is displayed on the display 14 so as to identify the modified area in the main pancreatic duct 31. Figure 10 is a diagram showing the display screen of the target image G0. As shown in Figure 10, the display screen 50 has an image display area 51 for displaying the target image G0. The image display area 51 displays one tomographic image D0 included in the target image G0. The displayed tomographic image D0 can be switched, for example, by scrolling the scroll button of the mouse, which is an input device 15.

[0046] As shown in Figure 10, in the target image G0, the region of the main pancreatic duct 31 is highlighted. In addition, at the point where the main pancreatic duct 31 is divided, a connection region 52 is displayed to indicate that the divided point has been connected. The connection region 52 is highlighted in a way that makes it distinguishable from the region of the pancreas 30. For example, the color of the connection region 52 is highlighted in a different color from that of the main pancreatic duct 31. In Figure 10, the difference in color is shown by hatching the region of the main pancreatic duct 31 and filling in the connection region 42.

[0047] Furthermore, below the image display area 51, the text "Modified" 53 is displayed to indicate that the extraction result of the main pancreatic duct 31 has been corrected. If the determination unit 25 determines that the divided portion does not need to be connected and therefore the extraction result of the main pancreatic duct 31 is not corrected, the text "No Correction" will be displayed. Even if the extraction result of the main pancreatic duct 31 is not corrected, the area of ​​the main pancreatic duct 31 may be highlighted by adding color or other means. In addition, the highlighting may be switched on or off by instructions from the input device 15.

[0048] If a lesion area exists at the point of separation, the separation point will not be connected. However, if a radiologist interprets the target image G0, they may find that although it appears narrowed, the main pancreatic duct 31 is not actually separated. In such cases, the correction unit 26 may be able to correct the extraction result of the main pancreatic duct 31 to connect it, based on instructions from the input device 15. Figure 11 is a diagram illustrating the correction of the extraction result. As shown in Figure 11, the text "No correction" 53 is displayed because a lesion area 38 exists at the point of separation of the main pancreatic duct 31 shown on the display screen 50. However, because a narrowed main pancreatic duct 31A exists above the lesion area 38, the main pancreatic ducts 31 on both sides of the lesion area 38 are not separated.

[0049] In this case, the radiologist selects the correction button 54. This allows drawing on the target image G0 using the input device 15. The radiologist can correct the extraction result of the main pancreatic duct 31 by drawing a line segment along the narrowed main pancreatic duct 31A using the input device 15, thereby connecting the main pancreatic ducts 31 on both sides of the lesion area 38.

[0050] Next, the processing performed in this embodiment will be described. Figure 12 is a flowchart showing the processing performed in this embodiment. First, the image acquisition unit 21 acquires the target image G0 from the storage 13 (step ST1), the tubular tissue extraction unit 22 extracts the target organ, the pancreas, from the target image G0 (step ST2), and further extracts the region of the main pancreatic duct, which is a tubular tissue region in the pancreas (step ST3).

[0051] Next, the lesion extraction unit 23 extracts the lesion region, which is a mass or other area surrounding the main pancreatic duct within the pancreatic region (step ST4). The following explanation assumes that the lesion region has been extracted. Then, the segmentation identification unit 24 identifies the segmentation point of the main pancreatic duct, which is the extracted tubular tissue region (step ST5). If the segmentation point cannot be identified (step ST5: NO), the segmentation identification unit 24 outputs information to that effect and proceeds to step ST12.

[0052] Once the point of separation is identified (Step ST5: YES), the determination unit 25 determines whether or not to connect the point of separation in the main pancreatic duct 31 based on the positional information of the lesion region extracted by the lesion extraction unit 23. That is, the determination unit 25 determines whether or not a lesion region exists at the point of separation (Step ST6). If it determines that no lesion region exists at the point of separation (Step ST6: NO), the determination unit 25 determines to connect the point of separation (Step ST7), and the correction unit 26 corrects the extraction result of the main pancreatic duct 31 so that the point of separation in the main pancreatic duct 31 is connected (Step ST8). Then, the display control unit 27 displays the target image G0 on the display 14 so that the corrected point in the main pancreatic duct 31 can be identified (Step ST9). After this, the radiologist interprets the target image G0.

[0053] On the other hand, if the determination unit 25 determines that a lesion area exists at the separation point (step ST6: YES), the determination unit 25 determines that the separation point should not be connected (step ST10), the display control unit 27 displays the result as uncorrected (step ST11), and the process proceeds to step ST12.

[0054] In step ST12, it is determined whether the correction button 54 was selected. If step ST12 is rejected, the process proceeds to step ST14. If step ST12 is confirmed, the pathway of the main pancreatic duct 31 is corrected according to the radiologist's instruction to correct the pathway of the main pancreatic duct 31 (step ST13). Then, it is determined whether the instruction to terminate has been given (step ST14). If step ST14 is rejected, the process returns to step ST12. If step ST14 is confirmed, the process is terminated.

[0055] Thus, in this embodiment, the extraction result of the tubular tissue region is corrected as necessary based on the location information of the lesion region. For this reason, if the lesion region is located at a point of division in the tubular tissue region, the tubular tissue can be considered to be actually divided by the lesion, and the extraction result of the tubular tissue region can be left uncorrected. On the other hand, if the lesion region is not located at a point of division in the tubular tissue region, the tubular tissue is not actually divided, and it can be considered that the tubular tissue region is divided due to a failure in the extraction of the tubular tissue region, etc., and the extraction result of the tubular tissue region can be corrected. In this way, the extraction accuracy of the tubular tissue can be improved by considering whether or not the tubular tissue is actually divided.

[0056] Furthermore, by displaying the target image G0 in a way that allows identification of whether or not modifications have been made to the tubular tissue region, it is easy to identify that modifications have been made to the tubular tissue region.

[0057] Furthermore, if the extraction results of the tubular tissue region are modified, the modified areas within the tubular tissue region can be easily identified by displaying the target image in a way that allows for identification of the modified areas.

[0058] Furthermore, by correcting the path of the tubular tissue region in response to instructions to correct the path of the tubular tissue region for the target image G0, it becomes possible to correct cases where the tubular tissue region is not fragmented or where the corrected path is incorrect.

[0059] The second embodiment of this disclosure will now be described. Figure 13 shows the functional configuration of the image processing apparatus according to the second embodiment. In Figure 13, the same reference numerals are used for components identical to those in Figure 3, and detailed explanations are omitted. As shown in Figure 13, the image processing apparatus 20A according to the second embodiment differs from the first embodiment in that, instead of the segmentation identification unit 24, determination unit 25, and correction unit 26, it is equipped with a trained model 28 that, when positional information of tubular tissue regions and lesion regions is input, outputs a tubular tissue region with corrected extraction results if necessary.

[0060] The trained model 28 is a machine learning model trained to derive a modified target image G0 from the target image G0, the tubular tissue regions extracted by the tubular tissue extraction unit 22, and the lesion regions 38 extracted by the lesion extraction unit 23, correcting the extraction results of the tubular tissue regions as needed. The trained model 28 consists of a convolutional neural network, similar to the SS model. The inputs to the trained model 28 are the tomographic image D0 that constitutes the target image G0, the tubular tissue regions extracted in the tomographic image D0, and the lesion regions extracted in the tomographic image D0. The output of the trained model 28 is the target image G0 with the extraction results of the tubular tissue regions corrected as needed. The trained model 28 is input with the coordinates of each pixel in the tubular tissue regions in the tomographic image D0, and with the coordinates of each pixel in the lesion regions in the tomographic image D0. Therefore, the lesion regions input to the trained model 28 include location information of the lesion regions.

[0061] Figure 14 shows the training data used to train the pre-trained model 28. As shown in Figure 14, the training data 60 consists of a training image 61, a tubular tissue region 62 extracted from the training image 61, a lesion region 63 extracted from the training image 61, and a modified image 64 in which the extraction result of the tubular tissue region 62 in the training image 61 has been corrected. The pre-trained model 28 is constructed by training a convolutional neural network to output the modified image 64 when the training image 61, the tubular tissue region 62, and the lesion region 63 are input.

[0062] As in the second embodiment, by using a trained model 28 that has been trained to derive a modified target image G0 from the target image G0, the tubular tissue region extracted by the tubular tissue extraction unit 22, and the lesion region extracted by the lesion extraction unit 23, the accuracy of tubular tissue extraction can be improved by considering whether or not the tubular tissue is actually fragmented.

[0063] In the embodiments described above, the tubular tissue is defined as the main pancreatic duct within the pancreas, but the invention is not limited to this. The technology of this disclosure can also be applied when extracting any tubular tissue, such as the bile duct, blood vessels, esophagus, bronchi, small intestine, and large intestine, from medical images.

[0064] Furthermore, although a CT image is used as the target image G0 in the above embodiment, it is not limited to this. Any image can be used as the target image G0, including 3D images such as MRI images, or radiographic images obtained by simple radiography.

[0065] Furthermore, in the above embodiment, the hardware structure of the Processing Unit that executes various processes such as the image acquisition unit 21, the tubular tissue extraction unit 22, the lesion extraction unit 23, the segmentation identification unit 24, the determination unit 25, the correction unit 26, the display control unit 27, and the trained model 28 can be any of the following types of processors. As mentioned above, these types of processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a Programmable Logic Device (PLD), which is a processor whose circuit configuration can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor with a circuit configuration specifically designed to execute a particular process, such as an ASIC (Application Specific Integrated Circuit).

[0066] A single processing unit may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor.

[0067] Examples of configuring multiple processing units with a single processor include, firstly, a configuration where one or more CPUs and software combine to form a single processor, as exemplified by client and server computers, and this processor functions as multiple processing units. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as exemplified by System-on-a-Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned processors.

[0068] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits (Circuitry) that combine circuit elements such as semiconductor elements. [Explanation of symbols]

[0069] 1 Computer 2. Imaging device 3. Image storage server 4 Network 11 CPU 12 memory 13 Storage 14 displays 15 Input Devices 16 memory 20,20A Image Processing Device 21 Image acquisition unit 22 Tubular tissue extraction part 22A 1st SS Model 22B 2nd SS Model 23 Lesion extraction area 23A SS model 24 Divided Specific Section 25 Judgment section 26 Correction section 27 Display Control Unit 28 Pre-trained models 30 Pancreas 31,31A Main pancreatic duct 34-36 yen 37 straight line 38 Lesion Areas 40 line segments 50 display screen 51 Image display area 52 Connection Areas 53 Text 54 Edit button 60 Training data 61 Teacher Image 62 Tubular tissue area 63 Lesion Areas 64 corrected images D0 fault image G0 Target Image P1~P4 end points

Claims

1. Equipped with at least one processor, The aforementioned processor, Extract tubular tissue regions from medical images containing tubular tissue. From the aforementioned medical image, the lesion area surrounding the tubular tissue region is extracted. Identify the location of the division in the tubular tissue region, Based on the location information of the lesion region, it is determined whether or not to connect the divided portion of the tubular tissue region. An image processing device that, when it is determined that the aforementioned divided portions should be connected, modifies the extraction result of the tubular tissue region so as to connect the divided portions in the tubular tissue region.

2. The image processing apparatus according to claim 1, wherein the processor determines whether or not the lesion region exists at the divided portion based on the location information of the lesion region, and determines to connect the divided portion if it is determined that the lesion region does not exist at the divided portion.

3. The image processing apparatus according to claim 1 or 2, wherein the processor displays the medical image in a manner that allows for identification of whether or not a modification has been made to the tubular tissue region.

4. The image processing apparatus according to claim 3, wherein, when the processor corrects the extraction result of the tubular tissue region, the medical image is displayed in a manner that allows identification of the corrected portion in the tubular tissue region.

5. The image processing apparatus according to claim 3 or 4, wherein the processor modifies the path of the tubular tissue region in response to an instruction to modify the path of the tubular tissue region in the medical image.

6. The aforementioned tubular tissue is the main pancreatic duct. The image processing apparatus according to any one of claims 1 to 5, wherein the lesion region is a lesion region in the pancreas.

7. Extract tubular tissue regions from medical images containing tubular tissue. From the aforementioned medical image, the lesion area surrounding the tubular tissue region is extracted. Identify the location of the division in the tubular tissue region, Based on the location information of the lesion region, it is determined whether or not to connect the divided portion of the tubular tissue region. An image processing method that modifies the extraction result of the tubular tissue region to connect the separated portions in the tubular tissue region when it is determined that the separated portions should be connected.

8. Procedure for extracting tubular tissue regions from medical images containing tubular tissue. A procedure for extracting a lesion area surrounding the tubular tissue region from the aforementioned medical image. Identify the location of the division in the tubular tissue region, A procedure for determining whether or not to connect the divided portion of the tubular tissue region based on the location information of the lesion region, An image processing program that causes a computer to perform a procedure to modify the extraction result of the tubular tissue region so that the separated portions in the tubular tissue region are connected, when it is determined that the separated portions should be connected.

Images