Image processing device, image processing method, and computer program

Abstract

To efficiently diagnose a tubular organ by referring to an image.SOLUTION: An image processing system comprises an image acquisition part, a region extraction part, and a display control part. The image acquisition part acquires an input image being a three-dimensional image obtained by imaging a tubular organ. The region extraction part extracts an organ region being a region expressing the tubular organ in the input image. The display control part superimposes and displays an organ image expressing the organ region and a direction designation image showing a direction from a predetermined position on the organ region toward an end point along the path of the organ region, on a display device.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The technology disclosed herein relates to image processing for handling images of tubular organs. [Background technology]

[0002] Intestinal obstruction is an abdominal emergency that requires rapid and accurate diagnosis and treatment. When diagnosing intestinal obstruction, clinicians identify the site of obstruction by referring to, for example, 3D abdominal CT images. However, because the intestinal tract is long and has complex, winding structures, identifying the site of obstruction using images can be difficult, especially for non-expert clinicians.

[0003] Conventionally, a technology has been proposed that supports the task of identifying obstructed areas in the intestinal tract by extracting intestinal regions from 3D abdominal CT images using a trained model (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Hirohisa Oda and five others, "Multi-class prediction for improving intestine segmentation on non-fecal-tagged CT volumes," SPIE Medical Imaging 2022, November 9, 2023, pp. 414-421 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-described conventional techniques have room for improvement in terms of effectively supporting the diagnosis of intestinal obstruction. This problem is not limited to intestinal obstruction, but is a common problem when diagnosing tubular organs by referring to images.

[0006] This specification discloses a technique that can solve the above-mentioned problems. [Means for solving the problem]

[0007] The technology disclosed in this specification can be realized, for example, in the following forms.

[0008] (1) The image processing device disclosed in this specification includes an image acquisition unit, a region extraction unit, and a display control unit. The image acquisition unit acquires an input image, which is a three-dimensional image obtained by capturing an image of a tubular organ. The region extraction unit extracts an organ region, which is a region representing the tubular organ, from the input image. The display control unit causes a display device to superimpose an organ image representing the organ region and a direction indication image indicating a direction from a predetermined position on the organ region to an end point along the path of the organ region. With this image processing device, by referring to the direction indication image, it is possible to easily determine the direction in which the tubular organ is traveling, thereby improving the efficiency of diagnosis of tubular organs.

[0009] (2) In the image processing device, the direction indicator image may be an image indicating a direction from the predetermined position on the organ region toward one of two endpoints along a path of the organ region that is closer to the predetermined position along the path. With this configuration, by referring to the direction indicator image, it is possible to easily understand in which direction to trace along the path of the tubular organ in order to quickly reach the endpoint, thereby further improving the efficiency of diagnosis of the tubular organ.

[0010] (3) In the image processing device, the direction indication image may be an image of an arrow indicating the direction. With this configuration, by referring to the direction indication image, the direction of the path of the tubular organ can be intuitively grasped, thereby further improving the efficiency of diagnosis of the tubular organ.

[0011] (4) In the image processing device, the organ image may be an image in which the color gradually changes from one end point to the other end point along the path of the organ region. With this configuration, by referring to the organ image representing the organ region, the path from one end point of the tubular organ to the other end point can be easily grasped, thereby further improving the efficiency of diagnosis of the tubular organ.

[0012] (5) In the image processing device, the display control unit may be configured to, when a user performs a predetermined movement operation, switch the organ image displayed on the display device to the organ image at a position moved along the path of the organ region by a distance corresponding to the amount of the movement operation. With this configuration, an image at a desired position in the organ region can be displayed, thereby further improving the efficiency of diagnosis of tubular organs.

[0013] (6) In the image processing device, the display control unit may be configured to, when a user performs a designation operation to designate a specific part of the organ region, change the color of the specific part in the organ image to a color that makes the specific part distinguishable from other parts. With this configuration, the difference in color makes it easy to recognize parts of the organ region that have already been interpreted, thereby further improving the efficiency of diagnosis of tubular organs.

[0014] (7) In the image processing device, the display control unit may be configured to change the color of a specific part of the organ region in the organ image to a color that makes it distinguishable from other parts after the specific part of the organ region in the organ image is displayed on the display device. With this configuration, the difference in color makes it easy to recognize parts of the organ region that have already been interpreted, thereby further improving the efficiency of diagnosis of tubular organs.

[0015] The technology disclosed in this specification can be realized in various forms, such as an image processing device, a diagnostic support device, an image processing method, a diagnostic support method, a computer program that realizes these methods, a non-transitory recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of an image processing device 100 according to an embodiment of the present invention. [Figure 2] 1 is a flowchart showing intestinal image display processing according to this embodiment. [Figure 3] FIG. 10 is an explanatory diagram showing an example of an input image I0. [Figure 4] FIG. 1 is an explanatory diagram showing an example of an image I1 representing a three-dimensionally reconstructed intestinal region R1. [Figure 5] FIG. 1 is an explanatory diagram showing an example of an image I2 in which a direction indicator image DD is superimposed on an image I1 representing an intestinal region R1. DETAILED DESCRIPTION OF THE INVENTION

[0017] (Configuration of image processing device 100) FIG. 1 is an explanatory diagram showing a schematic configuration of an image processing device 100 according to this embodiment. The image processing device 100 according to this embodiment is a device that supports the task of identifying an obstructed part in the intestinal tract by three-dimensionally reconstructing an intestinal tract region from, for example, a three-dimensional CT image of the abdomen and displaying an image of the intestinal tract region. In this specification, the intestinal tract includes the large intestine and the small intestine. The intestinal tract is an example of a tubular organ. The intestinal tract region is an example of an organ region.

[0018] The image processing device 100 is configured by a computer (PC, server, etc.). The image processing device 100 includes a control unit 110, a storage unit 120, a display unit 130, an operation input unit 140, and an interface unit 150. These units are connected to each other via a bus 190 so as to be able to communicate with each other. The image processing device 100 may also include a speaker as an output means.

[0019] The display unit 130 of the image processing device 100 is configured, for example, by a liquid crystal display or an organic EL display, and displays various images and information. The display unit 130 is an example of a display device. The operation input unit 140 is configured, for example, by a keyboard, a mouse, buttons, a microphone, a trackpad, and the like, and receives user operations and instructions. The display unit 130 may also function as the operation input unit 140 by including a touch panel. The interface unit 150 is configured, for example, by a LAN interface, a USB interface, and the like, and communicates with other devices via wired or wireless connection.

[0020] The storage unit 120 of the image processing device 100 is configured with, for example, a ROM, a RAM, a hard disk drive (HDD), etc., and is used to store various programs and data, and as a work area when executing various programs, and as a temporary storage area for data. For example, the storage unit 120 stores an image processing program CP, which is a computer program for executing an intestinal image display process described below. The image processing program CP is provided in a state stored in a computer-readable recording medium (not shown), such as a CD-ROM, a DVD-ROM, or a USB memory, or is provided in a state that can be obtained from an external device (a server or other terminal device on a network) via the interface unit 150, and is stored in the storage unit 120 in a state that is operable on the image processing device 100.

[0021] Furthermore, a trained model MO is stored in advance in the storage unit 120 of the image processing device 100. The trained model MO will be explained together with the explanation of the intestinal tract image display processing to be described later.

[0022] The control unit 110 of the image processing device 100 is configured with, for example, a CPU, and controls the operation of the image processing device 100 by executing a computer program read from the storage unit 120. For example, the control unit 110 reads and executes an image processing program CP from the storage unit 120, thereby functioning as an image processing unit 111 for executing intestinal tract image display processing, which will be described later. The image processing unit 111 includes an image acquisition unit 112, a region extraction unit 114, and a display control unit 116. The functions of these units will be described in conjunction with the description of the intestinal tract image display processing, which will be described later.

[0023] (Intestinal image display processing) 2 is a flowchart showing intestinal image display processing in this embodiment. The intestinal image display processing is processing for three-dimensionally reconstructing an intestinal region based on a three-dimensional CT image of the abdomen and displaying an image of the intestinal region. The intestinal image display processing is executed to assist in identifying an obstructed portion in the intestine, for example, when diagnosing intestinal obstruction.

[0024] First, the image acquisition unit 112 (FIG. 1) of the image processing device 100 acquires a 3D CT image obtained by imaging the abdomen using CT as an input image I0 (S110). FIG. 3 is an explanatory diagram showing an example of the input image I0. The input image I0 shows the intestinal tract. The input image I0 is input from the outside via the interface unit 150, for example. Note that the image processing device 100 may have an imaging function and acquire the input image I0 by imaging itself.

[0025] Next, the region extraction unit 114 (FIG. 1) of the image processing device 100 three-dimensionally reconstructs the intestinal region based on the input image I0 acquired in S110 (S120). The three-dimensional reconstruction of the intestinal region can be performed according to any known method, and is performed, for example, as follows.

[0026] First, multiple intestinal regions are extracted from a 3D CT image of the abdomen using a trained model MO generated by a specific machine learning method. For example, a 3D U-Net is used as the trained model MO to estimate a 2D normalized distance map from the 3D CT image. This distance map indicates the distance from the intestinal wall in a range of 0.0 to 1.0, with the center of the intestine taking a value of 1.0. Multiple intestinal regions (intestinal segments) are extracted using a watershed algorithm with the maximum value of this distance map as the seed point.

[0027] Alternatively, we use the trained model MO to obtain intestinal region labels for four channels (background, air, liquid, and feces) from 3D CT images, binarize the labels, and then perform 3D distance transformation to obtain a 3D distance map.Multiple intestinal regions are extracted using a watershed algorithm with the maximum value of this distance map as a seed point.

[0028] The intestinal regions extracted by the above method are connected as a graph, and the intestinal region is reconstructed in 3D using 3D volume rendering. At this time, the path of the intestinal region is identified using a constrained distance transform, and two endpoints (start and end points) along the path are obtained.

[0029] Next, the display control unit 116 (FIG. 1) of the image processing device 100 causes the display unit 130 to display an intestinal image representing the intestinal region (S130). FIG. 4 is an explanatory diagram showing an example of an image I1 representing a three-dimensionally reconstructed intestinal region R1. In this embodiment, the image I1 representing the intestinal region R1 is an image in which the color gradually changes from one end point EP1 to the other end point EP2 along the path of the intestinal region R1. The color change may be, for example, a change in hue (e.g., a change from red to orange, yellow, green, blue, and purple), a change in brightness (a change in shading) within the same hue, or a combination thereof.

[0030] The display control unit 116 also displays a direction indication image DD superimposed on the image I1 representing the intestinal region R1. FIG. 5 is an explanatory diagram showing an example of an image I2 in which the direction indication image DD is superimposed on the image I1 (FIG. 4) representing the intestinal region R1. The direction indication image DD is an image indicating a direction from a predetermined position on the intestinal region R1 toward an end point EP along the path of the intestinal region R1. In this embodiment, the direction indication image DD is an image of an arrow. In this embodiment, the direction indication image DD is an image indicating a direction from a predetermined position on the intestinal region R1 toward one of two end points EP along the path of the intestinal region R1 that is closer to the predetermined position along the path. Such a direction indication image DD is generated and displayed by calculating, at each position on the intestinal region R1, the distance to the two end points EP along the path of the intestinal region R1 and displaying an arrow with the arrowhead pointing toward the end point with the shorter distance.

[0031] The direction indicating image DD may be always displayed superimposed on the image I1 representing the intestinal region R1 when it is displayed, or may be displayed when a clinician (user) performs an instruction operation via the operation input unit 140. Furthermore, the position where the direction indicating image DD is displayed (the above-mentioned predetermined position) may be set automatically, or may be set based on a position designation operation by the clinician via the operation input unit 140.

[0032] By referring to the direction indicator image DD, clinicians can easily understand the direction of the intestine. During image diagnosis of intestinal obstruction, for example, the intestine is tracked by shifting an axial image cut from a horizontal plane of a 3D abdominal CT image in the craniocaudal direction. Although the coloring of the image representing the intestinal region R1 is reflected in the axial image, the image itself does not provide any indication of whether tracking head-to-tail will result in the most rapid arrival at the end point EP. However, by referring to the direction indicator image DD, clinicians can easily understand which direction to track head-to-tail to reach the end point EP. When the clinician performs a predetermined movement operation on the operation input unit 140 (e.g., by rotating the mouse wheel), the displayed image changes to an image of a position moved along the path of the intestinal region R1 by a distance corresponding to the amount of movement. By tracing the intestinal region R1 along its path in this manner, clinicians can identify the end point EP of the intestinal region R1, i.e., a candidate for the obstruction site.

[0033] Furthermore, when the clinician performs an operation to designate a specific portion of the intestinal tract region R1 via the operation input unit 140, the display control unit 116 may change the color of the specific portion of the intestinal tract region R1 to a color (e.g., gray) that makes it distinguishable from other portions. In this way, the portion of the intestinal tract region R1 that is considered to have already been interpreted by the clinician can be distinguished by, for example, graying it out, and the task of identifying the obstructed portion in the intestine can be made more efficient.

[0034] Furthermore, the display control unit 116 may change the color of a specific portion of the intestinal tract region R1 to a color (e.g., gray) that makes it distinguishable from other portions after the specific portion is displayed on the display unit 130. In this way, the portion of the intestinal tract region R1 that is considered to have already been interpreted by the clinician can be distinguished by, for example, graying it out, and the task of identifying an obstructed portion in the intestine can be made more efficient.

[0035] (Effects of this embodiment) As described above, the image processing device 100 of this embodiment includes an image acquisition unit 112, a region extraction unit 114, and a display control unit 116. The image acquisition unit 112 acquires an input image, which is a 3D image obtained by capturing an image of the intestinal tract, which is a tubular organ. The region extraction unit 114 extracts an intestinal region R1, which is a region representing the intestinal tract, from the input image. The display control unit 116 displays, on the display unit 130, an image representing the intestinal region R1 and a direction indication image DD indicating a direction from a predetermined position on the intestinal region R1 toward an endpoint EP along the path of the intestinal region R1, in a superimposed manner.

[0036] In this way, according to the image processing device 100 of this embodiment, by referring to the direction indication image DD, it is possible to easily understand the direction in which the intestinal tract runs, thereby making the diagnostic work of the intestinal tract more efficient.

[0037] In this embodiment, the direction indicating image DD is an image that indicates the direction from a predetermined position on the intestinal region R1 toward one of two endpoints EP along the path of the intestinal region R1 that is closer to the predetermined position along the path. By referring to the direction indicating image DD, the image processing device 100 of this embodiment can easily determine in which direction to trace along the path of the intestine in order to quickly reach the endpoint EP, thereby further improving the efficiency of intestinal diagnosis work.

[0038] In this embodiment, the direction indication image DD is an image of an arrow indicating the above-mentioned direction. According to the image processing device 100 of this embodiment, by referring to the direction indication image DD, the direction of the path of the intestinal tract can be intuitively grasped, and the efficiency of the intestinal tract diagnosis work can be further improved.

[0039] In this embodiment, the image representing the intestinal region R1 is an image in which the color gradually changes from one end point EP to the other end point EP along the path of the intestinal region R1. According to the image processing device 100 of this embodiment, by referring to the image representing the intestinal region R1, the path from one end point EP of the intestine to the other end point EP can be easily grasped, thereby further improving the efficiency of intestinal diagnosis work.

[0040] Furthermore, in this embodiment, when a predetermined movement operation is performed by the user, the display control unit 116 switches the image of the intestinal tract region R1 displayed on the display unit 130 to an image at a position moved along the path of the intestinal tract region R1 by a distance corresponding to the amount of the movement operation. According to the image processing device 100 of this embodiment, it is possible to display an image at a desired position in the intestinal tract region R1, thereby further improving the efficiency of intestinal diagnostic work.

[0041] Furthermore, in this embodiment, when a user performs a designation operation to designate a specific portion of the intestinal tract region R1, the display control unit 116 may change the color of the specific portion in the image representing the intestinal tract region R1 to a color that makes the specific portion distinguishable from other portions. In this way, the difference in color makes it easy to recognize portions of the intestinal tract region R1 that have already been interpreted, thereby further improving the efficiency of intestinal diagnostic work.

[0042] Furthermore, in this embodiment, after a specific portion of the intestinal region R1 in the image representing the intestinal region R1 is displayed on the display unit 130, the display control unit 116 may change the color of the specific portion in the image to a color that makes it distinguishable from other portions. In this way, the difference in color makes it easy to recognize portions of the intestinal region R1 that have already been interpreted, thereby further improving the efficiency of intestinal diagnostic work.

[0043] B. Variations: The technology disclosed in this specification is not limited to the above-described embodiments, and can be modified into various forms without departing from the spirit thereof, for example, the following modifications are also possible.

[0044] The configuration of the image processing device 100 in the above embodiment is merely an example and can be modified in various ways. For example, in the above embodiment, the image processing device 100 includes the display unit 130, and the image of the intestinal tract region R1 is displayed on the display unit 130. However, the image of the intestinal tract region R1 may be displayed on another display device connected to the image processing device 100 via the interface unit 150.

[0045] In the above-described embodiment, a part of the configuration realized by hardware may be replaced by software, and conversely, a part of the configuration realized by software may be replaced by hardware.

[0046] The contents of the intestinal image display process in the above embodiment are merely examples and can be modified in various ways. For example, in the above embodiment, the direction indication image DD is an arrow image, but the direction indication image DD may be an image of another form as long as it is an image that allows a direction to be recognized.

[0047] In the above embodiment, a 3D CT image is used as an input image, but other 3D images may be used. Also, in the above embodiment, an image of the intestinal tract is used, but the technology disclosed in this specification is similarly applicable to images of other tubular organs (e.g., bronchi and blood vessels). [Explanation of symbols]

[0048] 100: Image processing device 110: Control unit 111: Image processing unit 112: Image acquisition unit 114: Region extraction unit 116: Display control unit 120: Memory unit 130: Display unit 140: Operation input unit 150: Interface unit 190: Bus DD: Direction indication image EP: End point I0: Input image I1: Image I2: Image R1: Intestinal region

Claims

1. An image processing device, an image acquisition unit that acquires an input image, which is a three-dimensional image obtained by imaging a tubular organ; a region extraction unit that extracts an organ region that represents the tubular organ in the input image; a display control unit that displays, on a display device, an organ image representing the organ region and a direction indication image that indicates a direction from a predetermined position on the organ region toward an end point along a path of the organ region in a superimposed manner; An image processing device comprising:

2. 2. The image processing device according to claim 1, An image processing device, wherein the direction indicating image is an image that indicates a direction from the specified position on the organ area toward one of two endpoints along a path of the organ area that is closer to the specified position along the path.

3. 3. The image processing device according to claim 1, The direction indication image is an image of an arrow indicating the direction.

4. 3. The image processing device according to claim 1, An image processing device, wherein the organ image is an image in which color gradually changes from one end point to the other end point along the path of the organ region.

5. 3. The image processing device according to claim 1, The display control unit, when a predetermined movement operation is performed by the user, switches the organ image displayed on the display device to the organ image at a position moved along the path of the organ area by a distance corresponding to the amount of the movement operation.

6. 3. The image processing device according to claim 1, The image processing device wherein, when a user performs a designation operation to designate a specific part of the organ area, the display control unit changes the color of the specific part in the organ image to a color that is distinguishable from other parts.

7. 3. The image processing device according to claim 1, The image processing device, wherein after a specific portion of the organ area in the organ image is displayed on a display device, the display control unit changes the color of the specific portion in the organ image to a color that is distinguishable from other portions.

8. 1. An image processing method, comprising: acquiring an input image, which is a three-dimensional image obtained by imaging a tubular organ; extracting an organ region representing the tubular organ in the input image; a step of displaying, on a display device, an organ image representing the organ region and a direction indication image indicating a direction from a predetermined position on the organ region toward an end point along a path of the organ region in a superimposed manner; An image processing method comprising:

9. A computer program comprising: On the computer, acquiring an input image, which is a three-dimensional image obtained by imaging a tubular organ; extracting an organ region representing the tubular organ from the input image; a process of displaying, on a display device, an organ image representing the organ region and a direction indication image indicating a direction from a predetermined position on the organ region to an end point along a path of the organ region in a superimposed manner; A computer program that executes

Images