Medical image processing device and medical image processing method

Abstract

To efficiently and accurately draw a three-dimensional contour of a target structure on the basis of a three-dimensional scan image.SOLUTION: A medical image processing apparatus includes: an initial contour drawing unit which draws, as an initial contour, a contour of a target structure in a first surface direction image which is a cross-sectional image in a first surface direction of a three-dimensional scan image; a second surface direction contour drawing unit which draws a second surface direction contour of the target structure on the basis of the initial contour, for each of multiple second surface direction images which are cross-sectional images in a second surface direction of the three-dimensional scan image; and a three-dimensional contour drawing unit which draws a three-dimensional contour of the target structure on the basis of the initial contour and the second surface direction contours.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing device and a medical image processing method. [Background technology]

[0002] In clinical medicine, doctors are required to view a large number of medical scan images related to patients. With the development of image processing technology, medical image processing techniques for medical scan images are also developing rapidly. Image processing tasks for medical scan images, such as modeling, image segmentation, image annotation, and image measurement, often use contour information of target structures specified in the scan images.

[0003] Methods for drawing (annotating) the contours of a specified target structure in a 3D scan image include manual contour drawing methods and fully or semi-automatic contour drawing methods. The manual drawing method requires an operator to draw the contours of the target structure in each two-dimensional cross-sectional image of the 3D scan image, which requires a large amount of work and is time-consuming. Patent Document 1 discloses a semi-automatic contour drawing method that uses an interpolation algorithm to generate the contours of the target structure in cross sections between the two cross-sectional images of both ends of the target structure, based on the contours manually drawn by the operator in the two-dimensional cross-sectional images of both ends of the target structure. However, the technology in Patent Document 1 simply interpolates the contours drawn by the user and does not take into account anatomical information of the organs in the scanned body, which can result in insufficient contour accuracy. Furthermore, while fully automatic contour drawing methods using deep learning are known, deep learning methods require training using a large number of scanned images with contours already drawn. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 10,586,402 Summary of the Invention [Problem to be solved by the invention]

[0005] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to efficiently and accurately depict the 3D contour of a target structure based on a 3D scanned image. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0006] The medical image processing device of the embodiment includes an initial contour drawing unit that draws the contour of a target structure as an initial contour in a first plane direction image, which is a cross-sectional image in a first plane direction of a three-dimensional scan image; a second plane direction contour drawing unit that draws a second plane direction contour of the target structure based on the initial contour for each of a plurality of second plane direction images, which are cross-sectional images in a second plane direction of the three-dimensional scan image; and a three-dimensional contour drawing unit that draws a three-dimensional contour of the target structure based on the initial contour and the second plane direction contour. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a three-dimensional contour drawing device according to the first embodiment. [Figure 2] FIG. 2 is a flowchart showing an example of a three-dimensional contour drawing method performed by the three-dimensional contour drawing device according to the first embodiment. [Figure 3] FIG. 3 is a simplified schematic diagram showing an example of a start image and an end image in the first embodiment. [Figure 4] FIG. 4 is a simplified schematic diagram of an example of a second surface direction image in the first embodiment. [Figure 5] FIG. 5 is a diagram for explaining the detailed operation of step S400 in the three-dimensional contour drawing method according to the first embodiment. [Figure 6]FIG. 6 is a diagram for explaining the detailed operation of step S500 in the three-dimensional contour drawing method according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a distance image. [Figure 8] FIG. 8 is a diagram for explaining the detailed operation of step S600 in the three-dimensional contour drawing method according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating a problem with the three-dimensional contour drawing method according to the first embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of a three-dimensional contour drawing method according to the second embodiment. [Figure 11] FIG. 11 is a diagram for explaining the contour expanding method according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of a medical image processing apparatus and a medical image processing method will be described with reference to the drawings.

[0009] (First embodiment) In an embodiment, a medical image processing device renders a three-dimensional contour of a target structure in a three-dimensional scanned image of a biological organ, where the target structure may be a specified structure in the organ or the organ itself.

[0010] The following describes a three-dimensional contour drawing device 100 in FIG. 1 as an example of a medical image processing device. FIG. 1 is a diagram showing an example of the configuration of the three-dimensional contour drawing device 100 in the first embodiment. The three-dimensional contour drawing device 100 in the first embodiment includes a display unit 110, an input unit 120, a storage unit 130, an initial setting unit 140, an initial contour drawing unit 150, a second-plane direction contour drawing unit 160, a first-plane direction contour drawing unit 170, and a three-dimensional contour drawing unit 180. The display unit 110, the input unit 120, the storage unit 130, the initial setting unit 140, the initial contour drawing unit 150, the second-plane direction contour drawing unit 160, the first-plane direction contour drawing unit 170, and the three-dimensional contour drawing unit 180 are connected to each other so that they can communicate with each other.

[0011] The display unit 110 displays various types of information. For example, the display unit 110 displays a two-dimensional cross-sectional image of a three-dimensional scanned image used to draw a contour, and displays a user interface for receiving input operations from a user, such as a GUI (Graphical User Interface). For example, the display unit 110 is an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display.

[0012] The input unit 120 accepts an input operation from a user and outputs a signal based on the accepted input operation to the initial contour drawing unit 150. For example, the input unit 120 is implemented by a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch screen, or the like. The input unit 120 can be implemented by a user interface that accepts audio input from a microphone, for example. When the input unit 120 is a touch screen, the display unit 110 may be formed integrally with the input unit 120.

[0013] The storage unit 130 is realized by a storage device such as a ROM, a flash memory, a random access memory (RAM), a hard disk drive (HDD), a solid state drive (SSD), or a register. Flash memory, HDD, SSD, etc. are non-volatile storage media. These non-volatile storage media can be realized by other storage devices connected via a network, such as a network attached storage (NAS) or an external storage server device. Here, the above-mentioned network includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a carrier terminal, a wireless communication network, a wireless base station, a dedicated line, etc.

[0014] The storage unit 130 stores a three-dimensional scan image. The three-dimensional scan image is obtained by scanning a region of a human or animal body using imaging techniques such as CT (Computed Tomography), ultrasound imaging, or MRI (Magnetic Resonance Imaging). The three-dimensional scan image represents internal structural information of the scanned object and includes a specified target structure of the scanned object.

[0015] Hereinafter, the 3D scan image will be described as a grayscale image; however, the 3D scan image may also be a color image such as an RGB image. In this embodiment, the resolution of the 3D scan image is width W × height H × depth D, and each voxel in the 3D scan image represents structural information at a specific position on the scanned object in grayscale. Hereinafter, a two-dimensional image of an arbitrary two-dimensional cross section extracted from the 3D scan image is also referred to as a cross-sectional image of the corresponding three-dimensional image. Because it is difficult to directly display a 3D scan image on a display, a 3D scan image is typically represented as a two-dimensional cross-sectional image. For example, a 3D scan image may be represented as multiple cross-sectional images of consecutive positions that are parallel to each other. Furthermore, in the 3D scan image, unscanned areas are subjected to, for example, zero-fill processing.

[0016] The initial setting unit 140 provides the user with a user interface for initial setting and sets the first plane direction, the second plane direction, the start image, and the end image in response to user operation. The first plane direction and the second plane direction are plane directions such as a horizontal plane and a vertical plane, and the first plane direction and the second plane direction are perpendicular to each other. The three-dimensional contour drawing device 100 generates a three-dimensional contour of the target structure by drawing a two-dimensional contour of the target structure on a cross-sectional image parallel to the first plane direction and a cross-sectional image parallel to the second plane direction. The start image and the end image are cross-sectional images parallel to the first plane direction; details of the start image and the end image will be described later. Hereinafter, cross-sectional images parallel to the first plane direction, excluding the start image and the end image, will be referred to as first plane direction images, and cross-sectional images parallel to the second plane direction will be referred to as second plane direction images.

[0017] The initial contour drawing unit 150 provides the user with a user interface for drawing a two-dimensional contour of the target structure in the start image and the end image, and generates a first initial contour, which is the two-dimensional contour of the target structure in the start image, and a second initial contour, which is the two-dimensional contour of the target structure in the end image, in accordance with the user's drawing operation.

[0018] The second surface direction contour drawing unit 160 generates a two-dimensional contour of the target structure in each second surface direction image based on the first initial contour and the second initial contour drawn by the user, using an interactive shortest path algorithm such as the LiveWire algorithm.

[0019] The first surface direction contour drawing unit 170 generates a two-dimensional contour of the target structure in the first surface direction images excluding the start image and the end image based on the contour of the target structure in each second surface direction image generated by the second surface direction contour drawing unit 160.

[0020] The three-dimensional contour drawing unit 180 generates a three-dimensional contour of the target structure based on the first initial contour, the second initial contour, the contour of the target structure in each second plane direction image generated by the second plane direction contour drawing unit 160, and the contour of the target structure in each first plane direction image generated by the first plane direction contour drawing unit 170.

[0021] These components are realized by a hardware processor such as a CPU or GPU executing a program (software) stored in the storage unit 130. Some or all of these components may be realized by hardware (circuitry) such as an LSI, ASIC, or FPGA, or may be realized by cooperative operation of software and hardware. The above-mentioned program may be stored in advance in the storage unit 130, or may be stored in a removable storage medium such as a DVD or CD-ROM and installed into the storage unit 130 from the storage medium by attaching the storage medium to the drive device of the three-dimensional contour drawing device 100.

[0022] The following describes the flow of the three-dimensional contour drawing method performed by the three-dimensional contour drawing device 100 according to this embodiment, with reference to the drawings. Fig. 2 is a flowchart showing an example of the three-dimensional contour drawing method performed by the three-dimensional contour drawing device 100 according to the first embodiment.

[0023] In step S100, the initial setting unit 140 displays a user interface for setting the first surface direction and the second surface direction on the display unit 110, and the user sets an arbitrary surface direction as the first surface direction via the user interface, and then sets an arbitrary surface direction from among the surface directions perpendicular to the first surface direction as the second surface direction.

[0024] In step S200, the user selects a start image and an end image that is not adjacent to the start image from among the first face-orientation images.

[0025] In step S300, the initial contour drawing unit 150 displays on the display unit 110 a user interface for drawing a two-dimensional contour of the target structure in the start image and the end image, and the user draws a two-dimensional contour of the target structure in each of the start image and the end image via this user interface, and the initial contour drawing unit 150 generates a first initial contour and a second initial contour in accordance with the user's drawing operation.

[0026] In step S400, the second plane direction contour drawing unit 160 identifies, for each second plane direction image in the 3D scan image, the intersections between the first initial contour and the second initial contour and the cross section on which the second plane direction image is located.

[0027] In step S500, the second surface direction contour drawing unit 160 generates, for each second surface direction image in the three-dimensional scanned image, a two-dimensional contour of the target structure on the second surface direction image based on an interactive shortest path algorithm such as LiveWire, using the cross section on which the second surface direction image is located and the intersection between the first initial contour and the second initial contour.

[0028] In step S600, the first-plane direction contour drawing unit 170 identifies the intersection points between each first-plane direction image between the start image and the end image in the three-dimensional scan image and the two-dimensional contour of the target structure on each second-plane direction image, and sets the contour consisting of these intersection points as the two-dimensional contour of the target structure on the first-plane direction image.

[0029] In step S700, the three-dimensional contour drawing unit 180 generates a three-dimensional contour of the target structure based on the first initial contour, the second initial contour, the two-dimensional contour of the target structure in each second plane direction image generated by the second plane direction contour drawing unit 160, and the two-dimensional contour of the target structure in each first plane direction image generated by the first plane direction contour drawing unit 170.

[0030] An example of drawing the contours of interlobar fissures in a human lung by applying the three-dimensional contour drawing method of this embodiment will be described below.

[0031] First, the user sets the first and second surface directions based on the user interface displayed on the display unit 110 (step S100).

[0032] When the three-dimensional scan image is a scan image of a human body, the first and second plane directions are preferably coronal or sagittal. As known to those skilled in the art, in the case of a human body, the coronal plane is a cross section obtained by vertically dividing the human body into two parts, a front and a back, along the width direction (the left-right direction of the human body), and is a cross section parallel to the height and width directions and perpendicular to the depth direction. The sagittal plane is a cross section obtained by vertically dividing the human body into two parts, a left and a right, along the depth direction (the front-back direction of the human body), and is a cross section parallel to the height and depth directions and perpendicular to the width direction. Regarding the three-dimensional scan image according to this embodiment, since the resolution of the three-dimensional scan image is width W × height H × depth D, each sagittal plane image is a two-dimensional image with height H × depth D, and each coronal plane image is a two-dimensional image with width W × height H.

[0033] In this example, it is assumed that the sagittal plane direction is the first plane direction and the coronal plane direction is the second plane direction.

[0034] Next, the user selects a start image and an end image that is not adjacent to the start image from among sagittal plane images that are parallel to the sagittal plane (step S200).

[0035] The three-dimensional contour drawing method of this embodiment generates a three-dimensional contour of the target structure from a first initial contour and a second initial contour drawn by the user on the start image and the end image. Therefore, it is preferable to use images with a clear and wide contour of the target structure as the start image and the end image. Furthermore, since the three-dimensional contour drawing method of this embodiment generates a three-dimensional contour of the target structure between the start image and the end image, the greater the distance between the start image and the end image, the wider the range of the three-dimensional contour that can be generated. On the other hand, if the distance between the start image and the end image is too large, the accuracy of the generated three-dimensional contour may decrease. Therefore, it is necessary to appropriately set the distance between the start image and the end image.

[0036] FIG. 3 is a simplified schematic diagram showing an example of a start image and an end image in the first embodiment. FIG. 4 is a simplified schematic diagram showing an example of a second plane direction image (coronal plane image) in the first embodiment. FIG. 3(a) schematically shows an example of a start image located on the A-A' sagittal plane shown in FIG. 4. FIG. 3(b) schematically shows an example of an end image located on the B-B' sagittal plane shown in FIG. 4. In FIGS. 3 and 4, the contours of the pulmonary fissures are indicated by dashed lines.

[0037] Next, the user draws two-dimensional contours of the pulmonary fissures in each of the start image and the end image based on the user interface displayed on the display unit 110, and the initial contour drawing unit 150 generates a first initial contour and a second initial contour in accordance with the user's drawing operation (step S300).

[0038] The first initial contour and the second initial contour may be drawn manually by the user or semi-automatically by the user via auxiliary software.

[0039] Next, the second plane direction contour drawing unit 160 identifies, for each coronal plane image in the 3D scan image, the intersection points between the first initial contour and the second initial contour generated in step S300 and the coronal plane on which the corresponding coronal plane image is located (step S400).

[0040] 5 is a diagram illustrating the detailed operation of step S400 in the three-dimensional contour drawing method according to the first embodiment. In FIG. 5, plane SP1 represents a sagittal plane corresponding to the start image, plane SP2 represents a sagittal plane corresponding to the end image, plane CP3 represents an arbitrary coronal plane, curve C1 represents a first initial contour, and curve C2 represents a second initial contour. For an arbitrary coronal plane CP3, the second plane direction contour drawing unit 160 identifies an intersection line L1 between the coronal plane CP3 and the sagittal plane SP1 corresponding to the start image, identifies an intersection point I1 between this intersection line L1 and the first initial contour C1, and sets this intersection point I1 as the intersection point between the coronal plane CP3 and the first initial contour C1. Similarly, the second plane direction contour drawing unit 160 identifies the intersection line L2 between the coronal plane CP3 and the sagittal plane SP2 corresponding to the end image, identifies the intersection point I2 between this intersection line L2 and the second initial contour C2, and sets this intersection point I2 as the intersection point between the coronal plane CP3 and the second initial contour C2.

[0041] By identifying the intersections of the first initial contour and the second initial contour with the coronal plane on which the coronal image is located, points on the coronal plane that are located on the target structure can be identified, and further, the contour of the target structure can be drawn from the points on the target structure.

[0042] Next, the second plane direction contour drawing unit 160 determines, for each coronal plane image in the three-dimensional scan image, whether the coronal plane image includes an intersection point with the first initial contour and the second initial contour. For a coronal plane image that includes both an intersection point with the first initial contour and an intersection point with the second initial contour, the second plane direction contour drawing unit 160 generates a two-dimensional contour of the pulmonary fissure on the coronal plane image based on the intersection points of the coronal plane image with the first initial contour and the second initial contour, based on an interactive shortest path algorithm such as the LiveWire algorithm (step S500).

[0043] 6 is a diagram illustrating the detailed operation of step S500 in the three-dimensional contour drawing method according to the first embodiment. As in FIG. 5, in FIG. 6, plane SP1 represents a sagittal plane corresponding to the start image, plane SP2 represents a sagittal plane corresponding to the end image, plane CP3 represents an arbitrary coronal plane, curve C1 represents a first initial contour, curve C2 represents a second initial contour, and intersections I1 and I2 represent the intersections of plane CP3 with curve C1 and curve C2. For an arbitrary coronal plane CP3, the second plane direction contour drawing unit 160 generates a curve C3 as a two-dimensional contour of the pulmonary fissure on the coronal plane CP3, for example, by the LiveWire algorithm, based on the intersections I1 and I2 between the coronal plane CP3 and the sagittal plane SP1 corresponding to the start image and the sagittal plane SP2 corresponding to the end image.

[0044] A method for generating two-dimensional contours of pulmonary fissures in a coronal image using, for example, the LiveWire algorithm will be described in detail below. First, for any coronal image, the second-plane direction contour drawing unit 160 generates a distance map corresponding to the coronal image. Each pixel in the distance map corresponds one-to-one to a pixel in the coronal image, and the pixel value of the pixel in the distance map represents the distance between the pixel's position in the coronal image and a reference position set by the user. In this embodiment, the positions of intrapulmonary blood vessels and bronchi are set as reference positions, and the distance value is determined by the distance between the pixel's position and the intrapulmonary blood vessels or bronchi. The closer the pixel is to the intrapulmonary blood vessels or bronchi, the lower the distance value in the distance map. The farther the pixel is to the intrapulmonary blood vessels or bronchi, the higher the distance value in the distance map. Because pixels in regions where pulmonary fissures are located are far from the intrapulmonary blood vessels or bronchi, low distance values ​​are assigned to the pixels in the regions where pulmonary fissures are located. Fig. 7 is a diagram showing an example of a distance image, in which distance values ​​of the distance image are shown in grayscale. In Fig. 7, the highlighted parts represent areas where pulmonary fissures are located.

[0045] A cost image is then generated by taking the reciprocal of the pixel value of the pixel in the distance image. Each pixel in the cost image corresponds one-to-one to a pixel in the coronal image, and the pixel value of a pixel in the cost image represents the cost value of the pixel in the coronal image. In this embodiment, the cost value is determined by the distance between the pixel's position and the intrapulmonary blood vessels and bronchi. The closer the pixel is to the intrapulmonary blood vessels and bronchi, the higher the cost value in the cost image. The farther the pixel is from the intrapulmonary blood vessels and bronchi, the lower the cost value in the cost image. Because pixels in areas where pulmonary fissures are located are far from the intrapulmonary blood vessels and bronchi, low cost values ​​are assigned to the pixels in those areas. The distance image and cost image may be generated using the distance image and cost image calculation method in the LiveWire algorithm. However, they may also be obtained based on other interactive shortest path algorithms in accordance with the distance and cost value rules described above, details of which are omitted here.

[0046] In this embodiment, a cost image corresponding to the scanned image is generated from the anatomical information in the scanned image, and then the contour of the target structure is generated from this cost image using the LiveWire algorithm. This method is highly sensitive to contour regions in the image and can easily capture the location of the contour, so the contour of the target structure can be generated automatically and with high accuracy using the anatomical information in the scanned image.

[0047] After acquiring a cost image for the coronal image, the second plane direction contour drawing unit 160 searches for a path in the cost image that connects the intersection of the coronal image and the first initial contour as the starting point and the intersection of the coronal image and the second initial contour as the end point, with the path having the smallest cumulative cost. Because the starting point and end point are points on the first and second initial contours, they are located in the area where the fissures are located. Furthermore, because the cost values ​​of points in the fissure area are lower than those of other areas in the cost image, the shortest path is the path from the starting point along the fissures to the end point. All pixel points included in this path constitute the two-dimensional contour of the fissures in the coronal image.

[0048] The above operation is repeated for all coronal images that are determined to include both an intersection with the first initial contour and an intersection with the second initial contour, thereby obtaining two-dimensional contours of the interlobar fissures in each coronal image.

[0049] Then, for each sagittal plane image between the start image and the end image in the three-dimensional scan image, the first plane direction contour drawing unit 170 identifies the intersection points between that sagittal plane image and the two-dimensional contours of the interlobar fissures in each coronal plane image, and sets the contour consisting of these intersection points as the two-dimensional contours of the interlobar fissures in that sagittal plane image (step S600).

[0050] FIG. 8 is a diagram illustrating the detailed operation of step S600 in the three-dimensional contour drawing method according to the first embodiment. In FIG. 8(a), plane SP4 represents an arbitrary sagittal plane, and plane CP5 represents a coronal plane located at one end of the three-dimensional scan image. First, the first plane direction contour drawing unit 170 identifies an intersection I3 between the arbitrary sagittal plane SP4 and a curve C4 of the two-dimensional contour of the interlobar fissure in the coronal plane CP5 located at one end of the three-dimensional scan image, and records the pixel corresponding to the intersection I3. The method for identifying the intersection is the same as the method described in step S500, so a detailed description will be omitted. Then, the first plane direction contour drawing unit 170 identifies intersections between the sagittal plane SP4 and the curve of the two-dimensional contour of the interlobar fissure in each coronal plane, from one end of the three-dimensional scan image to the other end of the three-dimensional scan image, and constructs the two-dimensional contour of the interlobar fissure in the sagittal plane image of the sagittal plane SP4 using pixels corresponding to these intersections. Figure 8(b) shows a contour formed by the intersection of a coronal plane from one end of the 3D scan image to the middle of the 3D scan image with this arbitrary sagittal plane SP4. Figure 8(c) shows a contour formed by the intersection of a coronal plane from one end of the 3D scan image to the other end of the 3D scan image with this arbitrary sagittal plane SP4, i.e., a two-dimensional contour C5 of the pulmonary fissure in the sagittal plane image of this sagittal plane SP4.

[0051] The above operation is repeated for all sagittal plane images to obtain two-dimensional contours of the interlobar fissures in each sagittal plane image.

[0052] In this embodiment, not only the contour of the target structure in the first plane direction but also the contour of the target structure in the second plane direction is generated, so that the user can view the contour of the target structure from both directions that are perpendicular to each other.

[0053] Next, the three-dimensional contour drawing unit 180 generates a three-dimensional contour of the pulmonary fissure based on the first initial contour, the second initial contour, the two-dimensional contour of the pulmonary fissure in each coronal plane image generated by the second plane direction contour drawing unit 160, and the two-dimensional contour of the pulmonary fissure in each sagittal plane image generated by the first plane direction contour drawing unit 170 (step S700).

[0054] In S700, the specific operation of generating a three-dimensional contour of the pulmonary fissure using the first initial contour, the second initial contour, the two-dimensional contour of the pulmonary fissure in each coronal image, and the two-dimensional contour of the pulmonary fissure in each sagittal image can be realized by any method known in the art for generating a three-dimensional contour from two-dimensional contours in mutually orthogonal directions, and is not the focus of the present invention, so a detailed description thereof will be omitted.

[0055] According to this embodiment, a two-dimensional contour of the target structure in the second plane direction image is generated based on an interactive shortest path algorithm such as the LiveWire algorithm, so that a three-dimensional contour of the target structure can be generated with high accuracy based on anatomical information of the organs of the scanned body in the three-dimensional scan image.

[0056] Furthermore, according to this embodiment, a user can generate a three-dimensional contour of a target structure in a three-dimensional scan image simply by drawing a first initial contour and a second initial contour, thereby reducing the user's workload and time consumption.

[0057] (Second embodiment) According to the three-dimensional contour drawing method of the first embodiment, the contour of the target structure in the scanned image can be automatically generated with high accuracy. However, in the first embodiment, the contour of the target structure is drawn only on the second plane direction image that includes both the intersection with the first initial contour and the intersection with the second initial contour. Therefore, if the contour of the target structure in the start image and the end image is too short, there will not be enough second plane direction images to draw the contour of the target structure, and the complete contour of the target structure cannot be drawn.

[0058] FIG. 9 illustrates a problem with the three-dimensional contour drawing method according to the first embodiment. In the starting image shown in FIG. 9(a), the lung is partially missing, so the contours of the pulmonary fissures are very short and discontinuous. FIG. 9(b) shows the end image, and FIG. 9(c) shows the contours of the pulmonary fissures in an intermediate image automatically generated from the start and end images, using thick solid lines. In the starting image, the contours of the pulmonary fissures do not cover the central part of the lung region, so only the contours of some of the pulmonary fissures are generated in the intermediate image.

[0059] Therefore, in the three-dimensional contour drawing method according to the second embodiment of the present invention, after obtaining the first surface direction contour, the first surface direction contour is first expanded, and then three-dimensional contour drawing is performed using the expanded first surface direction contour.

[0060] That is, the three-dimensional contour drawing method according to the second embodiment of the present invention further includes a first surface direction contour extension step for extending the first surface direction contour by the endpoints of the first initial contour and the second initial contour, and in the three-dimensional contour drawing step, the three-dimensional contour of the target structure is obtained using the extended first surface direction contour.

[0061] Next, a three-dimensional contour drawing method according to a second embodiment will be described, focusing on the differences from the first embodiment and omitting a description of the commonalities with the first embodiment. In the description of the second embodiment, the same parts as those in the first embodiment will be denoted by the same symbols.

[0062] 10 is a flowchart showing an example of a three-dimensional contour drawing method according to the second embodiment. Compared to the first embodiment, the image segmentation method according to the second embodiment further includes step S601 between step S600 and step S700.

[0063] In step S601, the first-plane orientation contour drawing unit 170 extends the two-dimensional contour of the target structure in the first-plane orientation image. Specifically, for each first-plane orientation image, the first-plane orientation contour drawing unit 170 determines whether the two-dimensional contour of the target structure in the corresponding first-plane orientation image has reached a boundary position, and if the boundary position has not been reached, identifies an extension point approximately ahead in the direction of travel of the two-dimensional contour of the target structure, and generates an extended contour using the LiveWire algorithm using this extension point and an end point of the two-dimensional contour of the target structure.

[0064] An example of expanding the contour of a pulmonary fissure in a sagittal plane image will be described below with reference to the drawings. Fig. 11 is a diagram for explaining the contour expanding method according to the second embodiment.

[0065] First, the distance between the sagittal plane image to be expanded and the starting image is calculated, and this distance is designated as distance d1. Then, the distance between the initial image and the ending image is calculated, and this distance is designated as distance d2. Hereinafter, this sagittal plane image to be expanded is referred to as the expanded image.

[0066] Next, the end point X1 of the contour of the interlobar fissure in the dilated image is identified.

[0067] Next, a straight line is fitted based on the endpoint X1 and 5 to 10 points on the contour of the interlobar fissure adjacent to the endpoint X1, and a straight line L is obtained. x Generate.

[0068] Next, the line L x The intersection point between the line and the lung area is X2, and the line L x The intersection point between the circumscribed rectangle of the lung region and X2 is taken as X3, and the distance between X2 and X3 is calculated, and this distance is taken as distance d3.

[0069] Next, the projection P of the end points of the first initial contour and the second initial contour in the extended image is calculated. start and P end Determine.

[0070] Next, X2 to P start In the direction towards, let P' be a point that is a distance d3 away from X2. start Let X2 be P end In the direction towards, let P' be a point that is a distance d3 away from X2. end Let's say.

[0071] Next, P' start From P' end Calculate the vector up to and set this vector as vector V.

[0072] Next, the position of X4 is calculated using the following formula (1).

[0073]

number

[0074] Next, X2 is set as the start point and X4 is set as the end point, and the shortest path between X2 and X4 is calculated using the LiveWire algorithm. Of this path, the part that overlaps with the lung field region is set as the contour to be expanded.

[0075] According to this embodiment, a complete three-dimensional profile of the target structure can be obtained.

[0076] (Variation) In the above-described embodiment, the shortest path is calculated using the LiveWire algorithm, but the shortest path may be calculated using another interactive shortest path algorithm instead of the LiveWire algorithm.

[0077] In the above-described embodiment, the first plane direction is the sagittal plane direction and the second plane direction is the coronal plane direction, but the first plane direction may be the coronal plane direction and the second plane direction may be the sagittal plane direction. Furthermore, the first plane direction and the second plane direction may be directions obtained by tilting the sagittal plane direction and the coronal plane direction.

[0078] In the above-described embodiment, an example has been described in which a first initial contour and a second initial contour are drawn in the start image and the end image, respectively. However, the embodiment is not limited to this. For example, the initial contour drawing unit 150 may draw an initial contour only in either the start image or the end image. That is, the initial contour drawing unit 150 may draw an initial contour only in one first-face orientation image. Alternatively, the initial contour drawing unit 150 may draw an initial contour in three or more first-face orientation images.

[0079] Even when only one initial contour is drawn, the second face direction contour drawing unit 160 draws a second face direction contour for each of the plurality of second face direction images based on the initial contour. Here, the second face direction contour drawing unit 160 can draw the second face direction contour by extending the drawn initial contour to the second face direction image using any method. In an example where only one initial contour is drawn, the workload on the user can be further reduced.

[0080] In the above-described embodiment, an example has been described in which a second surface direction contour is drawn, and then a first surface direction contour is drawn based on the second surface direction contour, and a three-dimensional contour of a target structure is drawn based on the initial contour, the first surface direction contour, and the second surface direction contour. However, the embodiment is not limited to this, and drawing of the first surface direction contour may be omitted. In other words, the three-dimensional contour drawing device 100 shown in FIG. 1 may not include the first surface direction contour drawing unit 170.

[0081] When the drawing of the first surface direction contour is omitted, the three-dimensional contour drawing unit 180 draws the three-dimensional contour based on the initial contour and the second surface direction contour. Even in this case, the three-dimensional contour drawing unit 180 can draw the three-dimensional contour using any method of interpolating between curves.

[0082] In the above embodiment, the second surface direction image is a cross-sectional image parallel to the second surface direction, but the second surface direction image may include a cross-sectional image substantially parallel to the second surface direction. [Explanation of symbols]

[0083] 1: Tubular organ annotation system 10: Projection device 20: Mapping matrix acquisition device 30: Annotation device 40: Reverse mapping device 100: Three-dimensional contour drawing device 110:Display section 120: Input section 130: Storage section 140: Initial setting section 150: Initial contour drawing section 160: Second surface direction contour drawing unit 170: First surface direction contour drawing section 180: 3D contour drawing section

Claims

1. an initial contour drawing unit that draws a contour of a target structure as a first initial contour and a second initial contour, respectively, in a start image and an end image among a plurality of first plane direction images that are cross-sectional images in the first plane direction of the three-dimensional scan image, in accordance with a drawing operation by a user; a second plane direction contour drawing unit that automatically draws a second plane direction contour of the target structure based on the first initial contour and the second initial contour for each of a plurality of second plane direction images that are cross-sectional images in a second plane direction of the three-dimensional scanned image; a first surface direction contour drawing unit that automatically draws a first surface direction contour of the target structure on each of the first surface direction images between the start image and the end image among the plurality of first surface direction images based on the second surface direction contour; a three-dimensional contour drawing unit that automatically draws a three-dimensional contour of the target structure based on the first initial contour, the second initial contour, the second planar contour, and the first planar contour; A medical image processing device comprising:

2. 2. The medical image processing device according to claim 1, wherein the second surface direction contour drawing unit draws the second surface direction contour for each of the plurality of second surface direction images using an interactive shortest path algorithm, with the intersections with the first initial contour and the second initial contour as starting points and ending points, respectively.

3. 2. The medical image processing device according to claim 1, wherein the three-dimensional contour drawing unit extends the first planar direction contour based on endpoints of the first initial contour and the second initial contour, and generates the three-dimensional contour using the extended first planar direction contour.

4. The medical image processing device described in claim 3, wherein the three-dimensional contour drawing unit determines an extension point approximately forward in the direction of travel of the first surface direction contour, and generates an extended contour using an interactive shortest path algorithm using the extension point and an end point of the first surface direction contour.

5. The first plane direction is a sagittal plane direction, and the second plane direction is a coronal plane direction. The medical image processing device according to any one of claims 1 to 4.

6. The medical image processing apparatus according to claim 1 , wherein the first plane direction is a coronal plane direction and the second plane direction is a sagittal plane direction.

7. The medical image processing apparatus according to claim 2 , wherein the interactive shortest path algorithm is a LiveWire algorithm.

8. The medical image processing apparatus according to claim 1 , wherein the second planar direction image further includes a cross-sectional image substantially parallel to the second planar direction.

9. In each of a start image and an end image among a plurality of first plane direction images, which are cross-sectional images in the first plane direction of the three-dimensional scan image, a contour of the target structure is drawn as a first initial contour and a second initial contour, respectively, in accordance with a drawing operation by a user; automatically drawing a second plane direction contour of the target structure based on the first initial contour and the second initial contour for each of a plurality of second plane direction images which are cross-sectional images in a second plane direction of the three-dimensional scanned image; automatically drawing a first surface direction contour of the target structure on each of the first surface direction images between the start image and the end image among the plurality of first surface direction images based on the second surface direction contour; automatically drawing a three-dimensional contour of the target structure based on the first initial contour, the second initial contour, the second planar contour, and the first planar contour; A medical image processing method comprising:

Images