Image division device, image division method, and program
Patent Information
- Application Number
- JP2024130607
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-24
- Filing Date
- 2024-08-07
- Publication Date
- 2025-03-07
Smart Images
Figure 2025031597000001_ABST
Abstract
Description
[Technical field]
[0001] The embodiments disclosed in this specification and the drawings relate to an image segmentation device, an image segmentation method, and a program. [Background technology]
[0002] Locating a lesion in disease diagnosis is an important process for planning treatment. In the case of the lungs, for example, segmentation of the lungs into lobes and pulmonary segments is important for locating the lesion position.
[0003] In response to this, a conventional technique has been proposed to segment lung segments based on the distance from unlabeled regions to labeled regions (trachea, blood vessels, etc.) within the lung lobes. However, the segmentation results of this conventional technique have fine sawtooth on the boundary, which affects the doctor's observation.
[0004] In addition, a conventional technique has been proposed that performs surface fitting according to the boundary feature points of the lung area labeled by the user to divide the lung area. However, this labeling method is not intuitive and does not directly use information such as the trachea and blood vessels, so the user must confirm the lung area based on the generated surface. In addition, the labeling position varies depending on the doctor (user), which may result in a large difference in the results. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] Christina Stoecker, Stefan Welter, Jan H. Moltz, Bianca Lassen, Jan-Martin Kuhnigk, Stefan Krass, Heinz-Otto Peitgen, “Determination of Lung Segments in Computed Tomography Images Using the Euclidean Distance to the Pulmonary Artery” First published: 13 August 2013 Summary of the Invention [Problem to be solved by the invention]
[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to obtain smooth boundary division results, provide labeling information that is intuitive and easy for doctors to identify, and improve the accuracy of division. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]
[0007] The image segmentation device according to the present embodiment includes an acquisition unit, a rough segmentation unit, a classifier generation unit, and a fine segmentation unit. The acquisition unit acquires a lumpy region to which labeling information has been assigned and a tubular region to which labeling information has been assigned. The rough segmentation unit generates, by distance transformation using the labeling information of the lumpy region and the labeling information of the tubular region, a region to be segmented that is a region to which no labeling information has been assigned and is a boundary region of each substructure of the lumpy region, and a non-boundary region that is a rough segmentation result to which labeling information has been assigned. The classifier generation unit generates a classifier that classifies spatial coordinates using the labeling information of the non-boundary region and the labeling information of a specific position determined by the region to be segmented and the tubular region. The fine segmentation unit uses the classifier to segment voxels of the region to be segmented and determines a final segmentation result. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image segmentation device according to the first embodiment. [Figure 2A] FIG. 2A is a diagram showing an example of an image of a lung lobe region with labeling information. [Figure 2B] FIG. 2B is a diagram showing an example of an image of a trachea region with labeling information. [Diagram 3] FIG. 3 is a flowchart showing the process (image segmentation method) performed by the image segmentation device according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing a method for generating boundary regions and non-boundary regions (rough segmentation results). [Diagram 5] FIG. 5 is a flowchart showing details of the process (step S103) of generating a classifier using labeling information. [Figure 6] FIG. 6 is a diagram showing the process of uniformly sampling the rough segmentation result. [Figure 7] FIG. 7 is a diagram showing a method for generating a specific position with labeling information. [Figure 8] FIG. 8 is a diagram showing the process of generating a classifier using the first input and the second input. [Figure 9] FIG. 9 is a diagram showing the final division result obtained in the first embodiment and the division result of a comparative example for comparison. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of an image segmentation device according to the second embodiment. [Figure 11] FIG. 11 is a flowchart showing the process (image segmentation method) performed by the image segmentation device according to the second embodiment. [Figure 12] FIG. 12 is a comparison diagram showing the results after subdivision before and after empirical labeling and correction based on the empirical labeling. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, embodiments of an image segmentation device, an image segmentation method, and a program will be described with reference to the drawings. The embodiments described below do not limit the present invention. In the following description, components having substantially the same functions and configurations are given the same reference numerals, and are described repeatedly only when necessary. In addition, in each drawing, the same components may be described using different expressions.
[0010] As is commonly known, the lungs, trachea, and blood vessels of the human body have clear physical boundaries, so the difficulty of segmentation is small and they can be manually labeled by doctors or automatically labeled by machine learning. However, for locating lesions, it may be necessary to further segment the lungs into multiple sub-structures (lung segments) based on supply information such as the trachea and blood vessels. The lung lobes, as outer layer structures covering the trachea and blood vessels, have no clear physical boundaries inside, so it is difficult to segment the lung lobes into multiple sub-structures (lung segments).
[0011] Therefore, in this embodiment, an image segmentation device and an image segmentation method are provided. By using this image segmentation device, smooth boundary segmentation results can be obtained, and labeling information that is intuitive and makes it easier for doctors to identify the images can be provided, while improving the accuracy of the segmentation.
[0012] (First embodiment) Fig. 1 is a block diagram showing an example of a configuration of an image segmentation device 10 according to the first embodiment. As shown in Fig. 1, the image segmentation device 10 according to the present embodiment includes an input interface 101, a communication interface 102, a display 103, a processing circuitry 105, and a storage circuitry 104.
[0013] The input interface 101 is realized by a trackball for performing various settings, a switch button, a mouse, a keyboard, a touchpad for performing input operations by touching the operation surface, a touch screen in which a display screen and a touchpad are integrated, a non-contact input circuit using an optical sensor, and a voice input circuit, etc. The input interface 101 is connected to the processing circuit 105, and converts an input operation received from an operator into an electric signal and outputs it to the processing circuit 105. Note that, although the input interface 101 is provided within the image segmentation device 10 in FIG. 1, it may be provided externally.
[0014] The display 103 is connected to the processing circuit 105 and displays various information and various image data output from the processing circuit 105. For example, the display 103 is realized by a liquid crystal monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, or the like. For example, the display 103 displays a GUI (Graphical User Interface) for receiving instructions from an operator, various display images, and various processing results by the processing circuit 105. The display 103 is an example of a display unit. Note that, although the display 103 is provided within the image segmentation device 10 in FIG. 1, it may be provided externally.
[0015] The communication interface 102 is a NIC (Network Interface Card) or the like, and communicates with other devices. For example, the communication interface 102 is connected to the processing circuitry 105, and collects image data from an ultrasound diagnostic device, which is an ultrasound system, and an X-ray CT (Computed Tomography) device and an MRI (Magnetic Resonance Imaging) device, which are modalities other than the ultrasound system, and outputs the image data to the processing circuitry 105.
[0016] The memory circuitry 104 is connected to the processing circuitry 105 and stores various data. Specifically, the memory circuitry 104 stores at least various medical images for image registration and fusion images obtained after registration. For example, the memory circuitry 104 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, a hard disk, an optical disk, or the like. The memory circuitry 104 also stores a program corresponding to each processing function executed by the processing circuitry 105. Note that, although the memory circuitry 104 is provided in the image segmentation device 10 in FIG. 1, it may be provided externally.
[0017] The processing circuitry 105 controls each of the components of the image segmentation device 10 in response to an input operation received from an operator via the input interface 101 .
[0018] For example, the processing circuit 105 is realized by a processor. As shown in FIG. 1, the processing circuit 105 includes an acquisition function 11, a rough division function 12, a classifier generation function 13, a fine division function 14, and an output function 15. Here, the processing functions executed by the acquisition function 11, the rough division function 12, the classifier generation function 13, the fine division function 14, and the output function 15, which are components of the processing circuit 105 shown in FIG. 1, are recorded in the storage circuit 104 of the image segmentation device 10 in the form of a program executable by a computer, for example. The processing circuit 105 is a processor that realizes the processing function corresponding to each program by reading and executing each program from the storage circuit 104. In other words, the processing circuit 105 in a state in which each program has been read has each function shown in the processing circuit 105 of FIG. 1.
[0019] The term "processor" used in the above description means a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). When the processor is a CPU, for example, the processor reads out and executes a program stored in the memory circuit 104 to realize the function. On the other hand, when the processor is an ASIC, for example, instead of storing the program in the memory circuit 104, the program is directly incorporated in the circuit of the processor. Note that each processor in this embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining multiple independent circuits to realize the function. Furthermore, the multiple components in FIG. 1 may be integrated into one processor to realize the function.
[0020] Next, the processes of the acquisition function 11, the rough division function 12, the classifier generation function 13, the fine division function 14, and the output function 15 will be described with reference to Fig. 2A and Fig. 2B. Fig. 2A is a diagram showing an example of an image of a lung lobe region to which labeling information is added. Fig. 2B is a diagram showing an example of an image of a trachea region to which labeling information is added.
[0021] The acquiring function 11 acquires, from the medical image, a mass region to which labeling information has been added and a tubular region to which labeling information has been added. Here, the acquiring function 11 is an example of an "acquiring unit."
[0022] A lumpy region is a region that can cover a tubular region anatomically. In the case of the lung, for example, a lumpy region may be a lobe, and a tubular region may be a trachea or blood vessel. In the present embodiment, the trachea is used as an example, but a blood vessel may be used as a tubular region. Hereinafter, a lobe may be referred to as a lobar region, and a trachea may be referred to as a trachea region.
[0023] In this embodiment, the lobe region is divided into a first number of regions, and each region is assigned with different labeling information as an independent labeling region. Also, the trachea region is divided into a second number of regions, and each region is assigned with different labeling information as an independent labeling region.
[0024] The labeling region referred to above can be understood as a set of voxels with the same labeling information. For example, if all voxels in a lung lobe region are assigned with "RA", the region can be represented as an "RA labeling region", and if all voxels in a lung section are assigned with "r1", the lung section can be represented as an "r1 labeling region".
[0025] Specifically, in a lumpy region, the lung generally includes a left lung and a right lung, and the left lung and the right lung are divided into five lobes (lung segments) as the above-mentioned first quantity segment by interlobar fissures having clear physical boundaries. Here, among the five lobes, the left lung has two lobes, and the right lung has three lobes. In this embodiment, the lumpy region with labeling information refers to five labeling regions obtained by assigning different labeling information to each of the five lobes. For example, in the lobe region shown in FIG. 2A, among the three lobes of the right lung, the labeling information of "RA" is assigned to the voxels belonging to the right segment 1 lobe (upper lobe of the right lung), the labeling information of "RB" is assigned to the voxels belonging to the right segment 2 lobe (middle lobe of the right lung), and the labeling information of "RC" is assigned to the voxels belonging to the right segment 3 lobe (lower lobe of the right lung). In addition, of the two lobes of the left lung, the voxels belonging to the left 1st lobe (upper lobe of the left lung) are given labeling information of "LD", and the voxels belonging to the left 2nd lobe (lower lobe of the left lung) are given labeling information of "LE". In addition, in Fig. 2A, for the convenience of observation, each voxel having the same labeling is labeled with the same color, which allows the user to intuitively distinguish different lung segments by color.
[0026] In addition, in the tubular region, the trachea is divided into 18 regions as the second quantity region according to the supply relationship. In this embodiment, the tubular region with labeling information refers to 18 labeling regions obtained by assigning different labeling information to each of the 18 regions. For example, in the tracheal region shown in FIG. 2B, of the 18 regions, the bronchi belonging to the 8 regions on the left side are assigned labeling information of "l1" to "l8", and the bronchi belonging to the 10 regions on the right side are assigned labeling information of "r1" to "r10". Note that in FIG. 2B, for example, different labeling information is expressed only by color, and the description of labeling information by letters such as "l1" to "l8" and "r1" to "r10" is omitted.
[0027] The rough segmentation function 12 uses labeling information of the lobe (lobe region shown in FIG. 2A) which is a lumpy region and labeling information of the trachea (trachea region shown in FIG. 2B) which is a tubular region to generate a region to be segmented without labeling information and a rough segmentation result with labeling information by distance transformation. For example, the rough segmentation function 12 uses labeling information of the lobe and labeling information of the trachea to generate a region to be segmented which is a region to which labeling information is not assigned and which is a boundary region of each substructure (lung section) of the lobe and a non-boundary region which is a rough segmentation result to which labeling information is assigned by distance transformation. Here, the rough segmentation function 12 is an example of a "rough segmentation unit".
[0028] Distance transformation is an algorithm for estimating distance as an existing technology. The coarse division function 12 uses the algorithm to determine voxels whose difference in distance to two adjacent and different labeling regions is less than a threshold as regions to be divided. Specifically, the coarse division function 12 uses the algorithm to calculate voxels whose difference in distance to two adjacent and different labeling regions is less than a threshold, and determines a set of voxels at these positions as a boundary region. After that, the coarse division function 12 determines the boundary region as a region to be divided without leaving the original labeling information in the boundary region (without adding labeling information).
[0029] Then, the coarse segmentation function 12 determines the remaining area excluding the boundary area as a non-boundary area, and leaves the original labeling information. Specifically, the non-boundary area, which is the remaining lung area excluding the boundary area, is segmented into a plurality of labeling areas corresponding to the branching structure of the trachea by distance transformation. The plurality of labeling areas corresponding to the non-boundary area obtained by distance transformation correspond to the coarse segmentation result in this embodiment. The respective labeling areas of the trachea and the non-boundary area are assigned with corresponding labeling information.
[0030] Through the above-mentioned distance transformation, the remaining lung region is converted from the original five labeling regions into 18 labeling regions (l1 to l8, r1 to r10) corresponding to the branching structure of the trachea. As a result, the non-boundary region generated by the rough division function 12 is divided into a second quantity of "18" regions, and each region is assigned different labeling information as an independent labeling region.
[0031] The classifier generating function 13 generates a classifier for classifying spatial coordinates using labeling information in the rough segmentation result (labeling information of non-boundary regions) and labeling information of a specific position determined by the region to be segmented and the tubular region (trachea). Here, the classifier generating function 13 is an example of a "classifier generating unit."
[0032] The classifier can be obtained by using, for example, a conventional SVM (Support Vector Machine) algorithm. The principle is to visualize the classifier as a hyperplane. For example, the labeling information in the rough segmentation result and the labeling information at a specific position are used as the first input and the second input, respectively, and a hyperplane is generated such that a voxel of labeling information of a certain classification among the labeling information (l1 to l8, r1 to r10) in the rough segmentation result and a voxel of labeling information of the same classification at a specific position are located on the same side of the hyperplane, and the labeling information on both sides of the hyperplane belong to different classifications.
[0033] As the first input, voxels obtained by uniformly sampling voxels in the rough segmentation result are used. Here, for each sampled voxel, the labeling information to which it belongs is added after the spatial 3D coordinates. For example, a voxel in the right 1st lobe (upper lobe of the right lung) is assigned the labeling information "r1", so the point set of the voxel is {(x1 r1 ,y1 r1 ,z1 r1 ),(x2 r1 ,y2 r1 ,z2 r1For example, the voxel in the right 3rd lobe (lower lobe of the right lung) is given the labeling information “r3”, so the point set of the voxel is {(x1 r3 ,y1 r3 ,z1 r3 ),(x2 r3 ,y2 r3 ,z2 r3 ),···}.
[0034] As the second input, all voxels at a specific position are used without sampling. Here, for all voxels at a specific position, the labeling information to which they belong is added after the spatial 3D coordinates. For example, a voxel in the right 1st lobe (upper lobe of the right lung) is assigned the labeling information "r1", so the point set of the voxel is {(x1 r1 ,y1 r1 ,z1 r1 ),(x2 r1 ,y2 r1 ,z2 r1 For example, the voxel in the right 3rd lobe (lower lobe of the right lung) is given the labeling information “r3”, so the point set of the voxel is {(x1 r3 ,y1 r3 ,z1 r3 ),(x2 r3 ,y2 r3 ,z2 r3 ),···}.
[0035] The specific position is a position where the trachea overlaps with the region to be divided, which is a boundary region. Therefore, the part of the trachea that overlaps with the region to be divided is set as the specific position, and the labeling information at that position in the trachea is set as the labeling information of the specific position. The region to be divided is a set of voxels in which the difference in distance to two adjacent and different labeling regions is less than a threshold, and the specific position is a part of the region to be divided. Therefore, the voxels at the specific position are voxels in which the difference in distance to adjacent and different labeling regions is less than a threshold, and are important for distinguishing between the two adjacent labeling regions. Therefore, when the voxels at the specific position are used as input samples, all voxels are used as they are without sampling.
[0036] In this way, the classifier generation function 13 uniformly samples the non-boundary region for each different labeling information, and uses each sampled voxel as a first input, and also uses the labeling information of a specific position identified by the region to be divided and the tubular region (trachea) to which labeling information has been added as a second input without sampling, to generate a classifier that classifies spatial coordinates.
[0037] The generated classifier is a hyperplane that classifies voxels based on spatial three-dimensional position information and labeling information. Specifically, the features used in the first input and the second input as input samples used in generating the classifier (hyperplane) are spatial three-dimensional coordinate information and labeling information, and the spatial three-dimensional coordinate information and the labeling information in the first input and the second input are mutually related. Therefore, when classifying spatial coordinates using such a classifier, the spatial three-dimensional coordinate information can be used as an input to estimate the classification (labeling information) to which it belongs. In other words, the generated classifier can estimate the labeling information to which it belongs using voxels in the region to be divided that have only spatial three-dimensional coordinate information.
[0038] The subdivision function 14 divides the voxels of the region to be divided and determines the final division result, using the classifier generated by the classifier generation function 13. Here, the subdivision function 14 is an example of a "subdivision section".
[0039] The classifier, i.e., the hyperplane, can divide the region to be divided without labeling information into multiple regions, and can infer the labeling information to which each region belongs from the spatial three-dimensional coordinate information of each region. Therefore, all voxels in the region to be divided are given different labeling information by the hyperplane. That is, the subdivision function 14 uses the hyperplane to determine the labeling information of each voxel in the region to be divided based on the spatial three-dimensional position information of each voxel in the region to be divided, and the determined labeling information of each voxel in the region to be divided is combined with the labeling information of the non-boundary region (rough division result) to form the final division result.
[0040] The output function 15 outputs the final segmentation result generated by the subdivision function 14. For example, the output function 15 presents the final segmentation result to a user (doctor) by displaying it on the display 103. Here, the output function 15 is an example of an "output unit."
[0041] The output function 15 may also cause the display 103 to display the lung lobes with labeling information and the trachea with labeling information acquired by the acquisition function 11. The output function 15 may also cause the display 103 to display the region to be segmented without labeling information and the non-boundary region with labeling information (coarse segmentation result) generated by the rough segmentation function 12. The output function 15 may also cause the display 103 to display the classifier (hyperplane) generated by the classifier generation function 13.
[0042] FIG. 3 is a flowchart showing the process (image segmentation method) performed by the image segmentation device 10 according to the first embodiment.
[0043] First, in step S101 of FIG. 3, the acquisition function 11 acquires a mass region (lobe of the lung) and a tubular region (trachea) with labeling information from a medical image.
[0044] Next, in step S102 of FIG. 3, the coarse segmentation function 12 uses the labeling information of the lung lobes and the labeling information of the trachea to generate, by distance transformation, a region without labeling information that is the boundary region of the lung section as the region to be segmented, and further generates a non-boundary region (coarse segmentation result) with labeling information that is the remaining region excluding the boundary region.
[0045] Next, in step S103 of FIG. 3, the classifier generation function 13 generates a classifier that classifies spatial coordinates using labeling information of the non-boundary region and labeling information of the boundary region (region to be divided) and a specific position determined by the trachea.
[0046] Next, in step S104 of FIG. 3, the subdivision function 14 divides the voxels in the boundary region using a classifier to determine the final division result.
[0047] Finally, in step S105 of FIG. 3, the output function 15 outputs the obtained final segmentation result.
[0048] Hereinafter, the process (image segmentation method) by the image segmentation device 10 according to this embodiment will be described in detail with reference to Figs. 4 to 9. Fig. 4 is a diagram showing a method for generating a boundary region and a non-boundary region (rough segmentation result). Fig. 5 is a flowchart showing details of the process (step S103) for generating a classifier using labeling information. Fig. 6 is a diagram showing a process for uniformly sampling the rough segmentation result. Fig. 7 is a diagram showing a method for generating a specific position to which labeling information is added. Fig. 8 is a diagram showing a process for generating a classifier using a first input and a second input. Fig. 9 is a diagram showing the final segmentation result obtained in the first embodiment and the segmentation result of a comparative example for comparison.
[0049] In step S101 of FIG. 3, the acquisition function 11 acquires images of a mass region (lung lobe) and a tubular region (trachea). Here, the lung lobe is divided into a first number of regions "5", and each region is assigned with different labeling information as an independent labeling region. In addition, the trachea is divided into a second number of regions "18", and each region is assigned with different labeling information as an independent labeling region. The above-mentioned first number is "5", and the above-mentioned second number is "18", and in step S101, as shown in the upper diagram of FIG. 4, lung lobe images having five types of different labeling information and trachea images having 18 types of different labeling information are acquired.
[0050] Next, in step S102 of FIG. 3, the coarse division function 12 uses the images of the lung lobes with five kinds of labeling information and the images of the trachea with 18 kinds of labeling information obtained in step S101 to generate the boundary regions of the lung regions as regions to be divided by distance transformation, as shown in the middle diagram of FIG. 4. Specifically, if the difference in distance between one voxel and two different substructures (lung regions) is less than a certain threshold, it means that the voxel is close to the boundary of the substructure. Here, if the voxels close to the boundary are simply divided using distance transformation, the division result will have obvious sawtooth, which will affect the observation by the doctor. For this reason, in this embodiment, for the voxels close to the boundary, the boundary is left as the region to be divided, and in step S104 of FIG. 3, the fine division function 14 divides the region to be divided.
[0051] Next, the rough segmentation function 12 uses the image of the lung lobe to which five kinds of labeling information are assigned and the image of the trachea to which 18 kinds of labeling information are assigned to generate a non-boundary region (rough segmentation result) that is the remaining region after removing the region to be segmented, as shown in the lower diagram of FIG. 4, by distance transformation. The non-boundary region is segmented into a plurality of regions according to the branching structure of the trachea, which is segmented into a plurality of regions. Specifically, in each lung lobe, the lung lobe is segmented into a plurality of lung regions based on the supply relationship of the trachea, and the five lung lobe regions are segmented into a total of 18 lung regions, that is, lung regions having 18 kinds of different labeling information. The remaining region after removing the region to be segmented, that is, the non-boundary region, is assigned with labeling information.
[0052] Next, the process of the image segmentation device 10 proceeds to step S103 in Fig. 3. The process of step S103 includes steps S1031 to S1034 in Fig. 5. Fig. 5 shows the processes of steps S1031 to S1034.
[0053] In step S1031 of FIG. 5, the classifier generating function 13 uniformly samples the labeling information of the rough segmentation result, and sets it as a first input.
[0054] In step S1032 of FIG. 5, the classifier generation function 13 identifies a specific position with labeling information by using a boundary region without labeling information and the trachea with labeling information.
[0055] In step S1033 of FIG. 5, the classifier generating function 13 uses the labeling information of the specific position as a second input without sampling.
[0056] In step S1034 of FIG. 5, the classifier generating function 13 generates a classifier using the first input and the second input.
[0057] Next, steps S1031 to S1034 will be described with reference to FIGS.
[0058] As shown in FIG. 6, in step S1031, the classifier generation function 13 uniformly samples the rough segmentation result (non-boundary region) with labeling information, and divides the point set for the sampled voxels according to the lung segment to which they belong. For each voxel belonging to the right 1 segment lung lobe (upper lobe of the right lung) shown in the upper diagram of FIG. 6, the voxel is given the labeling information "r1" to which it belongs, and therefore, in terms of spatial three-dimensional coordinates, the point set for the voxel is expressed as {(x1 r1 ,y1 r1 ,z1 r1 ),(x2 r1 ,y2 r1 ,z2 r1 ),...}. In addition, for the voxels belonging to the right 3-segment lobe (lower lobe of the right lung) shown in the lower diagram of FIG. 6, the labeling information “r3” is assigned to the voxels, so the point set of the voxels is {(x1 r3 ,y1 r3 ,z1 r3 ),(x2 r3 ,y2 r3 ,z2 r3 ),...}. In step S1031, the classifier generating function 13 receives the above-mentioned sampled voxels having spatial three-dimensional position information and labeling information as the above-mentioned first input.
[0059] As shown in the left diagram of FIG. 7, in step S1032, the classifier generation function 13 uses the region to be divided and the trachea with labeling information to determine a specific position and labeling information that the specific position has. The specific position is a position where the trachea and the region to be divided, which is a boundary region, overlap, and the part of the trachea that overlaps with the region to be divided is set as the specific position, and the labeling information of the position of the trachea (the part that overlaps with the region to be divided) is set as the labeling information of the specific position. The voxel at the specific position is a voxel at a position where the difference in distance to the adjacent labeling regions is less than a threshold, so it is important to distinguish between two adjacent labeling regions. Therefore, when it is used as an input sample, it is not sampled, and all voxels are used as they are. Therefore, as shown in the right diagram of FIG. 7, in step S1033, the classifier generation function 13 sets all voxels at the specific position having spatial three-dimensional position information and labeling information as the above-mentioned second input.
[0060] As shown in FIG. 8, in step S1034, the classifier generating function 13 generates a classifier for classifying spatial coordinates using the first input obtained in step S1031 and the second input obtained in steps S1032 and S1033. As the generated classifier, a plurality of hyperplanes shown on the right side of FIG. 8 can be obtained. In the plurality of hyperplanes, labeling information belonging to the same classification is located on the same side of the hyperplane, and labeling information on both sides of the hyperplane belongs to different classifications, and the hyperplane is determined by an input having spatial three-dimensional position information and labeling information. In this way, there are a plurality of hyperplanes, and for each hyperplane, voxels located on the same side have the same labeling information, and voxels located on different sides have different labeling information. This has the effect that each space divided by the hyperplane corresponds to a unique and determined classification. Note that the right diagram of FIG. 8 also shows the trachea divided by the hyperplane, but this trachea is illustrated to show the effect of the hyperplane and is not part of the generated classifier.
[0061] Next, in step S104 of Fig. 3, the subdivision function 14 uses a hyperplane to determine labeling information of each voxel of the region to be divided based on the spatial three-dimensional position information of each voxel of the region to be divided, and the determined labeling information of each voxel of the region to be divided is combined with the rough division result (non-boundary region to which labeling information is added) to form the final division result. Specifically, the region to be divided shown in the upper part of Fig. 7 is divided into 18 lung regions having different labeling information by using the generated classifier. The division result of the region to be divided is combined with the division result of the non-boundary region (rough division result) to form the final division result.
[0062] The left side of Fig. 9 shows a final segmentation result obtained using the image segmentation method of the first embodiment, and the boundaries of the above-mentioned segmentation result are smooth as shown in the left side of Fig. 9. On the other hand, the right side of Fig. 9 shows a final segmentation result when an image is segmented using only distance transformation, and as shown in the right side of Fig. 9, sawtooth patterns exist at the boundaries of each segmented sub-structure, which affects the doctor's observation.
[0063] By using the image segmentation device 10 and image segmentation method according to this embodiment, it is possible to obtain a segmentation result with smooth boundaries. Furthermore, compared with a method in which a user (operator) labels boundary feature points and performs surface fitting, the labeling method according to this embodiment is more intuitive, can provide labeling information that makes it easier for doctors to identify, and can improve segmentation accuracy.
[0064] Second embodiment 10 is a block diagram showing an example of the configuration of an image segmentation device 20 according to the second embodiment. The difference between the second embodiment and the first embodiment is that the image segmentation device 20 requires a user to input experience labeling in addition to the components of the image segmentation device 10 according to the first embodiment.
[0065] Specifically, as shown in FIG. 10, the processing circuit 105 of the image segmentation device 20 includes an acquisition function 21, a coarse segmentation function 22, a classifier generation function 23, a fine segmentation function 24, a reception function 25, and an output function 26.
[0066] The processes of the above-mentioned acquisition function 21, rough division function 22, classifier generation function 23, fine division function 24, and output function 26 are the same as those of the acquisition function 11, rough division function 12, classifier generation function 13, fine division function 14, and output function 15 in the first embodiment, so duplicated explanations will be omitted. The following description will focus on the process of having the user input experience labeling.
[0067] Fig. 12 is a comparative diagram before and after empirical labeling is performed on the result after subdivision and correction is made based on the empirical labeling. As shown in the upper diagram of Fig. 12, in the final segmentation result obtained by processing (image segmentation method) by the image segmentation device 10 according to the first embodiment, the segmented boundary may not overlap with the vein. However, in reality, it is desired to obtain a segmentation result in which the segmented boundary overlaps with the vein. Therefore, it is necessary to correct the final segmentation result.
[0068] For example, a user inputs experience labeling using the input interface 101. The reception function 25 receives the experience labeling input by the input interface 101. The input interface 101 may be an input interface that allows a user to input experience labeling, and the user can artificially increase the experience labeling by using the input interface 101. Here, the input interface 101 is an example of an "input unit."
[0069] The user can use the input interface 101 at different stages of image segmentation. For example, after the subdivision function 24 generates a final segmented image and the output function 25 outputs the final segmented image, the user can use the input interface 101 to add empirical labeling as shown in the middle diagram of FIG. 12 for the boundary positions that do not match the expected ones, and the classifier generation function 23 can use the additional empirical labeling instead of the existing labeling to obtain a better segmentation result. Alternatively, the acquisition function 21 can acquire a lung lobe image with labeling information and a trachea image with labeling information, and also acquire the empirical labeling via the input interface 101. The empirical labeling inputted via the input interface 101 is used as labeling information for a specific position in the generation of a classifier.
[0070] According to the second embodiment, human intervention can be added during automatic segmentation of an image, and more accurate segmentation can be achieved simply by adjusting minute local positions.
[0071] Next, a second embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing a process (image segmentation method) by the image segmentation device 20 according to the second embodiment.
[0072] 11, the acquisition function 21 acquires at least the lung lobes and the trachea with labeling information. Here, in step S201, in addition to the lung lobes and the trachea with labeling information, empirical labeling may also be acquired via the input interface 101.
[0073] 11, the coarse segmentation function 22 uses labeling information of at least the lung lobes and the trachea to generate a boundary region without labeling information by distance transformation, and further, the remaining region is treated as a non-boundary region with labeling information. Also, the empirical labeling obtained in step S201 may be used in step S202.
[0074] In step S203 of FIG. 11, the classifier generating function 23 generates a classifier using the labeling information of the non-boundary region and the labeling information of the specific position. As described above, the specific position is a voxel at a position where the distance difference to the adjacent labeling region is less than a threshold, and is important for distinguishing between two adjacent labeling regions. In addition, the empirical labeling is a voxel of an important region artificially added, and both are essentially the same. Therefore, the labeling information of the specific position in step S203 further includes the empirical labeling artificially added in addition to the labeling information of the specific position specified by the boundary region and the trachea.
[0075] In step S204 of FIG. 11, the subdivision function 24 divides the voxels in the boundary region using a classifier to determine a final division result, and in step S205 of FIG. 11, the output function 25 outputs the obtained final division result.
[0076] The image segmentation method of the second embodiment further includes step S206 in which the user inputs empirical labeling via the input interface 101 after outputting the final segmentation result in step S205 and displaying it on a display or the like. Therefore, in the image segmentation device 20 of the second embodiment, the input empirical labeling is used again to generate a classifier, thereby updating the generated classifier, and the final segmentation result can be determined and output again by using the updated classifier (hyperplane).
[0077] This results in a segmentation result in which the boundaries shown in the lower diagram of FIG. 12 are pasted onto the veins.
[0078] According to the image segmentation device 20 and image segmentation method of the second embodiment, in addition to the effects of the first embodiment, it is possible to easily correct local segmentation results that do not match expectations, thereby eliminating the need for the user to make global corrections and improving the accuracy of image segmentation.
[0079] (Other embodiments) Each component of each device illustrated in the present embodiment is a functional concept, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to that illustrated, and all or a part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.
[0080] The method described in this embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be provided by being loaded in advance on a ROM (Read Only Memory) or a memory circuit. This program can also be recorded on a non-transitory computer-readable recording medium such as a CD (Compact Disc)-ROM, a FD (Flexible Disk), a CD-R (Recordable), or a DVD (Digital Versatile Disc) as a file in a format that can be installed or executed on these devices, and can be executed by being read from the recording medium by the computer.
[0081] This program may also be stored in a computer connected to a network such as the Internet and provided or distributed by downloading it via the network.
[0082] According to at least one of the embodiments described above, it is possible to obtain smoother boundary division results than when dividing an image using only distance transformation. Also, compared to a method in which the boundary feature points are labeled by the user (operator) and curved surface fitting is performed, it is possible to give the operator an intuitive feeling. Also, since empirical labeling can be added at different stages, it is possible to easily obtain a more accurate division result.
[0083] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]
[0084] 10 Image Segmentation Device 11 Acquisition Function 12 Rough division function 13 Classifier generation function 14 Subdivision function
Claims
1. an acquisition unit that acquires a block region to which labeling information has been added and a tubular region to which labeling information has been added; a coarse division unit that uses the labeling information of the lumpy region and the labeling information of the tubular region to generate, by distance transformation, a region to be divided that is a region to which no labeling information is assigned and that is a boundary region of each sub-structure of the lumpy region, and a non-boundary region that is a result of rough division to which labeling information is assigned; a classifier generating unit that generates a classifier for classifying spatial coordinates using labeling information of the non-boundary region and labeling information of a specific position determined by the region to be divided and the tubular region; a subdivision unit that uses the classifier to divide the voxels of the region to be divided and determines a final division result; An image segmentation device comprising:
2. the lumpy region acquired by the acquisition unit is a region capable of anatomically covering the tubular region, the lumpy region is divided into a first number of regions, and each region is assigned different labeling information as an independent labeling region; The tubular region acquired by the acquisition unit is divided into a second number of regions, and each region is assigned different labeling information as an independent labeling region; The non-boundary region generated by the rough division unit is divided into the second number of regions, and each region is assigned different labeling information as an independent labeling region.
2. The image segmentation device according to claim 1.
3. the classifier generation unit uniformly samples the non-boundary region for each different labeling information, and uses each sampled voxel as a first input, and generates the classifier by using labeling information of the specific position determined by the region to be divided and the tubular region to which labeling information has been added as a second input without sampling the labeling information.
3. An image segmentation device according to claim 1 or 2.
4. The mass region is a lung lobe and the tubular region is a trachea or a blood vessel.
2. The image segmentation device according to claim 1.
5. The non-boundary region is divided into a plurality of labeling regions corresponding to the branching structure of the tubular region by a distance transform.
5. The image segmentation device according to claim 4.
6. Corresponding labeling information is provided to each of the labeling regions of the tubular region and the non-boundary region.
6. An image segmentation device according to claim 5.
7. an output unit that outputs the final division result; The image segmentation device according to claim 1 , further comprising:
8. The coarse division unit determines, as the region to be divided, a voxel in which a difference in distance to two adjacent but different labeling regions is less than a threshold value.
4. The image segmentation device according to claim 3.
9. The specific position is all voxels in a portion of the tubular region that overlaps with the region to be divided.
2. The image segmentation device according to claim 1.
10. The classifier is a hyperplane that classifies voxels based on spatial three-dimensional position information and labeling information.
2. The image segmentation device according to claim 1.
11. the hyperplane is multiple, For each hyperplane, voxels located on the same side have the same labeling information, and voxels located on different sides have different labeling information. The image segmentation device according to claim 10.
12. the subdivision unit determines labeling information of each voxel of the region to be divided based on spatial three-dimensional position information of each voxel of the region to be divided using the hyperplane, and combines the determined labeling information of each voxel of the region to be divided with the rough division result to form the final division result. The image segmentation device according to claim 11.
13. an input unit, which is an input interface that allows a user to add experience labeling; The image segmentation device according to claim 1 , further comprising:
14. The empirical labeling inputted via the input unit is used as labeling information of the specific position in generating the classifier. The image segmentation device according to claim 13.
15. An acquisition step of acquiring a block region to which labeling information is assigned and a tubular region to which labeling information is assigned; a coarse division step of generating, by distance transformation using the labeling information of the lumpy region and the labeling information of the tubular region, a region to be divided which is a region to which no labeling information is assigned and which is a boundary region of each substructure of the lumpy region, and a non-boundary region which is a result of rough division to which labeling information is assigned; a classifier generating step of generating a classifier for classifying spatial coordinates using labeling information of the non-boundary region and labeling information of a specific position determined by the region to be divided and the tubular region; a subdivision step of dividing the voxels of the region to be divided using the classifier to determine a final division result; An image segmentation method comprising:
16. In the coarse division step, a voxel in which the difference in distance to two adjacent but different labeling regions is less than a threshold is determined as the region to be divided without adding labeling information, and the remaining region excluding the boundary region is determined as the non-boundary region, and the original labeling information is left. The image segmentation method according to claim 15.
17. An acquisition step of acquiring a block region to which labeling information is assigned and a tubular region to which labeling information is assigned; a coarse division step of generating, by distance transformation using the labeling information of the lumpy region and the labeling information of the tubular region, a region to be divided which is a region to which no labeling information is assigned and which is a boundary region of each substructure of the lumpy region, and a non-boundary region which is a result of rough division to which labeling information is assigned; a classifier generating step of generating a classifier for classifying spatial coordinates using labeling information of the non-boundary region and labeling information of a specific position determined by the region to be divided and the tubular region; a subdivision step of dividing the voxels of the region to be divided using the classifier to determine a final division result; A program that causes a computer to execute the following.
18. In the coarse division step, a voxel in which the difference in distance to two adjacent but different labeling regions is less than a threshold is determined as the region to be divided without adding labeling information, and the remaining region excluding the boundary region is determined as the non-boundary region, and the original labeling information is left.
18. The program according to claim 17.