Tubular object segmentation method, medical image processing device and program

Abstract

To improve the accuracy of segmentation of tubular objects. [Solution] The tubular object segmentation method includes: a first step of acquiring a medical image including a tubular object; a second step of acquiring the centerlines and endpoints on the centerlines of the main trunk and free branches of the tubular object; a third step of acquiring candidate branches by subtracting the main trunk of the tubular object from the second main trunk, and using the centerlines of the candidate branches to acquire the other endpoints of the candidate branches excluding the endpoints of the second main trunk; and a fourth step of determining whether the distance of each endpoint of each branch of the candidate branch from the corresponding endpoint on the main trunk of the tubular object is less than a second threshold, and if it is less than the second threshold, connecting the candidate branch to the corresponding endpoint of the main trunk of the tubular object and displaying it.

Description

Technical Field

[0001] The embodiments disclosed in the present specification and drawings relate to a tubular object segmentation method, a medical image processing apparatus, and a program.

[0002] Specifically, the present embodiment relates to a tubular object segmentation method, a medical image processing apparatus, and a program for segmenting a tubular object in a medical image based on deep learning.

Background Art

[0003] By performing three-dimensional reconstruction of tubular objects such as blood vessels based on medical images acquired by a medical image diagnostic apparatus, it is possible to assist doctors in quickly locating the positions of organs, tissues, tumors and the like in clinical medical research and applications. Therefore, rapid and accurate segmentation of tubular objects is an important task in medical image processing.

[0004] As computer technology advances, conventional algorithms used for segmenting tubular objects such as blood vessels generally include deep learning related algorithms that modify the loss function or modify the network structure, algorithms that utilize deep learning and tracking, in which initial segmentation is performed by a CNN (Convolutional Neural Network), the centerline of the initial segmentation is acquired, and the initial position and tracking direction are combined to realize detection of distal blood vessels, and algorithms that perform deep learning to generate candidate tubular objects, in which candidate tubular objects are generated based on a Hessian matrix or a probability map, and the disconnection of the candidate tubular objects is improved or the information amount at the peripheral portion of the tubular objects is increased by a connected region matching method.

[0005] However, the above algorithms still have several drawbacks. For example, algorithms that modify the loss function or network configuration based on the above deep learning improve the connectivity of tubular objects to some extent, but they cannot adequately address issues such as foreground-background imbalance in tubular object segmentation (e.g., when tubular objects are particularly thin) or the low probability of distal microtubular objects in the network output probability map. The above tracking algorithm is suitable for scenes where distal tubular objects can be extended, there are few tubular branching points, tubular objects are thick, and tubular object contrast is prominent, but it cannot adequately handle cases where there are many fine branching points distally, where the branching direction of tubular objects changes in various ways, or where tubular objects break at points other than the ends of the centerline. The above algorithm for generating candidate tubular objects has the drawback that, when using a Hessian matrix, it is computationally intensive and unsuitable when time performance is a high requirement, and when image contrast is poor, erroneous tubular objects extracted from the Hessian matrix tend to introduce noise. Furthermore, when using probability maps, although the probability maps are output from the model, setting a lower threshold provides richer connectivity information than the model's default results. Therefore, compared to the previous two methods, candidate tubular structures can be found more accurately on the image, the process of extracting tubular structures can be omitted, and the operation is faster. However, at lower thresholds, probability maps contain more uncertain information from the model. If the main trunk portion of tubular structures (i.e., the maximum connectivity area) at low thresholds is used as is, in addition to improving vascular connectivity and peripheral richness, unrelated vessels may be enlarged or noise may be introduced.

[0006] Therefore, when segmenting tubular objects using the conventional deep learning algorithm described above, if the tubular objects are widely distributed and vary in thickness, the thinner sections are particularly prone to fracture. Furthermore, if there is some kind of disease, or if the image enhancement effect is low, the image intensity of tubular objects in different regions may not match, potentially leading to fracture. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Chinese Patent Application Publication No. 106296664 Specification [Patent Document 2] Chinese Patent Application Publication No. 110047078 Specification [Patent Document 3] Chinese Patent Application Publication No. 116485700 Specification [Overview of the project] [Problems that the invention aims to solve]

[0008] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to improve the accuracy of segmentation of tubular objects. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0009] The tubular object segmentation method according to the embodiment includes a first step of acquiring a medical image including a tubular object, a second step of outputting a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a communication region with a relatively large volume or a relatively large number of branches, and free branches, which are communication regions other than the main trunk of the tubular object that do not communicate with the main trunk of the tubular object, and acquiring the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively, and acquiring the region in the probability map that is within the range of a first threshold as the tubular object, and the acquired results The method includes: a third step of extracting a second main trunk which is a connecting region with a relatively large volume or a relatively large number of branches from the fruit, obtaining the center line and endpoints on the second main trunk, obtaining candidate branches by subtracting the tubular main trunk from the second main trunk, and using the center line of the candidate branches to obtain the other endpoints of the candidate branches excluding the endpoints of the second main trunk; and a fourth step of determining whether the distance of each endpoint of each branch of the candidate branch from the corresponding endpoint on the tubular main trunk is less than a second threshold, and if it is less than the second threshold, connecting the candidate branch to the corresponding endpoint of the tubular main trunk and displaying it. [Brief explanation of the drawing]

[0010] [Figure 1A] Figure 1A is a schematic diagram showing the network segmentation results according to the first embodiment. [Figure 1B] Figure 1B is a schematic diagram showing the main vascular region in the network segmentation results according to the first embodiment. [Figure 1C] Figure 1C is a schematic diagram showing free branches in the network segmentation results according to the first embodiment. [Figure 2] Figure 2 is a schematic diagram showing the procedure for determining and connecting candidate branches according to the first embodiment. [Figure 3] Figure 3 is a schematic diagram showing the procedure for identifying and connecting candidate branches related to the modification. [Figure 4A]FIG. 4A is a schematic diagram showing a network segmentation result according to the second embodiment. [Figure 4B] FIG. 4B is a schematic diagram showing the main tracheal trunk in the network segmentation result according to the second embodiment. [Figure 4C] FIG. 4C is a schematic diagram showing the center line and end points of the main tracheal trunk in the network segmentation result according to the second embodiment. [Figure 4D] FIG. 4D is a schematic diagram showing the center line and end points of a free branch in the network segmentation result according to the second embodiment. [Figure 5A] FIG. 5A is a schematic diagram showing a second main trunk according to the second embodiment. [Figure 5B] FIG. 5B is a schematic diagram showing the center line of the second main trunk according to the second embodiment. [Figure 5C] FIG. 5C is a schematic diagram showing candidate branches according to the second embodiment. [Figure 5D] FIG. 5D is a schematic diagram showing the center line and end points of candidate branches according to the second embodiment. [Figure 6A] FIG. 6A is a schematic diagram showing a procedure for determining and connecting candidate branches according to the second embodiment. [Figure 6B] FIG. 6B is a schematic diagram showing a procedure for determining and connecting candidate branches according to the second embodiment. [Figure 6C] FIG. 6C is a schematic diagram showing a procedure for determining and connecting candidate branches according to the second embodiment. [Figure 6D] FIG. 6D is a schematic diagram showing a procedure for determining and connecting candidate branches according to the second embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of the configuration of a medical image processing apparatus according to the present embodiment. [Mode for Carrying Out the Invention]

[0011] The present application was made in view of the problems of the above-described prior art, and aims to provide a deep learning-based tubular object segmentation method, medical image processing apparatus, and program that improve the connectivity of tubular object segmentation results through post-processing, and by designing a plurality of thresholds in the post-processing, can selectively connect tubular object rupture regions based on the tubular object centerline and endpoints, reduce rupture phenomena or extend peripheral portions, and remove irrelevant noise. Note that the tubular object targeted by the present embodiment may include a branched structure, or may not include a branched structure.

[0012] To achieve the above objective, one of the technical solutions of the present embodiment is a tubular object segmentation method characterized by comprising: a first step of acquiring a medical image containing a tubular object; a second step of outputting a tubular object probability map for tubular object data, and a network segmentation result including a tubular object main trunk which is a communication region with the largest volume or the most abundant branches among the segmentation results, and free branches which are tubular object portions that are communication regions not communicating with the tubular object main trunk other than the tubular object main trunk, and acquiring the respective centerlines of the tubular object main trunk and the free branches, and endpoints on the centerlines; a third step of thresholding the tubular object probability map, acquiring regions within a predetermined first threshold range in the tubular object probability map as tubular objects, extracting a second main trunk which is a communication region with the largest volume or the most abundant branches from the acquired results, acquiring the centerline of the second main trunk and endpoints on the centerline, acquiring candidate branches by subtracting the tubular object main trunk from the second main trunk, and acquiring other endpoints of the candidate branches excluding the endpoint of the second main trunk using the centerlines of the candidate branches; a fourth step of determining whether the distance from the corresponding endpoint on the tubular object main trunk to each endpoint of each branch of the candidate branches is smaller than a predetermined second threshold, and if the distance is smaller than the second threshold, connecting and displaying the relevant candidate branch to the corresponding endpoint of the tubular object main trunk. Note that the above-mentioned tubular object main trunk may be a communication region with a relatively large volume or relatively abundant branches.

[0013] Another technical proposal of this embodiment includes a first step of acquiring a medical image including a tubular object, a second step of outputting a tubular object probability map for the tubular object data, and a network segmentation result that includes the tubular object main trunk, which is the communication region with the largest volume or the most branched, and free branches, which are tubular object parts that are communication regions other than the tubular object main trunk and do not communicate with the tubular object main trunk, and obtaining the center line and endpoints on the center line of the tubular object main trunk and free branches, respectively, and thresholding the tubular object probability map, acquiring the region within a predetermined first threshold range in this tubular object probability map as a tubular object, and from the acquired results, the region with the largest volume or the most branched The tubular object segmentation method is characterized by comprising: a third step of extracting a second main trunk which is a communication region, obtaining the center line and endpoints on the center line of the second main trunk, obtaining candidate branches by subtracting the tubular main trunk from the second main trunk, and obtaining the other endpoints of the candidate branch excluding the endpoint of the second main trunk using the center line of the candidate branch; and a fourth step of, if a corresponding endpoint which corresponds to the endpoint of the free branch is identified in the candidate branch, determining whether the distance to the corresponding endpoint from the corresponding endpoint in the tubular main trunk is smaller than the second threshold, and if it is smaller than the second threshold, connecting the candidate branch to which the corresponding endpoint belongs to the corresponding endpoint of the tubular main trunk and displaying it. Note that the tubular main trunk described above may be a communication region which has a relatively large volume or a relatively abundant number of branches.

[0014] Another technical proposal of this embodiment is a medical image processing apparatus comprising an image acquisition unit for acquiring a medical image including a tubular object, a processing unit, and a display unit, wherein the processing unit executes the tubular object segmentation method.

[0015] Another technical proposal of this embodiment is a program for causing a computer to execute the tubular object segmentation method.

[0016] As described above, according to the present invention, since a probability map is output using a model, it is possible to include more connectivity information with smaller probability values, while also effectively eliminating noise from the probability map.

[0017] The tubular object segmentation method, medical image processing apparatus, and program according to this application will be described in detail below with reference to the drawings. Here, the tubular objects according to this embodiment include blood vessels (including arteries, veins, etc.), trachea, large intestine, etc. That is, the tubular objects according to this embodiment may include a branched structure or may not include a branched structure.

[0018] (First embodiment) The first embodiment of this application will be described below with reference to Figures 1A to 4D. In the first embodiment, the case in which blood vessels are used as the tubular object will be described as an example.

[0019] The tubular (blood vessel) segmentation method according to the first embodiment includes the following steps:

[0020] Step 1: Obtain a medical image that includes blood vessels (e.g., a CT (Computed Tomography) image).

[0021] Step 2: Output a vascular probability map for the vascular data and network segmentation results including vascular mains and free branches, and obtain the centerlines and endpoints on the centerlines for each of the vascular mains and free branches. Here, the probability map may be output by the model and includes a foreground probability map and a background probability map, which are used to represent the probability that each pixel belongs to the foreground or background, respectively. A vascular main (Figure 1B) is a connected region (or a connected region with a relatively large volume (e.g., the largest) in the segmentation results (Figure 1A) (or a connected region with a relatively large number of branches (e.g., the most abundant (most)): for example, this can be determined by the length of the centerline). Free branches (Figure 1C) are vascular parts that are connected regions other than the vascular main in the segmentation results and do not connect to the vascular main. Network segmentation is a known method of segmenting medical images using a neural network model in deep learning. Furthermore, the main tubular portion (main vascular portion) is not limited to the one with the largest volume or the most branches; it may also be arbitrarily selected depending on the situation, such as the one with the second largest volume or the second most branches.

[0022] Step 3: A threshold is set (threshold) on the vascular probability map (in this case, the foreground map), regions within a predetermined first threshold range in the vascular probability map are obtained as blood vessels, and from the resulting binarized probability map, a second main trunk with a relatively large volume (e.g., the largest) (or a relatively large number of branches (e.g., the most abundant)) is extracted, and the centerline of the second main trunk and the endpoints on the centerline are obtained. Then, candidate branches are obtained by subtracting the vascular main trunks obtained by segmentation from the second main trunk, the intersection of the candidate branch and the centerline of the second main trunk is taken, the centerline of the candidate branch is obtained, and the other endpoints in the candidate branch, excluding the endpoints of the second main trunk (the endpoints of the main trunk in the original probability map), are obtained using the centerline of the candidate branch. Finally, candidate branches and other endpoints in the candidate branch, excluding the endpoints of the second main trunk, are obtained. Here, for example, the probability map may be binarized by setting the first threshold to 0.1 to provide richer connectivity information. Furthermore, regarding the method for obtaining the centerlines and endpoints described above, the centerlines are extracted only for the main vascular trunk, free branches, and the second main trunk. The endpoints are then obtained using these centerlines. For candidate branches, the centerline is obtained by taking the intersection of the centerline of the second main trunk and the candidate branch, and the endpoints of the candidate branch are obtained using this centerline. The endpoints of the main vascular trunk and candidate branches are used not only for matching, which will be described later, but also for removing noise that may occur in the foreground map of the probability map binarized with a smaller threshold, which will be described later. Note that the second main trunk is not limited to the one with the largest volume or the most branches; it may be arbitrarily extracted depending on the situation, such as the one with the second largest volume or the second most branches.

[0023] Step 4: As shown in Figure 2, it is determined whether the distance between each endpoint of a candidate branch (for example, the points within the black circles in the left diagram of Figure 2) and the corresponding endpoint on the main vascular trunk (for example, the points within the white circles in the right diagram of Figure 2) is sufficiently small, for example, whether it is less than a predetermined second threshold. If it is less than the second threshold, the candidate branch is connected to the corresponding endpoint on the main vascular trunk and displayed. This allows for more comprehensive vascular segmentation results.

[0024] In the fourth step, the judgment criteria may be further relaxed. That is, if there is no endpoint on the main vascular trunk corresponding to the endpoint of a candidate branch, and the closest distance of the endpoint of the candidate branch from the main vascular trunk is less than a predetermined third threshold (the third threshold may or may not be equal to the second threshold), the candidate branch to which the endpoint belongs may be connected to the closest part of the main vascular trunk and displayed. This allows for more comprehensive vascular segmentation results.

[0025] As a modified example, as shown in Figure 3, in the fourth step, if a corresponding endpoint (e.g., the point in the black circle in the center of Figure 3) is identified in a candidate branch that corresponds to an endpoint in a free branch (e.g., the point in the black circle in the left of Figure 3), it is determined whether the distance of the corresponding endpoint from the corresponding endpoint in the main vascular trunk (e.g., the point in the white circle in the right of Figure 3) is less than a second threshold. If it is less than the second threshold, the candidate branch to which the corresponding endpoint belongs may be connected to the corresponding endpoint in the main vascular trunk and displayed. This makes the vascular segmentation results more accurate.

[0026] To enrich the results of vascular segmentation, in the above modification, if there is no endpoint corresponding to a corresponding endpoint on the main vascular trunk, and the nearest distance of the corresponding endpoint from the main vascular trunk is sufficiently small, for example, less than a predetermined third threshold, the candidate branch to which the corresponding endpoint belongs may be connected to the nearest part of the main vascular trunk and displayed.

[0027] In the fourth step, larger branches among the free branches that do not correspond to candidate branches may also be displayed. Here, for example, a free branch whose communication region is larger than a certain threshold, or whose distance between the two furthest points on the branch is greater than a certain threshold, may be considered a large branch.

[0028] Preferably, after the completion of the fourth step, the fifth step further includes removing noise and displaying the results by using the endpoints of the candidate branches and their corresponding endpoints on the main vessel, or the centerlines of the candidate branches, to remove the portions of the candidate branches connected to the main vessel that would thicken the main vessel.

[0029]

number

[0030] After the fifth step is completed, it is preferable to have a further interaction step in which, if the user clicks on a candidate branch connected to the main vascular trunk in the displayed result image, the system deletes the candidate branch or performs noise reduction processing on the candidate branch. Specifically, if the user determines that a candidate branch connected to the main vascular trunk is noise, or if the user decides not to connect that part, the user can click on that part, and in response to the click operation, the system deletes that part by communication area analysis. Alternatively, if the user wants to keep a candidate branch connected to the main vascular trunk but wants to remove noise because it has occurred, the system can improve the noise in one of the following ways, provide the results, and allow the user to make a selection. In this embodiment, the click operation includes pressing a mouse button once, pressing it twice, drag and drop operations, touch operations on a touch panel (touchscreen), and selection operations using a keyboard.

[0031] By adjusting the fourth threshold value used when removing noise using the centerline of candidate branches to a smaller value, more noise can be removed. Alternatively, if the connected blood vessel is considered too thin to correspond to the blood vessel in the image, the fourth threshold value can be increased to improve the correspondence between the connected blood vessel and the blood vessel in the image.

[0032] If noise was removed in the previous fifth step using the endpoints of candidate branches and their corresponding endpoints on the main vascular trunk, switch to removing noise again using the centerlines of the candidate branches. Alternatively, if noise was previously removed using the centerlines of the candidate branches, switch to removing noise again using the endpoints of candidate branches and their corresponding endpoints on the main vascular trunk.

[0033] Apply a different first threshold to the probability map and binarize it again, then repeat the subsequent steps accordingly.

[0034] Here, if a large branch is also displayed as described above, the system may have an interaction step in which, when the user clicks on the large branch in the displayed result image, the system reduces the first threshold and re-determines whether the large branch can be connected to the main vascular trunk. More specifically, the system may calculate and provide the largest first threshold that allows the large branch to be connected to the main vascular trunk (i.e., the second trunk can include the large branch), or it may provide the user with a selection of connection results at several different first thresholds, including the largest first threshold, or if it determines that the large branch cannot be connected to the main vascular trunk at any small threshold, it may indicate to the user that connection is not possible.

[0035] Additionally, the following interaction steps may be included:

[0036] If a user clicks on the blank space in the displayed result image, it may indicate that the user believes a blood vessel in that area should be connected or extended. In this case, the device reduces the first threshold and re-determines whether or not there is a branch in the blank space that can be connected to the main blood vessel. More specifically, the device may calculate and provide the largest first threshold at which a branch can be connected to the main blood vessel in the blank space (i.e., the second main vessel may include a branch located in the blank space), or it may provide the user with a selection of connection results at multiple different first thresholds, including the largest first threshold, or if it determines that no branch can be connected to the main blood vessel in the blank space at any small threshold, it may indicate to the user that connection is not possible.

[0037] According to the first embodiment described above, compared to the results of segmentation using conventional deep learning, fracture phenomena can be reduced, the peripheral portion can be extended, and irrelevant noise can be removed. In other words, according to the first embodiment, the accuracy of segmentation of tubular objects can be improved.

[0038] (Second embodiment) Next, a second embodiment will be described with reference to Figures 4A to 6D. In the second embodiment, the trachea will be used as an example of a tubular object. Note that some of the same content and drawings as in the first embodiment may be omitted.

[0039] The tubular (tracheal) segmentation method according to the second embodiment includes the following steps:

[0040] Step 1: Obtain medical images including the trachea.

[0041] Step 2: Output a tracheal probability map for the tracheal data and network segmentation results including the tracheal main trunk and free branches, and obtain the centerlines and endpoints on the centerlines of the tracheal main trunk and free branches, respectively (Figures 4C and 4D). Here, the tracheal main trunk (Figure 4B) is a connecting region (or a connecting region with a relatively large volume (e.g., the largest) in the segmentation results (Figure 4A) (or a connecting region with a relatively large number of branches (e.g., the most abundant (most)): can be determined by the length of the centerline). The free branches are tracheal portions that are connecting regions other than the tracheal main trunk and do not communicate with the tracheal main trunk.

[0042] Step 3: A threshold is set (threshold) in the tracheal probability map, and regions within a predetermined first threshold range in the tracheal probability map are acquired as the trachea. From the resulting binarized probability map, the second main trunk (Figure 5A), which has a relatively large volume (e.g., the largest) connecting region (or a relatively large number of branches (e.g., the most abundant) connecting region), is extracted, and the centerline of the second main trunk (Figure 5B) and the endpoints on the centerline are acquired. Then, a candidate branch (Figure 5C) is obtained by subtracting the tracheal main trunk (Figure 4B) from the second main trunk (Figure 5A), and after taking the intersection of the candidate branch and the centerline of the second main trunk, the centerline of the candidate branch is acquired, and the other endpoints of the candidate branch, excluding the endpoints of the second main trunk, are acquired using the centerline of the candidate branch (Figure 5D). Note that the method for acquiring the first threshold, centerline, and endpoints is the same as in the first embodiment, so the explanation is omitted here.

[0043] Step 4: As shown in Figure 6A, it is determined whether the distance between the endpoint of each candidate branch and the corresponding endpoint of the main tracheal trunk is smaller than a predetermined second threshold. If it is smaller than the second threshold, the candidate branch is connected to the corresponding endpoint of the main tracheal trunk and displayed. Here, since the centerlines and endpoints of the main tracheal trunk and free branches were obtained in Step 2, and the centerlines and endpoints of the candidate branches were obtained in Step 3, matching can be performed using these centerlines and endpoints as they are, as shown in Figure 6A. This allows for obtaining more comprehensive tracheal segmentation results.

[0044] Furthermore, as in the modified version of the first embodiment, for example, in the fourth step, if there is no endpoint on the main tracheal canal corresponding to the endpoint of a candidate branch, and the closest distance of the endpoint of the candidate branch from the main tracheal canal is less than a predetermined third threshold (the third threshold may or may not be equal to the second threshold), the candidate branch to which the endpoint belongs may be connected to and displayed at the closest point on the main tracheal canal (Figure 6B). This allows for more comprehensive tracheal segmentation results to be obtained.

[0045] As a modification, in step 4, if a corresponding endpoint is identified in a candidate branch that corresponds to an endpoint in a free branch, it is determined whether the distance of that corresponding endpoint from the corresponding endpoint in the main tracheal trunk is less than a second threshold. If it is less than the second threshold, the candidate branch to which the corresponding endpoint belongs may be connected to the corresponding endpoint in the main tracheal trunk and displayed (Figure 6C). This makes the tracheal segmentation results more accurate.

[0046] To enrich the tracheal segmentation results, in the above modified example, if there is no endpoint corresponding to a given endpoint on the main tracheal canal, and the nearest distance of the given endpoint from the main tracheal canal is sufficiently small, for example, less than a predetermined third threshold, the candidate branch to which the given endpoint belongs may be connected to the nearest part of the main tracheal canal and displayed (Figure 6D).

[0047] While the above methods improve tracheal connectivity and richness, they may also increase the probability of introducing noise. Users can select and use any of these methods depending on the specific situation, such as the model segmentation results and the binarization of probability maps.

[0048] Furthermore, in the fourth step, as in the first embodiment, larger branches among the free branches that do not correspond to candidate branches may also be displayed.

[0049] The subsequent fifth step and the steps for interaction are the same as in the first embodiment, and therefore will not be described.

[0050] According to the second embodiment described above, compared to the results of segmentation by deep learning, it is possible to connect the entire peripheral or fractured trachea, extend the entire trachea, and remove irrelevant noise.

[0051] As described above, according to this embodiment, since a probability map is output using the model, it is possible to include more connectivity information with smaller probability values, while also effectively eliminating noise from the probability map.

[0052] It should be noted that the first and second embodiments described above are merely illustrative and do not limit the present application. For example, the first embodiment may be applied to the trachea and the second embodiment to blood vessels. However, in cases where tubular structures branch from the main trunk rather than from the periphery, or when the candidate trachea is large, it is difficult to select an appropriate cross-section from the entire fractured trachea and remove the noise superimposed on the main trunk of the tubular structure. Therefore, it is preferable to use the method of the second embodiment, which sets the fourth threshold to select which voxel points of the candidate branches can be added to the segmentation result. Furthermore, the present invention may also be realized as an apparatus using the tubular object segmentation method described above. Figure 7 is a block diagram showing an example of the configuration of the medical image processing apparatus 1 according to this embodiment. As shown in Figure 7, for example, the medical image processing apparatus 1 has a communication interface 11, an input interface 12, a display 13, a storage circuit 14, and a processing circuit 15. The medical image processing apparatus 1 can be realized by an information processing device such as a tablet terminal or a workstation.

[0053] The communication interface 11 is connected to the processing circuit 15 and controls the transmission and communication of various data between the system and medical imaging diagnostic equipment, image storage equipment, etc., which are connected via a network (not shown). For example, the communication interface 11 can be implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0054] The input interface 12 is implemented by a trackball for various settings, switch buttons, a mouse, a keyboard, a touchpad for input operations by touching the operating surface, a touch monitor integrating a display screen and a touchpad, a non-contact input circuit using an optical sensor, and an audio input circuit. The input interface 12 is connected to the processing circuit 15 and converts the input operations received from the operator into electrical signals and outputs them to the processing circuit 15. In this specification, the input interface 12 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, a processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to a control circuit is also included as an example of an input interface.

[0055] The display 13 is connected to the processing circuit 15 and displays various information and images output from the processing circuit 15. For example, the display 13 can be implemented as an LCD monitor, a CRT (Cathode Ray Tube) monitor, a touch monitor, etc. For example, the display 13 can display a UI (User Interface) for receiving instructions from the operator, various images, and various processing results from the processing circuit 15.

[0056] The memory circuit 14 is connected to the processing circuit 15 and stores various types of data. For example, the memory circuit 14 can be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or by means of a hard disk or optical disc. For example, the memory circuit 14 stores medical images, including tubular objects. The memory circuit 14 also stores various types of information used in the processing of the processing circuit 15, the processing results from the processing circuit 15, and various programs that are read and executed by the processing circuit 15.

[0057] The processing circuit 15 controls each component of the medical image processing device 1 in response to input operations received from the operator via the input interface 12. The processing circuit 15 stores the medical image output from the communication interface 11 in the storage circuit 14. The processing circuit 15 also reads the medical image from the storage circuit 14 and displays the read medical image on the display 13.

[0058] As shown in Figure 7, the processing circuit 15 performs, for example, an acquisition function 151 and a processing function 152. Here, for example, each processing function performed by the acquisition function 151 and the processing function 152, which are components of the processing circuit 15 shown in Figure 7, is recorded in the memory circuit 14 in the form of a program that can be executed by a computer. The processing circuit 15 is, for example, a processor, and it reads each program from the memory circuit 14 and executes it to realize the function corresponding to each program that has been read. In other words, the processing circuit 15 in the state in which each program has been read will have the functions shown in the processing circuit 15 of Figure 7.

[0059] The acquisition function 151 acquires medical images including tubular objects. That is, the acquisition function 151 performs the processing of the first step described above. Note that the acquisition function 151 is an example of an image acquisition unit.

[0060] The processing function 152 executes each of the processes described in each of the embodiments described above. That is, the processing function 152 executes various processes related to the second step, third step, fourth step, and fifth step described above. Note that the processing function 152 is an example of the first processing unit, second processing unit, and third processing unit.

[0061] It should be noted that the components of each device illustrated in the above description of the embodiments are functional concepts and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those illustrated, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be implemented, in whole or in any part, by a CPU and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

[0062] Furthermore, the method described in the above-mentioned embodiments can be implemented by executing a pre-prepared program on a computer such as a personal computer or workstation. This program can be distributed via a network such as the Internet. Alternatively, this program can be recorded on a computer-readable non-temporary recording medium such as a hard disk, flexible disk (FD), CD-ROM, MO, DVD, USB memory, and SD card memory, and executed by being read from the non-temporary recording medium by a computer.

[0063] According to at least one embodiment described above, the accuracy of segmentation of tubular objects can be improved.

[0064] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0065] 1. Medical image processing device 151 Acquisition function 152 Processing Functions

Claims

1. The first step is to acquire a medical image including a tubular object, The second step involves outputting a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a connected region with a relatively large volume or a relatively large number of branches, and free branches, which are connected regions other than the main trunk of the tubular object that do not connect to the main trunk of the tubular object, and obtaining the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively. A third step involves obtaining the region within the range of the first threshold in the probability map as the tubular object, extracting a second main trunk which is a connecting region with a relatively large volume or a relatively large number of branches from the obtained results, obtaining the center line and endpoints on the center line of the second main trunk, obtaining candidate branches by subtracting the tubular object main trunk from the second main trunk, and using the center line of the candidate branch to obtain the other endpoints of the candidate branch excluding the endpoints of the second main trunk, A fourth step involves determining whether the distance from the corresponding endpoint on the main trunk of the tubular object to each endpoint of the candidate branch is less than a second threshold, and if it is less than the second threshold, connecting the candidate branch to the corresponding endpoint on the main trunk of the tubular object and displaying it; A tubular object segmentation method, including [the specified method].

2. The fourth step is to connect and display the candidate branch to the nearest part of the tubular main trunk if there is no endpoint on the main trunk of the tubular object that corresponds to the endpoint of the candidate branch, and the distance of the endpoint of the candidate branch from the main trunk of the tubular object is less than the third threshold. A tubular object segmentation method according to claim 1, comprising the following:

3. The first step is to acquire a medical image including a tubular object, The second step involves outputting a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a connected region with a relatively large volume or a relatively large number of branches, and free branches, which are connected regions other than the main trunk of the tubular object that do not connect to the main trunk of the tubular object, and obtaining the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively. A third step involves obtaining regions within the range of a first threshold in the probability map as tubular objects, extracting a second main trunk which is a connected region with a relatively large volume or a relatively large number of branches from the obtained results, obtaining the center line and endpoints on the center line of the second main trunk, obtaining candidate branches by subtracting the tubular main trunk from the second main trunk, and using the center line of the candidate branch to obtain the other endpoints of the candidate branch excluding the endpoints of the second main trunk, A fourth step is to determine whether the distance from the corresponding endpoint on the main trunk of the tubular object to the corresponding endpoint is less than a second threshold when a corresponding endpoint is identified in the candidate branch, and if it is less than the second threshold, connect the candidate branch to which the corresponding endpoint belongs to the corresponding endpoint on the main trunk of the tubular object and display it. A tubular object segmentation method, including [the specified method].

4. The fourth step is to connect and display the candidate branch to which the corresponding endpoint belongs to the closest part of the main trunk of the tubular object if the endpoint corresponding to the corresponding endpoint does not exist on the main trunk of the tubular object and the distance of the corresponding endpoint from the main trunk of the tubular object is less than the third threshold. A tubular object segmentation method according to claim 3, comprising the following:

5. Centerlines are extracted for the main trunk of the tubular object, the free branches, and the second main trunk; the endpoints of each are obtained using the centerlines; the centerlines of the candidate branches are obtained by intersection of the centerlines of the candidate branches and the second main trunk; and the endpoints of the candidate branches are obtained using the centerlines of the candidate branches. A method for segmenting a tubular object according to any one of claims 1 to 4, including the following:

6. The fourth step is performed using the centerlines and endpoints of the main trunk of the tubular object, the free branches, and the candidate branches, respectively. A method for segmenting a tubular object according to any one of claims 1 to 4, including the following:

7. The fourth step involves displaying, among the free branches, branches of a size that do not correspond to the candidate branches. A method for segmenting a tubular object according to any one of claims 1 to 4, including the following:

8. A fifth step in which noise is removed and the image is displayed by using the endpoints on the main trunk of the tubular object that correspond to the endpoints of the candidate branches, or by using the center line of the candidate branches, to remove the parts of the branches connected to the main trunk of the tubular object that would thicken the main trunk of the tubular object. A tubular object segmentation method according to any one of claims 1 to 4, further comprising:

9. The fifth step, when removing noise using the endpoints on the main trunk of the tubular object corresponding to the endpoints of the candidate branches, involves subtracting the endpoints on the main trunk of the tubular object from the endpoints of the candidate branches to obtain a vector, obtaining a corresponding normal plane from the obtained vector and the endpoints on the main trunk of the tubular object, and retaining only the communication region on the endpoint side of the candidate branches of the normal plane, or, when removing noise using the center line of the candidate branches, calculating the shortest distance between each voxel point of the tubular object to be denoised and the corresponding point on the center line of the candidate branch, and excluding the voxel point from the connection result if the shortest distance is greater than the fourth threshold. A tubular object segmentation method according to claim 8, comprising the following:

10. In response to a user click on a candidate branch connected to the main trunk of the tubular object in the displayed result image, the candidate branch is deleted. A tubular object segmentation method according to any one of claims 1 to 4, further comprising:

11. In response to a user click on a candidate branch connected to the main trunk of the tubular object in the displayed result image, the fourth threshold is changed, and the noise reduction process on the candidate branch is performed again and displayed. The tubular object segmentation method according to claim 9, further comprising:

12. In response to a user click on a candidate branch connected to the main trunk of the tubular object in the displayed result image, the fifth step involves switching between using the endpoint of the candidate branch and the corresponding endpoint on the main trunk of the tubular object, or using the center line of the candidate branch, and then removing the noise and displaying the result image again. The tubular object segmentation method according to claim 8, further comprising:

13. In response to a user click operation on a branch connected to the main trunk of the tubular object in the displayed result image, the first threshold in the third step is modified, and the third step is performed again. The tubular object segmentation method according to claim 8, further comprising:

14. In response to a user click on a branch in the displayed result image that does not correspond to the candidate branch, the first threshold is reduced, and it is determined again whether a branch of a size that does not correspond to the candidate branch can be connected to the main trunk of the tubular object. The tubular object segmentation method according to claim 7, further comprising:

15. A step of displaying the largest first threshold that allows a branch of a size not corresponding to the candidate branch to be connected to the main trunk of the tubular object, The tubular object segmentation method according to claim 14, further comprising:

16. A step of calculating and displaying connection results for a plurality of different first thresholds, including the largest first threshold, The tubular object segmentation method according to claim 15, further comprising:

17. If it is determined that a branch of a size not corresponding to the candidate branch cannot be connected to the main trunk of the tubular object by reducing the first threshold, the step of indicating that connection is not possible, The tubular object segmentation method according to claim 14, further comprising:

18. In response to a user click on a margin in the displayed result image, the first threshold is reduced to determine again whether or not there is a branch connected to the main trunk of the tubular object in the margin. A tubular object segmentation method according to any one of claims 1 to 4, further comprising:

19. A step of calculating and displaying the maximum first threshold value at which a branch connected to the main trunk of the tubular object can appear in the margin, The tubular object segmentation method according to claim 18, further comprising:

20. A step of displaying connection results at a plurality of different first thresholds, including the largest first threshold, The tubular object segmentation method according to claim 19, further comprising:

21. By reducing the first threshold, if it is determined that there are no branches in the margin that can be connected to the main trunk of the tubular object, the step of indicating that connection is not possible, The tubular object segmentation method according to claim 18, further comprising:

22. The tubular object segmentation method according to any one of claims 1 to 4, wherein the tubular object is a blood vessel and / or a trachea.

23. An image acquisition unit that acquires medical images including tubular objects, A first processing unit outputs a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a connected region with a relatively large volume or a relatively large number of branches, and free branches, which are connected regions other than the main trunk of the tubular object that do not communicate with the main trunk of the tubular object, and acquires the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively. A second processing unit acquires the region within the range of a first threshold in the probability map as the tubular object, extracts a second main trunk which is a connected region with a relatively large volume or a relatively large number of branches from the acquired results, acquires the center line and endpoints on the center line of the second main trunk, acquires candidate branches by subtracting the tubular object main trunk from the second main trunk, and uses the center line of the candidate branch to acquire the other endpoints of the candidate branch excluding the endpoints of the second main trunk, A third processing unit determines whether the distance from the corresponding endpoint on the main trunk of the tubular object to each endpoint of the candidate branch is less than a second threshold, and if it is less than the second threshold, connects the candidate branch to the corresponding endpoint on the main trunk of the tubular object and displays it. A medical image processing device equipped with [a specific feature].

24. Computers, Image acquisition unit that acquires medical images including tubular objects. A first processing unit outputs a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a connected region with a relatively large volume or a relatively large number of branches, and free branches, which are connected regions other than the main trunk of the tubular object that do not communicate with the main trunk of the tubular object, and acquires the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively. A second processing unit acquires the region within the range of a first threshold in the probability map as the tubular object, extracts a second main trunk which is a connected region with a relatively large volume or a relatively large number of branches from the acquired results, acquires the center line and endpoints on the center line of the second main trunk, acquires candidate branches by subtracting the tubular object main trunk from the second main trunk, and uses the center line of the candidate branch to acquire the other endpoints of the candidate branch excluding the endpoints of the second main trunk. A third processing unit is configured to determine whether the distance from the corresponding endpoint on the main trunk of the tubular object to each endpoint of the candidate branch is less than a second threshold, and if it is less than the second threshold, connect the candidate branch to the corresponding endpoint on the main trunk of the tubular object and display it. program.

25. An image acquisition unit that acquires medical images including tubular objects, A first processing unit outputs a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a connected region with a relatively large volume or a relatively large number of branches, and free branches, which are connected regions other than the main trunk of the tubular object that do not communicate with the main trunk of the tubular object, and acquires the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively. A second processing unit acquires regions within a predetermined first threshold range in the probability map as tubular objects, extracts a second main trunk which is a connected region with a relatively large volume or a relatively large number of branches from the acquired results, acquires the center line and endpoints on the center line of the second main trunk, acquires candidate branches by subtracting the tubular main trunk from the second main trunk, and uses the center line of the candidate branch to acquire the other endpoints of the candidate branch excluding the endpoints of the second main trunk, When a corresponding endpoint is identified in the candidate branch that corresponds to an endpoint in the free branch, a third processing unit determines whether the distance from the corresponding endpoint in the main trunk of the tubular object to the corresponding endpoint is less than a second threshold, and if it is less than the second threshold, connects the candidate branch to which the corresponding endpoint belongs to the corresponding endpoint in the main trunk of the tubular object and displays it. A medical image processing device equipped with [a specific feature].

26. Computers, Image acquisition unit that acquires medical images including tubular objects. A first processing unit outputs a probability map of the tubular object and a network segmentation result that includes the main trunk of the tubular object, which is a connected region with a relatively large volume or a relatively large number of branches, and free branches, which are connected regions other than the main trunk of the tubular object that do not communicate with the main trunk of the tubular object, and acquires the centerlines and endpoints on the centerlines of the main trunk of the tubular object and the free branches, respectively. A second processing unit acquires regions within a predetermined first threshold range in the probability map as tubular objects, extracts a second main trunk which is a connected region with a relatively large volume or a relatively large number of branches from the acquired results, acquires the center line and endpoints on the center line of the second main trunk, acquires candidate branches by subtracting the tubular main trunk from the second main trunk, and uses the center line of the candidate branch to acquire the other endpoints of the candidate branch excluding the endpoints of the second main trunk. When a corresponding endpoint is identified in the candidate branch that corresponds to an endpoint in the free branch, the third processing unit determines whether the distance from the corresponding endpoint in the main trunk of the tubular object to the corresponding endpoint is less than a second threshold, and if it is less than the second threshold, it connects the candidate branch to which the corresponding endpoint belongs to the corresponding endpoint in the main trunk of the tubular object and displays it. program.

Images