Tubular organ labeling method, tubular organ labeling result correction method, and system

Abstract

The present application provides a tubular organ labeling method, a tubular organ labeling result correction method and system. The tubular organ labeling method comprises: a projection step of performing local density projection on a two-dimensional image constituting three-dimensional body data containing a tubular organ to obtain a projection image; a mapping matrix acquisition step of acquiring a mapping matrix of the local density projection performed in the projection step; a labeling step of labeling the tubular organ in the projection image; and an inverse mapping step of inversely mapping the tubular organ labeled in the projection image to the three-dimensional body data by using the mapping matrix to generate a three-dimensional tubular organ image.

Description

Technical Field

[0001] This invention relates to a method for annotating tubular organs, a method for correcting tubular organ annotation results, and a system. In particular, it relates to a method for annotating tubular organs guided by local intensity projection, a method for correcting tubular organ annotation results, and a system. Background Technology

[0002] Human organs and tissues include tubular organs or tissue structures, such as blood vessels and trachea. In this specification, these tubular organs or tissue structures are collectively referred to as tubular organs. Accurate annotation and segmentation of tubular organs, especially in three-dimensional volumetric data, are of paramount value for the diagnosis and surgical treatment of diseases involving tubular organs—such as blood vessels.

[0003] Patent document 1 (US2017 / 0178405A) describes a method for marking the centerline of a tubular structure. In patent document 1, a contrast agent is injected into the bloodstream to enhance the image display of the vessel centerline. Furthermore, patent document 1 uses an iterative algorithm to find the path from the starting position to the ending position located at the centerline of the tubular structure. However, this method in patent document 1, which uses traditional algorithms such as iteration to trace a vessel centerline through a start and end point, is difficult to apply to small, tortuous, or intersecting vessels.

[0004] Patent Document 2 (CN101732061) discloses an image processing device and method for image diagnosis of vascular diseases. Patent Document 2 proposes a technique for generating display image data from volume data based on a determined viewing direction and position. This involves performing three-dimensional image processing on the volume data, such as MPR (Multiplanar Reconstruction), CPR (Curved Planar Reconstruction), SPR (Stretched CPR), volume rendering, surface rendering, and MIP (Maximum Intensity Projection), to generate the display image data. Furthermore, Patent Document 2 also describes the inclusion of labels for blood vessels; however, these "blood vessel labels" in Patent Document 2 are annotations displayed on the image that indicate the anatomical classification names of blood vessel branches.

[0005] Existing technologies include other methods for labeling and segmenting tubular organs such as blood vessels, particularly data-driven artificial intelligence (AI) methods for tubular organ labeling and segmentation proposed in recent years. However, due to the complex geometric and topological variations and the presence of minute structures in tubular organs such as blood vessels, labeling blood vessels in 3D volumetric data remains a very challenging task. Summary of the Invention

[0006] Problems in the prior art

[0007] Manually labeling blood vessels in images constituting three-dimensional volumetric data requires sifting through multiple images, examining each vessel from its root to its endpoint, which is extremely time-consuming. Existing techniques, such as those in Patent Document 1, use traditional algorithms to segment blood vessels and then manually edit them to obtain ground truth (GT). However, the quality of GT in some publicly available datasets is not very high. Furthermore, due to the complex shape, minute structure, and presence of noise / disease clutter in blood vessels, manual inspection and modification of vascular GT is also very time-consuming and inefficient.

[0008] Figure 1 This diagram illustrates a method for manually annotating three-dimensional blood vessels in the prior art. Figure 1 The image shown is a series of consecutive scans of blood vessels in the lungs, for example. When manually annotating blood vessels, the user needs to continuously scan and trace the vessel points in each image, from root to branch, following the order indicated by the arrows. This tracing is difficult due to the presence of other blood vessels and clutter in the images, often resulting in lost tracking and the easy omission of small branches. Even if the user can obtain a vessel centerline through tracing, it will be a very time-consuming process.

[0009] Means for solving technical problems

[0010] This invention addresses the problems of the prior art. It proposes a method and system for annotating tubular organs under local density projection guidance, including a method for correcting the annotation results. First, it generates local maximum / minimum density projection (LMIP) images of tubular organs such as blood vessels, and extracts the index mapping matrix between these LMIP images and the original 3D volume data. Then, the user annotates key points (landmarks) on the centerline of the tubular organs in these LMIP images. The index mapping matrix is ​​then used to back-map these key points on the centerline back to the original 3D volume data to obtain seed points for the blood vessel centerline. Using the various sub-points of the obtained blood vessel centerline, the blood vessel centerline is fitted to obtain the 3D annotation of the blood vessel under the guidance of the blood vessel centerline.

[0011] Specifically, according to one aspect of the present invention, a method for annotating tubular organs is provided, characterized by comprising the following steps: a projection step, performing local density projection on a two-dimensional image constituting three-dimensional volume data containing a tubular organ to obtain a projected image; a mapping matrix acquisition step, acquiring the mapping matrix of the local density projection performed in the projection step; an annotation step, annotating the tubular organ in the projected image; and an inverse mapping step, using the mapping matrix to inverse map the tubular organ annotated in the projected image to the three-dimensional volume data to generate a three-dimensional image of the tubular organ.

[0012] Therefore, this invention utilizes the property that local density projection can enhance the intensity and geometric connectivity of tubular organs (such as blood vessels and trachea) in an image. Key points on the centerline of the tubular organ are annotated in the LMIP image, thus easily distinguishing the centerline of the tubular organ from other clutter (other blood vessels / disease areas / noise). Furthermore, because LMIP enhances geometric connectivity, even minute blood vessel points can be annotated. The annotated center points in the LMIP are then back-mapped to the original image using a constructed index relation mapping matrix, thereby generating the centerline of the tubular organ in three-dimensional space.

[0013] According to another aspect of the present invention, a method for correcting the annotation results of tubular organs is provided, characterized by comprising the following steps: an input step, inputting three-dimensional volume data containing tubular organs and pre-annotation results of tubular organ annotation in the three-dimensional volume data; a projection step, performing local density projection on a two-dimensional image constituting the three-dimensional volume data to obtain a projected image; a mapping matrix acquisition step, obtaining the mapping matrix of the local density projection performed in the projection step; a comparison step, comparing the projected image and the pre-annotation results to obtain the missing portion of the tubular organ in the pre-annotation results; an annotation step, annotating the missing portion of the tubular organ in the projected image; and a reverse mapping step, using the mapping matrix, reverse mapping the missing portion of the tubular organ annotated in the projected image to the three-dimensional volume data, and synthesizing it with the pre-annotation results to generate a three-dimensional tubular organ image.

[0014] The present invention can also be implemented as a tubular organ annotation system, a tubular organ annotation result correction system or device having functional modules that can implement the various steps of the above methods, or as a computer program that enables a computer to execute the steps included in the above methods, or as a recording medium that records the above computer program.

[0015] According to the present invention, key points on the centerline of a tubular organ are labeled in an LMIP image, thereby easily distinguishing the centerline of the tubular organ from other clutter (other blood vessels / disease areas / noise). Furthermore, due to the enhanced geometric connectivity of LMIP, even minute blood vessel points can be labeled. The labeled center points in LMIP are then back-mapped to the original image using a constructed index relation mapping matrix, thereby generating the centerline of the tubular organ in three-dimensional space. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating a method for manually annotating three-dimensional blood vessels in the prior art;

[0017] Figure 2 This is a schematic diagram used to illustrate the concept of maximum density projection;

[0018] Figure 3 This is a flowchart illustrating a method for labeling tubular organs according to a first embodiment of the present invention;

[0019] Figure 4 This is a schematic diagram illustrating the local maximum density projection effect of step S200;

[0020] Figure 5 This is a schematic diagram illustrating the operations performed in steps S400 to S500;

[0021] Figure 6 This is a schematic diagram illustrating the operations performed in step S600;

[0022] Figure 7 This is a schematic diagram illustrating the density projection processing of multiple sets of two-dimensional images according to a variation of the first embodiment;

[0023] Figure 8 This is a functional block diagram illustrating the tubular organ labeling system according to the first embodiment of the present invention;

[0024] Figure 9 This is a flowchart illustrating a method for correcting the annotation results of tubular organs according to a second embodiment of the present invention;

[0025] Figure 10 This is a schematic diagram illustrating the operations performed in steps S300' and S400'. Detailed Implementation

[0026] This invention relates to methods and systems for labeling tubular organs and correcting labeling results. These methods can be implemented using software programs executed by a stand-alone computer or other device with a CPU (central processing unit). These systems can be implemented as such a stand-alone computer or as hardware circuits capable of executing the various steps of the methods. Furthermore, the system of this invention can also be pre-installed as part of a medical image acquisition device such as a magnetic resonance imaging (MRI) device.

[0027] Furthermore, in different embodiments, the same reference numerals are used for the same components, and repeated descriptions are omitted where appropriate.

[0028] Before describing specific embodiments of the present invention, the local maximum / minimum intensity projection (LMIP), which forms the basis of the technical solution of the present invention, will be described first. In this specification, the local maximum / minimum intensity projection is sometimes collectively referred to as local intensity projection.

[0029] Maximum / minimum intensity projection (MIP) is a direct volume rendering (DVR) method that projects the maximum / minimum density values ​​on the projection ray onto the projection plane to visualize volume data. It is widely used in fields such as medical image processing.

[0030] The local maximum / minimum density projection used in this invention is an improvement on MIP. It is not for the complete volume data, but for a part of the volume data. It uses the same method as MIP, that is, when the imaginary light rays pass through the part of the volume data in sequence, the maximum / minimum density values ​​of the part of the volume data are projected onto the projection plane to obtain the projected image.

[0031] The following combination Figure 2 The local density projection in this invention will be explained in detail using local maximum density projection as an example.

[0032] like Figure 2 As shown in (a), for example, five slice images constituting partial volume data, suppose an imaginary ray is emitted from back to front in a direction perpendicular to each slice image (hereinafter referred to as the slice direction), as shown in the figure. The maximum density value in each slice image through which the ray passes is projected onto the projection plane to form a projected image, i.e., an LMIP image. For example... Figure 2(a) shows that the pixel values ​​of five slice images at a certain pixel location are 135, 166, 238, 141, and 169. The maximum value among these five pixel values, "238", is taken as the pixel value at that pixel location in the LMIP image. The above processing is performed on all pixels in the slice images, ultimately obtaining the following... Figure 2 The image (b) shows the LMIP image along the slice direction. It is clear from the image that the projection image obtained after LMIP processing significantly enhances the intensity and geometric connectivity of the blood vessels, and the centerline of the blood vessels can be clearly distinguished from other clutter (other blood vessels / disease areas / noise).

[0033] When generating a local maximum density projection image using N slice images, the pixel value at each pixel location in the projection image originates from one of the N slice images that has the maximum pixel value at that location. Therefore, a correspondence can be established between the image number of the slice image that serves as the source of the pixel value at each pixel location in the LMIP image and that pixel location, forming a structure like... Figure 2 The table shown in (c) is the index mapping matrix of LMIP in this invention (hereinafter sometimes simply referred to as the "mapping matrix").

[0034] The image numbering rule of this invention is as follows: Figure 2 As shown in (a), the middle image in a series of consecutive sliced ​​images is numbered 0, and then sequentially numbered forward and backward according to their distance from the 0th image, as image 1, image -1, image 2, image -2, and so on. When, for example, there are more than one image corresponding to the maximum pixel value at a certain pixel position, the image with the highest number among those images can be used as the index value for that position. For example, if the 1st and -1st images at a certain pixel position have the same pixel value and are both the maximum pixel value, then the index value for that pixel position is set to "1" to form a mapping matrix.

[0035] The above-mentioned image numbering rules and the front and back directions are just one example. Other situations are also applicable to this invention. The mapping matrix in this invention only needs to represent which of the multiple two-dimensional images used to generate the projection image the pixel value of each pixel in the projection image originates from. In other words, it can represent the correspondence between each pixel in the projection image and which of the multiple two-dimensional images used to generate the projection image.

[0036] like Figure 2As shown in (c), the mapping matrix of this invention indicates which of the multiple two-dimensional images used to generate the projected image the pixel value of each pixel in the projected image originates from. Taking an 8*8 pixel slice image as an example, the value of the bottom left cell (table position 0,0) is 0, indicating that in the LMIP image, the pixel value at pixel position (0,0), i.e., the bottom left corner position, comes from the 0th image. In other words, among all 5 images, the pixel value at pixel position (0,0) in the 0th image is the largest. Similarly, the value of the cell immediately to the right of the bottom left cell is -2, indicating that in the LMIP image, the pixel value at pixel position (0,1), i.e., the pixel position immediately to the right of the bottom left corner, comes from the -2nd image. In other words, among all 5 images, the pixel value at pixel position (0,1) in the -2nd image is the largest.

[0037] In addition, such as Figure 2 As shown in (d), for three-dimensional volume data, in addition to the above-mentioned slicing direction ( Figure 2 In addition to the direction ① in the middle, it can also be in the row direction and column direction perpendicular to the slicing direction ( Figure 2 LMIP projection imaging is also performed in directions ② and ③ to obtain LMIP images in the row and column directions.

[0038] As mentioned above, LMIP can not only reconstruct blood vessel points in each slice image into continuous blood vessels, but also enhance the contrast display of small density changes on MIP images. Therefore, it can well display vascular stenosis, dilation, defects, etc., and is particularly suitable for image reconstruction of tubular organs such as blood vessels.

[0039] The above description uses local maximum density projection as an example to illustrate the local density projection of the present invention. Local maximum density projection is used in situations such as... Figure 2 The image shown is used when the target part (blood vessel) is bright, meaning the target part has a higher intensity compared to the background. Conversely, if the target part (blood vessel) is dark, then local minimum density projection is used. The difference between local minimum density projection and local maximum density projection is that local minimum density projection takes the minimum value of the pixel at that pixel position in each slice image through which the light passes, instead of the maximum value, as the pixel value of that pixel position in the projected image. The rest is the same as local maximum density projection, and its details will not be elaborated further.

[0040] Both local maximum density projection and local minimum density projection are applicable to the technical solutions of this invention. The following embodiments use local maximum density projection as the LMIP for description; however, those skilled in the art will clearly understand that replacing local maximum density projection with local minimum density projection in various technical solutions can also constitute the technical solutions of this invention and achieve the same technical effects.

[0041] The following describes specific embodiments of the present invention.

[0042] <First Implementation Method>

[0043] The present invention according to the first embodiment is a method for annotating tubular organs, characterized by comprising: a projection step, wherein a local density projection is performed on a two-dimensional image constituting three-dimensional volume data containing tubular organs to obtain a projected image; a mapping matrix acquisition step, wherein a mapping matrix of the local density projection performed in the projection step is obtained; an annotation step, wherein tubular organs are annotated in the projected image; and an inverse mapping step, wherein the annotated tubular organs in the projected image are inversely mapped to the three-dimensional volume data using the mapping matrix to generate a three-dimensional image of the tubular organs.

[0044] The first embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] Figure 3 This is a flowchart illustrating a method for labeling tubular organs according to a first embodiment of the present invention. The first embodiment of the present invention will be described using the labeling of blood vessels as an example.

[0046] like Figure 3 As shown, the tubular organ annotation method of the present invention includes the following steps: Step S100, inputting original three-dimensional volume data containing blood vessels; Step S200, performing local maximum density projection (LMIP) on multiple consecutive slice images in a certain direction to obtain LMIP images; Step S300, obtaining the projection mapping matrix of the LMIP performed in Step S200; Step S400, annotating the centerline of the blood vessels in the LMIP images; Step S500, using the mapping matrix obtained in Step S300, back-mapping each point on the centerline of the blood vessels marked on the LMIP images in Step S400 to the original slice images to obtain seed points of the centerline of the blood vessels in each slice image; Step S600, completing 3D blood vessel annotation under the guidance of the obtained seed points.

[0047] The above steps are explained in detail below with reference to the accompanying drawings.

[0048] Details regarding steps S200 and S300, including performing local maximum density projection and obtaining the projection mapping matrix of the local maximum density projection, are provided above in conjunction with the appendix. Figure 2 This has already been explained, so I will not repeat it here. Figure 4 This is a schematic diagram illustrating the local maximum density projection effect of step S200.

[0049] Figure 4The three images at the top are axial, sagittal, and coronal scan slices of the pulmonary vessels, respectively. The three images at the bottom are Local Maximum Intensity Projection (LMIP) images obtained by performing LMIP on multiple scan slices of each of the aforementioned axial, sagittal, and coronal planes. As can be seen from the images, by performing LMIP at regular intervals, the obtained LMIP images enhance vascular strength and geometric connectivity, and the vascular centerline can be easily labeled in the LMIP images. It should be noted that in this invention, LMIP can be performed in any direction, such as the aforementioned axial, sagittal, and coronal planes, or other directions specified by the user. For example, the direction of LMIP can be selected based on the orientation of tubular organs (such as blood vessels). If the angle between the direction of the blood vessel and the plane of each scanned slice image is relatively large (e.g., nearly perpendicular), more slice images need to be processed to obtain the complete centerline of the blood vessel. Conversely, if the angle between the direction of the blood vessel and the plane of each scanned slice image is relatively small (e.g., basically parallel), fewer slice images are needed to obtain the complete centerline of the blood vessel. Therefore, it is preferable to perform LMIP processing on scanned slice images where the plane of the blood vessel has a smaller angle with the direction of the blood vessel.

[0050] Figure 5 This is a schematic diagram illustrating the operations performed in steps S400 to S500.

[0051] Figure 5 The image shown in the upper left corner is an LMIP image obtained through step S200, such as an axial LMIP image.

[0052] like Figure 5 As shown, according to the tubular organ annotation method of the present invention, in step S400, the centerline of the blood vessel is annotated in the LMIP image obtained in step S200. For example, key points of the blood vessel centerline can be annotated in the LMIP image manually or by an automatic method known in the art. These key points can be key location points determined based on the shape characteristics of the blood vessel, such as starting points, ending points, obvious inflection points, bifurcation points, etc. In addition, when selecting key points, points overlapping between blood vessels in the LMIP image can be avoided.

[0053] After marking the key points on the center line of the blood vessel in step S400, in step S500, for the pixel position of each marked key point in the LMIP image, the mapping matrix obtained in step S300 is used to backmap each key point to the original three-dimensional volume data.

[0054] The following details the operations within the S500. (Still using...) Figure 2The LMIP of an 8*8 matrix and five images (-2, -1, 0, 1, 2) will be used as an example for explanation.

[0055] Given that an LMIP image is composed of 8x8 pixels, the pixel position of each point in the LMIP image is known, for example... Figure 5 The bright lines in the upper left LMIP image show the key points on the vessel centerline marked in step S400. For key point c1 located at the root of the vessel, assuming c1 is the 4th horizontal and 3rd vertical pixel in an 8*8 pixel array, its pixel position is (4,3). Since the index value corresponding to pixel position (4,3) in the mapping matrix is ​​"2", meaning that pixel position (4,3) takes the pixel value of the "2nd" image out of five images (-2, -1, 0, 1, 2), and the pixel value of the "2nd" image at pixel position (4,3) is the largest, the inverse mapping result of key point c1 is that the point with pixel coordinates (4,3) in the "2nd" image out of multiple consecutive slice images is determined as the seed point on the vessel centerline in three-dimensional space corresponding to c1 in the LMIP image.

[0056] Repeat this process, performing the above-mentioned reverse mapping on all key points marked in the LMIP image in step S400, to obtain the three-dimensional spatial coordinates of various sub-points on the 3D centerline, which are composed of pixel coordinates (e.g., (4,3) above) and image number (e.g., "2" above), thereby obtaining the seed points of the blood vessel centerline in three-dimensional space.

[0057] The above is an explanation of the basic operations in step S500.

[0058] Furthermore, step S500 can also use the seed points on the vessel centerline in the three-dimensional space obtained by inverse mapping to fit the 3D centerline of the vessel in the three-dimensional space using fitting methods known in the art. Additionally, known image processing methods can be used to generate CPR (Curved Planar Reconstruction), SPR (Stretched Curved Planar Reconstruction), and cross-sectional views of the vessel centerline from the fitted 3D centerline for various medical applications. The above-mentioned fitting and generation of CPR, SPR, and cross-sectional views are not mandatory in step S500 and can be omitted.

[0059] Figure 6 This is a schematic diagram illustrating the operations performed in step S600.

[0060] In step S600, three-dimensional blood vessel annotation is completed under the guidance of the obtained seed points. Specifically, as follows: Figure 6 As shown in (a), for example, the centerline fitted according to each seed point obtained in step S500 can be directly used as the sparse labeled blood vessel GT; additionally, as Figure 6 As shown in (b), users can also generate a complete vascular domain based on the fitted vascular centerline through methods such as manual annotation or traditional adaptive thresholding.

[0061] Step S600 can be implemented by any method known in the art. For example, generating a complete vascular domain from the fitted vascular centerline can be achieved by traditional vascular segmentation methods based on image intensity or geometric features, or by deep learning-based segmentation methods. The details will not be elaborated here.

[0062] The above describes the tubular organ annotation method guided by local density projection according to the first embodiment of the present invention. As described above, the present invention utilizes the characteristic that local density projection can enhance the intensity and geometric connectivity of tubular organs (such as blood vessels, trachea, etc.) in an image. Key points on the centerline of the tubular organ are annotated in the LMIP image, thereby easily distinguishing the centerline of the tubular organ from other clutter (other blood vessels / disease areas / noise). Moreover, because LMIP enhances geometric connectivity, even minute blood vessel points can be annotated. The center points annotated in LMIP are then back-mapped to the original image using a constructed index relation mapping matrix, thereby generating the centerline of the tubular organ in three-dimensional space.

[0063] Furthermore, the present invention can also include various modifications. For example, in the first embodiment described above, only one LMIP and subsequent annotation and inverse mapping processing is performed on a set of multiple slice images (e.g., 5 images). However, performing multiple LMIP, annotation, and inverse mapping processing on multiple sets of slice images can yield better 3D vessel centerline annotation results. In addition, performing LMIP processing on only one set of slice images is effective when the annotation object is only one vessel. However, when it is desired to annotate multiple vessels in the entire pulmonary vascular system, the above-mentioned LMIP, annotation, and inverse mapping processing can also be achieved by performing the above-mentioned LMIP, annotation, and inverse mapping processing on multiple sets of slice images separately.

[0064] Figure 7 This is a schematic diagram illustrating density projection processing of multiple sets of two-dimensional images according to a first variation of the first embodiment. For example... Figure 7As shown, in addition to performing local density projection (1) on the two-dimensional image group (1) numbered -2, -1, 0, 1, 2 among the multiple two-dimensional images constituting the three-dimensional volume data as in the first embodiment, to obtain the projected image (1) and obtain the corresponding mapping matrix (1), local density projection (2) to (M) can also be performed on other multiple two-dimensional image groups (2) to (M) to obtain the projected image (2) to (M) and obtain the corresponding mapping matrix (2) to (M) (the local density projection (2) to (M) and the mapping matrix (2) to (M) are omitted in the figure). For example, different blood vessels are labeled in the projected images (2) to (M), and the labeled blood vessels are back-mapped to the original three-dimensional volume data using the corresponding mapping matrix (2) to (M), and finally synthesized into multiple blood vessels, such as those in the pulmonary vascular system.

[0065] The number of images in the aforementioned multiple sets of two-dimensional images can be the same. For example, another set can be formed by shifting each image in the two-dimensional image set (1) forward (or backward) by one (or several) images, and this process can be repeated multiple times to form multiple sets of two-dimensional images. The number of images in the aforementioned multiple sets of two-dimensional images can also be different. For example, the number of images in each set can be appropriately selected based on the length and orientation of the target blood vessel. Furthermore, the above synthesis can be achieved using any method known in the art, which will not be elaborated upon here.

[0066] In other words, according to the first variation of the first embodiment, local density projection can be performed on multiple sets of two-dimensional images to obtain the projected images. Furthermore, in the mapping matrix acquisition step, a mapping matrix is ​​acquired for each of the multiple sets of two-dimensional images. In the annotation step, tubular organs are annotated in the multiple projected images. In the inverse mapping step, the mapping matrix of each set is used to inverse map the tubular organs to three-dimensional volume data, and the resulting images are synthesized into a three-dimensional image of the tubular organs.

[0067] In addition, for example, in the first embodiment described above, only the slice image in one direction of the coronal plane was subjected to LMIP and subsequent annotation and inverse mapping. However, sometimes, for example, due to the influence of the blood vessel orientation, or because some blood vessel parts have spatial intersections with other blood vessels, it is not possible to obtain an accurate blood vessel centerline by LMIP image annotation and inverse mapping in only one direction. In this case, better 3D blood vessel centerline annotation results can be obtained by using LMIP annotation and inverse mapping in multiple directions.

[0068] In other words, according to the second variation of the first embodiment, local density projection can be performed on two-dimensional images constituting three-dimensional volume data in multiple directions to obtain projected images in multiple directions. Furthermore, in the mapping matrix acquisition step, a mapping matrix for each of the multiple directions is acquired. In the annotation step, tubular organs are annotated in each projected image in the multiple directions. In the inverse mapping step, using the mapping matrix for each direction, key points of the LMIP tubular organ centerline in the multiple directions are inversely mapped into the three-dimensional volume data, ultimately synthesizing a three-dimensional image of the tubular organ.

[0069] Furthermore, the number of two-dimensional images used to generate the LMIP image in step S200 is variable and can be selected according to the actual situation. The more two-dimensional images used to generate the LMIP image, the more ideal the intensity of the tubular organs and the geometric connectivity in the obtained LMIP projection image, but the data processing volume also increases accordingly. This invention achieves a balance between data processing volume and LMIP effect by selecting the number of images used to generate the LMIP image according to the actual situation. For example, the denser the distribution of tubular organs, the fewer images are used to generate the LMIP image.

[0070] The tubular organ labeling system of the first embodiment is described below.

[0071] Figure 8 This is a functional block diagram illustrating a tubular organ labeling system according to a first embodiment of the present invention. For example... Figure 8 As shown, the tubular organ annotation system 1 of the first embodiment includes: a projection device 10, which performs local density projection on a two-dimensional image constituting three-dimensional volume data including a tubular organ to obtain a projected image; a mapping matrix acquisition device 20, which acquires the mapping matrix of the local density projection performed by the projection device 10; an annotation device 30, which annotates the tubular organ in the projected image; and a reverse mapping device 40, which uses the mapping matrix to reverse map the tubular organ annotated in the projected image to the three-dimensional volume data to generate a three-dimensional tubular organ image. The processing of each functional module in the tubular organ annotation system of the present invention corresponds to each step of the tubular organ annotation method described above, and will not be described in detail here.

[0072] The above description pertains to the tubular organ annotation method and system of the first embodiment. According to the present invention, key points of the centerline of the tubular organ are annotated in the LMIP image, thereby easily distinguishing the centerline of the tubular organ from other clutter (other blood vessels / disease areas / noise). Furthermore, since LMIP enhances geometric connectivity, even minute blood vessel points can be annotated. The annotated center points in the LMIP are then back-mapped to the original image using a constructed index relation mapping matrix, thereby generating the centerline of the tubular organ in three-dimensional space.

[0073] <Second Implementation Method>

[0074] According to the second embodiment of the present invention, the tubular organ annotation method of the first embodiment is applied to the correction of the tubular organ annotation results.

[0075] When the initial vascular ground truth (GT) domain annotation has been completed using traditional algorithms such as thresholding methods, clinicians often need to check and edit the annotation results for erroneous or missing vascular branches. If a vascular tree has many vascular branches, it is difficult to check which vessel's annotation result is missing or interrupted.

[0076] Therefore, according to the second embodiment of the present invention, the tubular organ annotation method of the first embodiment is applied to the correction of the tubular organ annotation results. By comparing the LMIP image and the corresponding pre-annotated vascular domain, the user can easily and immediately identify which vascular branch is missing or interrupted, and then use the method of the first embodiment to correct the missing or interrupted part in the LMIP image.

[0077] The second embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0078] Figure 9 This is a flowchart illustrating a method for correcting the labeling results of tubular organs according to a second embodiment of the present invention.

[0079] like Figure 9 As shown, the method for correcting the labeling results of tubular organs according to the present invention includes the following steps.

[0080] In step S100', input the three-dimensional volume data containing the tubular organ and the pre-annotation result after annotating the tubular organ on the three-dimensional volume data.

[0081] In step S200', the two-dimensional image constituting the three-dimensional volume data is subjected to, for example, local maximum density projection to obtain an LMIP projected image. Further, the mapping matrix of this local maximum density projection is obtained.

[0082] In step S300', the LMIP projection image obtained in S200' is compared with the pre-annotation result input in S100' to obtain the missing part of the tubular organ in the pre-annotation result, and the missing part of the tubular organ is annotated in the LMIP projection image.

[0083] In step S400', the center point of the blood vessels of the missing part of the tubular organ marked in the LMIP projection image is back-mapped to the three-dimensional volume data using a mapping matrix.

[0084] In S500', the inverse mapping results are combined with the pre-annotation results to generate a three-dimensional image of a tubular organ.

[0085] The above steps are explained in detail below with reference to the accompanying drawings.

[0086] Details regarding step S200', including performing local maximum density projection and obtaining the mapping matrix of the local maximum density projection, are consistent with the first embodiment and the accompanying appendix. Figure 2 The corresponding steps are the same as those in the instructions, and will not be repeated here.

[0087] Figure 10 This is a schematic diagram illustrating the operations performed in steps S300' and S400'.

[0088] like Figure 10 As shown in (a), by projecting the maximum density image ( Figure 10 Comparing the gray (or dark) parts of the blood vessel pattern in the upper left appendix with the pre-annotated results (the red (or light) parts of the blood vessel pattern), for example, by simply overlaying the two images, it is clear that the maximum density projection image contains more blood vessel information, for example in... Figure 10 In (a), the middle right portion contains a gray blood vessel pattern m1 that is not covered by the red blood vessel pattern. This gray blood vessel pattern represents the missing portion of the tubular organ (blood vessel) in the pre-annotation results. Therefore, as... Figure 10 As shown in (b), in step S300', the missing portion m1 is annotated as described in step S400 of the first embodiment. Next, as... Figure 10 As shown in (c), the mapping matrix is ​​used to perform inverse mapping and fitting, as described in step S500 of the first embodiment, on the marked missing parts.

[0089] The missing part can be a complete missing blood vessel, or it can be a missing section of a blood vessel, such as a missing branch, a missing terminal part, or a missing segment.

[0090] Here, steps S300' to S400' are implemented using the corresponding steps in the tubular organ annotation method described in the first embodiment. Specifically, the annotation in step S300' and the reverse mapping in step S400' are the same as S400 and S500 in the first embodiment. The only difference is that in the first embodiment, S400 and S500 are performed on the entire blood vessel, while in this embodiment, S300' and step S400' are performed only on the missing parts obtained by comparison in step S300'. Therefore, their detailed description is omitted.

[0091] Other steps, such as the comparison in step S300' and the synthesis in step S500', can be implemented using techniques known in the art, and will not be elaborated here.

[0092] In addition to achieving the effects of the first embodiment, the second embodiment can also correct the pre-labeling results of tubular organs such as blood vessels obtained by other methods.

[0093] The above describes the tubular organ annotation result correction method of the second embodiment. Of course, the second embodiment can also be implemented as a system or device corresponding to the above tubular organ annotation result correction method.

[0094] <Other variations>

[0095] The present invention is not limited to the embodiments described above, and can be modified in many ways.

[0096] For example, the above embodiments have been described using blood vessels as an example, but the present invention can also be used for other tubular organs such as the trachea.

[0097] The system of the present invention can also be installed in medical devices as a circuit capable of realizing the functions described in the various embodiments, or it can be distributed as a program that can be executed by a computer, stored on storage media such as disks (floppy disks, hard disks, etc.), optical disks (CD-ROMs, DVDs, BDs, etc.), optical disks (MOs), semiconductor memories, etc.

[0098] Furthermore, middleware such as an OS (operating system), database management software, and network software that runs on a computer based on instructions from a program installed on the computer from a storage medium can also execute a portion of the processes used to implement the above-described embodiments.

[0099] The foregoing has described several embodiments of the present invention, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in a wide variety of other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments or variations thereof are included within the scope or spirit of the invention, and also within the scope of the invention and its equivalents as described in the claims.

Claims

1. A method for labeling tubular organs, characterized in that, Includes the following steps: The projection step involves performing local density projection on a two-dimensional image that constitutes three-dimensional volume data containing tubular organs to obtain a projected image. The mapping matrix acquisition step obtains the mapping matrix of the local density projection implemented in the projection step; The annotation step involves annotating the tubular organ in the projected image; as well as The inverse mapping step involves using the mapping matrix to inversely map the tubular organs marked in the projected image onto the three-dimensional volume data, thereby generating a three-dimensional image of the tubular organs. The mapping matrix represents the correspondence between each pixel in the projected image and which of the multiple two-dimensional images the pixel corresponds to. In the reverse mapping step, according to the correspondence, the two-dimensional image corresponding to each pixel is determined according to the pixel position of each pixel of the tubular organ marked in the projected image. The three-dimensional image of the tubular organ is generated by using the pixel position and the image number information indicating which image among the plurality of two-dimensional images the determined two-dimensional image is.

2. The method for labeling tubular organs as described in claim 1, characterized in that, The annotation step involves marking key points on the center line of the tubular organ; The inverse mapping step generates key points on the central line of the three-dimensional tubular organ.

3. The method for labeling tubular organs as described in claim 2, characterized in that, The inverse mapping step also fits the three-dimensional tubular organ centerline based on key points on the three-dimensional tubular organ centerline. This method for labeling tubular organs also includes: The tubular organ annotation step involves annotating the three-dimensional tubular organ image based on the fitted three-dimensional tubular organ centerline.

4. The method for labeling tubular organs as described in any one of claims 1 to 3, characterized in that, In the projection step, local density projection is performed on multiple sets of the two-dimensional images to obtain multiple projected images. In the mapping matrix acquisition step, the mapping matrix for each of the multiple sets of two-dimensional images is acquired; In the annotation step, the tubular organ is annotated in the plurality of projected images respectively; In the reverse mapping step, the tubular organ is reverse mapped to the three-dimensional volume data using the mapping matrix of each group, and a three-dimensional image of the tubular organ is synthesized.

5. The method for labeling tubular organs as described in any one of claims 1 to 3, characterized in that, In the projection step, local density projection is performed on the two-dimensional images constituting the three-dimensional volume data in multiple directions to obtain the projected images in the multiple directions. In the mapping matrix acquisition step, the mapping matrix for each of the plurality of directions is acquired; In the annotation step, the tubular organs are annotated in the projected images in the multiple directions respectively; In the inverse mapping step, the mapping matrix in each direction is used to inverse map the tubular organs in the multiple directions to the three-dimensional volume data, and a three-dimensional image of the tubular organs is synthesized.

6. The method for labeling tubular organs as described in any one of claims 1 to 3, characterized in that, The number of the plurality of two-dimensional images used to generate the projected image is variable.

7. The method for labeling tubular organs as described in any one of claims 1 to 3, characterized in that, The tubular organ is a blood vessel.

8. A method for correcting annotation results of tubular organs, characterized in that, Includes the following steps: The input steps involve inputting three-dimensional volume data containing tubular organs and pre-annotation results after annotating the tubular organs on the three-dimensional volume data; The projection step involves performing local density projection on the two-dimensional image constituting the three-dimensional volume data to obtain a projected image. The mapping matrix acquisition step obtains the mapping matrix of the local density projection implemented in the projection step; The comparison step involves comparing the projected image with the pre-annotation result to obtain the missing portion of the tubular organ in the pre-annotation result; The annotation step involves annotating the missing portion of the tubular organ in the projected image; as well as The inverse mapping step involves using the mapping matrix to inversely map the missing portion of the tubular organ marked in the projected image to the three-dimensional volume data, and then combining it with the pre-annotation result to generate a three-dimensional image of the tubular organ. The mapping matrix represents the correspondence between each pixel in the projected image and which of the multiple two-dimensional images the pixel corresponds to. In the reverse mapping step, according to the correspondence, the two-dimensional image corresponding to the pixel is determined according to the pixel position of each pixel of the missing part of the tubular organ marked in the projected image. The three-dimensional image of the tubular organ is generated by using the pixel position and the image number information indicating which image among the plurality of two-dimensional images the determined two-dimensional image is.

9. A tubular organ labeling system, characterized in that, have: A projection device performs local density projection on a two-dimensional image that constitutes three-dimensional volume data containing tubular organs to obtain a projected image; A mapping matrix acquisition device acquires the mapping matrix of the local density projection implemented by the projection device; A labeling device for labeling the tubular organ in the projected image; as well as The inverse mapping device uses the mapping matrix to inverse map the tubular organs marked in the projected image onto the three-dimensional volume data, generating a three-dimensional image of the tubular organs. The mapping matrix represents the correspondence between each pixel in the projected image and which of the multiple two-dimensional images the pixel corresponds to. The anti-mapping device determines the two-dimensional image corresponding to each pixel according to the pixel position of each pixel of the tubular organ marked in the projected image, based on the correspondence relationship. It then generates a three-dimensional image of the tubular organ by using the pixel position and the image number information indicating which image among the plurality of two-dimensional images the determined two-dimensional image is.

Images