Tree-structure anatomy method using 3D implicit local-global segmentation model through point-wise map, and system therefor

Abstract

A tree-structure anatomy method using a 3D implicit local-global segmentation model through a point-wise map, and a system therefor are provided. The method comprises the steps of: cropping CT image data into segmented image data; extracting a local feature from the segmented image data, and extracting a final local-global feature by using the local feature and a global location; and detecting a tree-structure shape of the segmented image data by using the final local-global feature.

Description

Tree structure anatomy method and system using a 3D implicit local-global partitioning model via point-unit map

[0001] The present disclosure relates to a method and system for tree-structure anatomy using a 3D implicit local-global segmentation model through a point-unit map, and more specifically, to a technique for segmenting the tree-structure lung anatomy in CT images using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map.

[0002] Medical segmentation models were developed to segment cluster-like structures, such as multi-organ segmentation, and generally focus on patch-based architectures. When segmenting tree-structure anatomy, these general segmentation models have difficulty capturing the fine details of distal parts and the connections of branches.

[0003] Segmentation is traditionally considered as pixel-level (or 3D voxel-level) classification of a given input image. Based on this concept, there are various types of deep neural networks with highly advanced performance. For example, existing techniques, such as powerful transformer-based models, have been continuously developed to segment multi-institutional CT data, such as BTCV [Gibson et al., 2018]. These models are optimized for segmenting objects into shapes such as sufficient volume and characterized clusters.

[0004] Conversely, segmenting tree-structure anatomy is difficult due to morphological characteristics. Such anatomy exhibits varying thicknesses, consisting of a thick proximal section (main trunk) and much thinner distal branches. Consequently, models struggle when segmenting the distal branches. This is because most segmentation models, such as U-Net [Kerfoot et al., 2018] and Transformer-based variants, adopt a strategy of segmenting CT images by volume for training and inference. Models lack a unique encoding module for connections, and they easily achieve training objectives by simply segmenting the dominant volume representing the proximal part and the background. Furthermore, details are lost due to iterative downsampling and upsampling algorithms.

[0005] To address these problems, the present disclosure proposes a 3D implicit local-global segmentation (3D-ILGP) model using a point-based map for segmenting tree-structured lung anatomy in CT images.

[0006] The embodiments disclosed in this disclosure aim to propose a 3D Implicit Local-Global Segmentation (3D-ILGP) model through a point-based map, which is an implicit function-based end-to-end segmentation model for generalized tree structure segmentation for medical images.

[0007] The purpose of the embodiments disclosed in this disclosure is to propose a technique that extracts local features from segmented images while maintaining the high resolution of the original CT image using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map, and detects the tree structure anatomical shape of the segmented images by utilizing global locations.

[0008] The embodiments disclosed in this disclosure aim to propose a point encoding algorithm for capturing fine details and local connections, and for integrating global information to capture shapes.

[0009] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0010] A method for tree structure anatomy using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map performed by a computer according to one aspect of the present disclosure for achieving the technical problem described above comprises the steps of: cropping CT image data into segmented image data; extracting local features from the segmented image data and extracting a final local-global feature by utilizing the local features and a global location; and detecting the tree structure form of the segmented image data using the final local-global feature.

[0011] A computer program stored on a computer-readable storage medium according to another aspect of the present disclosure for achieving the technical problem described above, wherein, when the computer program is executed on one or more processors, the computer program performs the following operations to perform a tree structure anatomy method using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map, the operations include cropping CT image data into segmented image data, extracting local features from the segmented image data, extracting a final local-global feature using the local features and a global location, and detecting the tree structure form of the segmented image data using the final local-global feature.

[0012] A tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point unit map according to another aspect of the present disclosure for achieving the technical problem described above comprises: a segmentation unit that crops CT image data into segmented image data; a feature extraction unit that extracts local features from the segmented image data and extracts a final local-global feature by utilizing the local features and a global location; and a shape detection unit that detects the tree structure shape of the segmented image data using the final local-global feature.

[0013] In addition, a computer-readable recording medium for recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0014] According to the aforementioned means for solving the problem of the present disclosure, local features are extracted from segmented images while maintaining the high resolution of the original CT image using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map, and the tree structure form of the segmented image is detected by utilizing global locations, thereby enabling application to the study and technology of tree-structured body anatomy.

[0015] According to the aforementioned means for solving the problem of the present disclosure, by detecting the tree structure shape using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map, it exhibits excellent performance in segmenting multiple tree structures in which components are complexly intertwined, and can also operate powerfully in vascular segmentation where the thickness of arteries and veins is highly uneven.

[0016] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0017] FIG. 1 illustrates a flowchart of the operation of a tree structure anatomy method using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure.

[0018] FIG. 2 illustrates an anatomical structure having three tree structures of the lung according to an embodiment of the present disclosure.

[0019] Figure 3 is a table showing the results of demonstrating a 3D implicit local-global segmentation (3D-ILGP) model through point-unit maps according to an embodiment of the present disclosure using an ATM dataset for bronchial segmentation and a PARSE dataset for pulmonary artery segmentation.

[0020] Figure 4 is a table showing the results of a multi-class segmentation experiment performed using an internal dataset for bronchus, artery, and vein segmentation.

[0021] FIG. 5 is a block diagram illustrating the detailed configuration of a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure.

[0022] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general content in the art to which this disclosure pertains or content that overlaps between embodiments is omitted. The terms “part, module, component, block” as used in the specification may be implemented in software or hardware, and depending on the embodiments, a plurality of “parts, modules, components, blocks” may be implemented as a single component, or a single “part, module, component, block” may include a plurality of components. Throughout the specification, when a part is described as being “connected” to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0023] Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0024] Throughout the specification, when it is stated that a component is located "on" another component, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0025] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0026] Singular expressions include plural expressions unless there is an obvious exception in the context.

[0027] In each step, identification codes are used for convenience of explanation and do not describe the order of the steps; the steps may be performed differently from the specified order unless a specific order is clearly indicated in the context.

[0028] The operating principles and embodiments of the present disclosure will be described below with reference to the attached drawings.

[0029] Prior to the explanation, the meanings of the terms used in this specification are briefly explained. However, since the explanation of terms is intended to aid in understanding this specification, it should be noted that they are not used to limit the technical scope of this disclosure unless explicitly stated to be a limiting factor.

[0030] In this specification, the term "device according to the present disclosure" includes all various devices capable of performing computational processing and providing results to a user. For example, the device may include a computer, a server device, and a portable terminal, or be any one of these forms.

[0031] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0032] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0033] The above portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0034] The present disclosure is to detect the tree-structured anatomical shape of lungs using a 3D implicit local-global segmentation (3D-ILGP) model through point-unit maps.

[0035] In the present disclosure, the 3D-ILGP model utilizes global location to detect the shape of the segmented image while maintaining the high resolution of the original CT image, while extracting local features from segmented images cropped from the original CT image. This is based on an implicit model that utilizes a relative grid for the input of interest. While learning continuous representations, the volume-based model learns discretized representations at the voxel level. Point-by-point mapping according to the grid enables the model to capture fine details, and the function of learning continuity encodes connectivity for the model by considering neighbor features and the contours of the proximal part.

[0036] Furthermore, through a flexible point encoding and decoding structure, the model can encode both local and global information. Global information plays a crucial role in medical imaging because the images are oriented and the segmentation targets are located at fixed positions. When cropping bronchi in chest CT, the distribution of branch thickness correlates with location. Branches located in the center and upper part of a given CT image tend to be thicker than those located in the lower periphery. Therefore, global locations that correspond to the cropped area can serve as important hints for the model. Since the implicit model extracts features corresponding to the points of interest and then decodes them one-to-one with the segmentation output, global locations are easily encoded and match the cropped volume.

[0037] The 3D-ILGP according to the embodiments of the present disclosure can efficiently combine local and global information without additional modules or complex training methods for thickness and balanced learning. The 3D-ILGP has been shown to demonstrate superior performance compared to other volume-based baselines in segmenting various types of tree structures in the lungs. The 3D-ILGP performs marginal improvements in voxel-based measurements, dice scores as well as structural measurements, 95% Hausdorff distance (HD95) and tree detection ratios, bronchial segmentation using the ATM dataset, arterial segmentation using the PARSE dataset, and bronchial, artery, and vein segmentation using the internal dataset. In particular, in bronchial, artery, and vein multi-class segmentation models, the 3D-ILGP can achieve better performance with minimal degradation. This indicates that the method of the present disclosure demonstrates excellent performance in segmenting multiple tree structures where components are complexly intertwined. Furthermore, the 3D-ILGP model can operate robustly even in highly unbalanced vascular segmentation where the main pulmonary artery and vein are much thicker than other organs.

[0038] In addition, 3D-ILGP is expected to serve as an additional morphological encoder for segmenting lung segmentation models when integrated into ImPulse [Kaiming Kuang, Li Zhang, Jingyuan Li, Hongwei Li, Jiajun Chen, Bo Du, and Jiancheng Yang. What makes for automatic reconstruction of pulmonary segments. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2022.], which is also an implicit function-based segmentation model for lung segmentation. This is because, as highlighted in ImPulse, such tree structures serve as important criteria for hierarchically segmenting the lungs into lung lobes and lung segments.

[0039] Research related to the present disclosure includes implicit functions and 3D segmentation for medical images.

[0040] Recently, research has been published on applying implicit neural representations to various medical imaging tasks. Previous studies have focused on implicitly learning spline representations of segmentation boundaries instead of explicit image-level prediction, and have used implicit neural representations for image upscaling to enhance fine details for segmentation in retinal images. While this focuses on 2D medical images, the present disclosure aims to focus more on 3D medical image segmentation using implicit neural representations.

[0041] Following the introduction of U-Net, which achieved great success in 2D image segmentation, various U-Net-based fully convolutional models were utilized for 3D medical image segmentation. 3D U-Net extends conventional convolution to 3D convolution to enable the capture of volumetric information. Subsequently, SegResNet integrated input reconstruction assistance tasks into variable autoencoder branches for decoder normalization. Both nn-U-Net and Unet++ demonstrated the superiority of U-Net-based architectures by showing significant performance improvements while making partial modifications to the 3D U-Net model architecture. After ViT demonstrated the effectiveness of Transformers in computer vision, Transformer-based methods were applied to various downstream tasks, including object detection and semantic segmentation. Consequently, existing technologies applied Transformer structures to 2D medical image segmentation by directly combining them with standard U-Net.

[0042] UNETR, which applies ViT patch embeddings to 3D inputs, achieves state-of-the-art performance in 3D medical segmentation. Subsequently, SwinUNETR applies the shifted window approach of SwinTransformer, and UNETR++ proposes a computationally more efficient transformer structure by utilizing channel attention blocks.

[0043] The present disclosure proposes a 3D implicit local-global segmentation (3D-ILGP) model through point-based mapping to provide a tree-structured anatomical shape while maintaining the high resolution of 3D CT images.

[0044] The present disclosure will be described in detail below with reference to FIGS. 1 to 5.

[0045] FIG. 1 illustrates a flowchart of the operation of a tree structure anatomy method using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure. FIG. 2 also illustrates an anatomical structure having three tree structures of the lung according to an embodiment of the present disclosure.

[0046] The tree structure anatomy method of FIG. 1 is performed by a tree structure anatomy system (500) using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure illustrated in FIG. 5.

[0047] Referring to FIG. 1, in step S110, a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure divides CT image data into segmented image data.

[0048] In step S110, a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure can obtain segmented image data cropped from CT image data of a 3D original image into volume units of a preset interval and size. At this time, the volume representing the proximal section and the background is cropped, and segmented image data including the proximal section can be cropped.

[0049] 3D Implicit Local-Global Segmentation (3D-ILGP) model f through a point-unit map, which is the ideal segmentation model proposed in this disclosure ideal It classifies a point p of 3D CT image data X into class y. Accordingly, y = f ideal (X, p) becomes. This represents the goal or ideal state of the 3D-ILGP model.

[0050] In step S110, a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure can obtain segmented image data X' by randomly extracting a volume of a specific size from CT image data X of a 3D original image.

[0051] In step S120, a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure extracts local features from segmented image data and extracts final local-global features by utilizing local features and global locations.

[0052] In step S120, a tree-structured anatomical system using a 3D implicit local-global segmentation (3D-ILGP) model with a point-unit map according to an embodiment of the present disclosure classifies a relative position (p') in segmented image data X'. At this time, the relative position p' in segmented image data X' and the global position (p) in CT image data X can represent the same physical location. In the present disclosure, both the relative coordinate p' and the global coordinate p are normalized to the range [-1, 1] in each dimension, and the center position corresponds to (0, 0) in both the CT image data of the 3D original image and the segmented image data.

[0053] A 3D implicit local-global segmentation (3D-ILGP) model using a point-unit map according to an embodiment of the present disclosure can classify y = f(X', p') using only segmented image data X' and relative position p'. In this case, the 3D implicit local-global segmentation (3D-ILGP) model using a point-unit map according to an embodiment of the present disclosure uses an encoder f E , point encoder fP , decoder f D It consists of three main components. Encoder f E It extracts features from image and video data, and the point encoder f P encodes location information, and decoder f D It decodes the combined features and point encoding to output the final classification.

[0054] The aforementioned components are applied sequentially as shown in [Equation 1] below.

[0055]

[0056]

[0057] Using this formula, the 3D Implicit Local-Global Segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure uses an encoder f E It makes full use of the feature extraction capability, and decoder f uses the extracted features. D A point encoder f that enables easier decoding in P Focus on maximizing the usefulness of.

[0058] In step S120, a tree-structured anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure extracts local features Z' from segmented image data X', and can extract a feature vector z' using the feature matrix Z' of the local features and the relative position p'. More specifically, local feature Z' = f E (X') is a feature extracted from the segmented image data, which corresponds to the intermediate output of the residual block in the model using the ResNet-3D backbone. Accordingly, the feature vector z' = f from the feature matrix Z' and relative coordinates p' P (Z', p') can be extracted.

[0059] To extract a feature vector z' from a feature matrix Z', a tree-structured anatomy system using a 3D Implicit Local-Global Segmentation (3D-ILGP) model with a point-unit map according to an embodiment of the present disclosure may use a channel rescaling block and a relative coordinate encoding. However, unlike the existing IFA (Implicit Feature Alignment function) point encoder (Prior research - Hu, Hanzhe et al. “Learning Implicit Feature Alignment Function for Semantic Segmentation.” European Conference on Computer Vision (2022).), the present disclosure is characterized by extracting the feature vector z' using bilinear interpolation after channel rescaling. Using the bilinear interpolation allows the model to consider neighbor features located in the neighborhood of a relative position p' as shown in [Equation 2] below.

[0060]

[0061]

[0062] At this time, a tree-structured anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure can extract sampled neighborhood features more accurately by using the position encoding algorithm of IFA, which encodes the sampled relative position p' + δp' and the difference between relative positions δp' as sinusoidal embeddings. The aforementioned method naturally allows for the consideration of neighborhood information. This is advantageous over other CNN-based architectures because it requires an additional explicit module to capture neighborhood information.

[0063] In addition to sophisticated feature extraction, a tree-structured anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model with a point-unit map according to an embodiment of the present disclosure proposes utilizing additional information when decoding features. Although segmented image data X' preserves local details well, there is an inherent limitation in that information related to the global position (absolute position) p or shape within the CT image data X of the 3D original image is lost. To overcome this limitation, the present disclosure also utilizes the global position (absolute position) p as an encoding strategy.

[0064] Because the model learns point-based representations rather than patch-based representations, the mechanism for encoding global position (absolute position) p is simpler in the implicit function backbone than in other CNN or Transformer-based architectures. Furthermore, while the compared architectures have complex entanglement structures that concatenate hierarchically encoded features and then iteratively upscale them, the implicit function simply extracts features and uses a light decoder consisting of a few linear layers with activation functions such as ReLU. Encoding global information is particularly advantageous in medical imaging because most images have a fixed shape, whereas other segmentation operations help focus on regions of interest while ignoring other information.

[0065] Through the absolute position p, the 3D-ILGP model can determine the difficulty of segmentation at the relative position p' within the segmented image data X'. For example, if the global position of a given image is at the top center for bronchial segmentation, it is easy to segment the image because the major bronchus is located there. Additionally, it can eliminate false positives that may occur in areas outside the lungs without the need for a post-processing step of extracting the largest connected component to artificially remove artifacts, which is common in most crop-and-cat models.

[0066] Finally, in step S120, the tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure can extract the final local-global feature ζ as shown in [Equation 3] below.

[0067]

[0068]

[0069] In this case, X can be scaled to create the size of the feature.

[0070] In step S130, a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure detects the tree structure form of the segmented image data using the final local-global features.

[0071] A tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure inputs final local-global features into a decoder to calculate a final probability and can detect the tree structure of the lung shape in the segmented image data.

[0072] After extracting features using [Equation 3], the present disclosure can obtain a final probability y by using the final local-global feature ζ as input to a decoder, which is as in [Equation 4].

[0073]

[0074]

[0075] Accordingly, a tree-structured anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure classifies volume-structured (210) and tree-structured (220) in segmented image data (200) as shown in FIG. 2, and can detect anatomical structures having three tree structures in the tree structure (220): bronchus (221), pulmonary artery (222), and pulmonary vein (223).

[0076] FIG. 3 is a table showing the results of demonstrating a 3D implicit local-global segmentation (3D-ILGP) model using point-unit maps according to an embodiment of the present disclosure using the ATM dataset for bronchial segmentation and the PARSE dataset for pulmonary artery segmentation. FIG. 4 is a table showing the results of performing a multi-class segmentation experiment using internal datasets for bronchus, pulmonary artery, and pulmonary vein segmentation.

[0077] The ATM dataset is a benchmark for lung airway segmentation in large-scale CT scans. It consists of a total of 500 CT images collected from subjects of various health conditions. Each CT image consists of 512×512 pixel slices, with the number of slices varying from 157 to 1125 and the thickness ranging from 0.450 mm to 1.000 mm. In this disclosure, 300 scans from the publicly available training dataset were randomly divided, with 255 used for training and 44 for validation. One sample with abnormal spacing of a Z-axis interval of 5 was excluded from training and validation.

[0078] The PARSE dataset is a public dataset for pulmonary artery segmentation that provides 203 CT scans collected from subjects diagnosed with pulmonary nodular disease. Each CT scan consists of slices with a resolution of 512×512 pixels, with the number of slices ranging from 228 to 408 and the thickness ranging from 0.5039 mm to 0.9238 mm. The training dataset and test set are divided in a ratio of 0.85 to 0.15, respectively, and in this disclosure, 100 scans from the public training dataset were randomly divided to use 85 for training and 15 for validation.

[0079] The in-house dataset consists of 118 CT scans annotated with bronchi, pulmonary arteries, pulmonary veins, and five lung lobes. This dataset was used to train an ex-house model that crops anatomical structures with tree structures and crops foreground volumes, and to crop the foreground before training and validation. Each CT image consists of slices with a resolution of 512×512 pixels, with slice thicknesses ranging from 0.99 mm to 1.0 mm and pixel spacing between 0.64 mm and 0.88 mm. In this disclosure, the entire dataset was used with 100 scans for training and 18 scans for validation.

[0080] In this disclosure, two evaluation metrics were used. First, the Dice Similarity Coefficient (DSC) was used to evaluate the performance of binary semantic partitioning. The Dice Similarity Coefficient evaluates the degree of overlap between predicted values ​​and annotations in voxel units and is defined as [Equation 5] below.

[0081]

[0082]

[0083] Here, Ntp , N fp , N fn Each represents the number of true positive, false positive, and false negative voxels.

[0084] Considering the characteristics of the anatomical structure of the lung, the present disclosure used topological measurements such as Tree Detected (TD) and the 95% Hausdorff distance (HD95). The total length of the true centerline is L ref , the length of the centerline detected in the model prediction is L seg When that is the case, tree detection (TD) is defined as [Equation 6] below, reflecting the sensitivity of tree length prediction.

[0085]

[0086]

[0087] HD95 is used to measure the spatial distance between the predicted surface and the actual annotated surface and is defined as [Equation 7] below.

[0088]

[0089]

[0090] Here, h(A, B) is max a∈A min b∈B ||a=b|| 95% Unlike TD, HD95 is particularly effective at capturing outliers that occur in predictions and ensures that segmented lung anatomical structures match the annotated locations exactly.

[0091] Accordingly, the present disclosure compared a 3D implicit local-global segmentation (3D-ILGP) model with point-unit maps according to an embodiment of the present disclosure on two public datasets (ATM and PARSE) and an internal dataset with four reference models (U-net, SegResnet, UNETR, and Swin-UNETR).

[0092] Referring to FIG. 3, it can be seen that the 3D-ILGP method according to an embodiment of the present disclosure exhibits superior performance in structural indicators, such as the 95% Hausdorff distance (HD95) and the tree detection rate (TD), compared to other methods. This means that the 3D-ILGP model according to an embodiment of the present disclosure captures the boundaries of the segmentation target well, handles local details well, and ensures that connectivity is not broken.

[0093] Referring to FIG. 4, it can be seen that the 3D-ILGP method according to the embodiment of the present disclosure consistently demonstrates superior performance compared to other methods in bronchial, artery, and vein 4-class segmentation tasks using an internal dataset.

[0094] It is shown that the 3D-ILGP model according to the embodiments of the present disclosure captures fine details better than existing volume-based models. The 3D-ILGP model according to the embodiments of the present disclosure can effectively utilize encoded global locations for segmentation.

[0095] FIG. 5 is a block diagram illustrating the detailed configuration of a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure.

[0096] Referring to FIG. 5, a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure segments the tree structure of lung anatomy in a CT image using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map.

[0097] To this end, a tree structure anatomy system (500) using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map according to an embodiment of the present disclosure (the tree structure segmentation system includes a crop module (510), a feature extraction unit (520), and a shape detection unit (530).

[0098] The crop module (510) crops the CT image data into cropped image data. That is, the cropped image data may be cropped images of the CT image data.

[0099] The segmentation unit (510) can obtain segmented image data cropped from CT image data of a 3D original image into volume units of a preset interval and size. At this time, the volume representing the proximal section and the background is cropped, and the segmented image data including the proximal section can be segmented.

[0100] The feature extraction unit (520) extracts local features from segmented image data and extracts final local-global features by utilizing local features and global locations.

[0101] The feature extraction unit (520) classifies the relative position (p') in the segmented image data X'. At this time, the relative position p' in the segmented image data X' and the global position (p) in the CT image data X can represent the same physical location. In the present disclosure, both the relative coordinate p' and the global coordinate p are normalized to the range [-1, 1] in each dimension, and the center position corresponds to (0, 0) in both the CT image data of the 3D original image and the segmented image data.

[0102] A 3D implicit local-global segmentation (3D-ILGP) model using a point-unit map according to an embodiment of the present disclosure can classify y = f(X', p') using only segmented image data X' and relative position p'. In this case, the 3D implicit local-global segmentation (3D-ILGP) model using a point-unit map according to an embodiment of the present disclosure uses an encoder f E , point encoder f P , decoder f D It consists of three main components. Encoder f E It extracts features from image and video data, and the point encoder f P encodes location information, and decoder f D It decodes the combined features and point encoding to output the final classification.

[0103] A 3D implicit local-global segmentation (3D-ILGP) model using a point-unit map according to an embodiment of the present disclosure uses an encoder f E It makes full use of the feature extraction capability, and decoder f uses the extracted features. D A point encoder f that enables easier decoding in P Focus on maximizing the usefulness of.

[0104] The feature extraction unit (520) extracts local features Z' from segmented image data X', and can extract a feature vector z' using the feature matrix Z' of the local features and the relative position p'. To explain in more detail, local feature Z' = f E (X') is a feature extracted from the segmented image data, which corresponds to the intermediate output of the residual block in the model using the ResNet-3D backbone. Accordingly, the feature vector z' = f from the feature matrix Z' and relative coordinates p' P (Z', p') can be extracted.

[0105] To extract a feature vector z' from a feature matrix Z', the feature extraction unit (520) may use a channel rescaling block and a relative coordinate encoding. However, unlike conventional IFA point encoders, the present disclosure is characterized by extracting the feature vector z' using bilinear interpolation after channel rescaling. Using the bilinear interpolation allows the model to consider neighbor features located in the neighborhood of a relative position p'.

[0106] At this time, the feature extraction unit (520) can extract sampled neighbor features more accurately by using the position encoding algorithm of IFA, which encodes the sampled relative position p' + δp' and the difference between relative positions δp' as a sinusoidal embedding. The aforementioned method allows for the consideration of neighbor information naturally. This is advantageous over other CNN-based architectures because it requires an additional explicit module to capture neighbor information.

[0107] In addition to sophisticated feature extraction, the feature extraction unit (520) suggests utilizing additional information when decoding features. Although the segmented image data X' preserves local details well, there is an inherent limitation in that information related to the relative position p or shape within the CT image data X of the 3D original image is lost. To overcome this limitation, the present disclosure also utilizes the relative position p as an encoding strategy.

[0108] Because the model learns point-based representations rather than patch-based representations, the mechanism for encoding relative positions p is simpler in the implicit function backbone than in other CNN or Transformer-based architectures. Furthermore, while the compared architectures have complex entanglement structures that concatenate hierarchically encoded features and then iteratively upscale them, the implicit function simply extracts features and uses a light decoder consisting of a few linear layers with activation functions such as ReLU. Encoding global information is particularly advantageous in medical imaging because most images have a fixed shape, whereas other segmentation operations help focus on regions of interest while ignoring information.

[0109] By encoding the global location, the 3D-ILGP model according to the embodiment of the present disclosure implies the difficulty of the relative location p' in the segmented image data X'. For example, if the global location of a given image is the top center for bronchial segmentation, the image is easy to segment because the major bronchus is located there. In addition, false positives that may occur in the outer lung region can be eliminated without the post-processing step of extracting the largest connected component to artificially remove artifacts in most crop-and-cat models.

[0110] Finally, the feature extraction unit (520) can extract the final local-global features.

[0111] The shape detection unit (530) detects the tree structure shape of the segmented image data using the final local-global features. The shape detection unit (530) inputs the final local-global features into a decoder to calculate the final probability and can detect the closed shape of the tree structure in the segmented image data.

[0112] The system or device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and component described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a Field Programmable Gate Array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0113] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0114] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include hard disks, magneto-optical media, solid-state drives (SSDs), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0115] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0116] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A method for tree structure anatomy using a 3D implicit local-global segmentation (3D-ILGP) model through a point-unit map performed by a computer, A step of cropping CT image data into segmented image data; A step of extracting local features from the segmented image data and extracting a final local-global feature by utilizing the local features and global location; and A method comprising the step of detecting the tree structure form of the segmented image data using the above-mentioned final local-global features, Tree structure anatomy method.

2. In Paragraph 1, The above cropping step is, Acquiring the segmented image data cropped from the above CT image data, which is a 3D original image, into volume units of preset intervals and sizes, Tree structure anatomy method.

3. In Paragraph 2, The above cropping step is, A segmented image data including said proximal portion is obtained by cropping a volume representing the proximal portion and the background. Tree structure anatomy method.

4. In Paragraph 1, The above extraction step is, Classifying relative positions in the above segmented image data, The relative position in the segmented image data and the global position in the CT image data represent the same physical point. Tree structure anatomy method.

5. In Paragraph 4, The above extraction step is, Extracting local features from the segmented image data, and extracting a feature vector using the feature matrix of the local features and the relative position, Tree structure anatomy method.

6. In Paragraph 5, The above extraction step is, Using a channel rescaling block and relative coordinate encoding to extract the feature vector from the above feature matrix, Tree structure anatomy method.

7. In Paragraph 6, The above extraction step is, After channel rescaling, considering neighbor features in the neighborhood of the relative position using bilinear interpolation, Tree structure anatomy method.

8. In Paragraph 7, The above extraction step is, Extracting the neighborhood feature containing neighborhood information using a position encoding algorithm that encodes the difference between the sampled relative position and the target position as a sinusoidal embedding, Tree structure anatomy method.

9. In Paragraph 7, The above extraction step is, Extracting the final local-global feature from the encoded global location encoded from the above global location and the above feature vector, Tree structure anatomy method.

10. In Paragraph 1, The above-mentioned detection step is, The above final local-global features are input into a decoder to calculate the final probability, and the tree-structured lung shape is detected in the above segmented image data. Tree structure anatomy method.

11. A computer program stored on a computer-readable storage medium, wherein, when executed on one or more processors, the computer program performs the following operations to perform a tree structure anatomy method using a 3D implicit local-global partitioning (3D-ILGP) model through a point-unit map, said operations being: The operation of cropping CT image data into segmented image data; The operation of extracting local features from the segmented image data and extracting a final local-global feature by utilizing the local features and global location; and A method comprising detecting the tree structure form of the segmented image data using the above-mentioned final local-global features, A computer program stored on a computer-readable storage medium.

12. In a tree structure anatomy system using a 3D implicit local-global segmentation (3D-ILGP) model through point-unit maps, A segmentation unit that crops CT image data into segmented image data; A feature extraction unit that extracts local features from the segmented image data and extracts a final local-global feature by utilizing the local features and a global location; and A shape detection unit comprising a tree structure shape of the segmented image data using the above final local-global features, Tree structure anatomical system.

Images