CBCT (Cone Beam Computed Tomography) projection metal trajectory segmentation and artifact removal method of sparse labeling fine-tuning SAM (Sparse Assisted
By combining sparse annotation and multi-contrast projection frames with image domain cues, the SAM model was fine-tuned, solving the problem of metal trajectory segmentation in the projection domain of CBCT. This enabled efficient removal of metal artifacts and image reconstruction, thus improving the quality of CBCT images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-03-31
AI Technical Summary
Existing technologies struggle to efficiently segment metal trajectories in the CBCT projection domain, resulting in metal artifacts that severely impact image quality. Furthermore, deep learning models require a large amount of high-quality labeled data, making them difficult to promote and apply in clinical settings.
A dataset is constructed using sparse annotation. The SAM model is fine-tuned by combining multi-contrast projection frames and image domain threshold Prompt. High-precision segmentation of metal trajectories is achieved through complementary information between the projection domain and the image domain. Morphological processing and reconstruction are then performed to remove artifacts.
While reducing data annotation costs, it achieves high-precision metal trajectory segmentation and artifact removal, improves CBCT image quality, adapts to metal implants of different shapes and locations, and effectively removes metal artifacts and truncation artifacts.
Smart Images
Figure CN121767511A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for CBCT projection domain metal trajectory segmentation and metal artifact removal using sparse annotation based on threshold Prompt and SAM fine-tuning, belonging to the field of computer image processing. Background Technology
[0002] Cone-beam computed tomography (CBCT) is currently one of the most important imaging techniques for obtaining local three-dimensional images of the human body, and it is widely used in oral surgery, image-guided radiotherapy, and interventional procedures. However, in clinical applications, some patients have metal implants, such as dental fillings, orthopedic internal fixation screws, and hip replacement devices. Because metals have a high linear attenuation coefficient, they cause beam hardening, photon starvation, and scattering, resulting in severe metal artifacts (e.g., radial streaks) in the reconstructed CBCT images. These artifacts severely impair image quality, making it impossible to provide doctors with accurate image guidance, thus affecting subsequent treatment.
[0003] To address this, several Metal Artifact Reduction (MAR) algorithms have been proposed. Currently, mainstream MAR algorithms are mainly divided into two categories: image domain-based and projection domain-based. Image domain-based methods mostly segment the metal region using a threshold and then fill it with a substitution value. However, this method has some limitations: because the CT value (HU value) of metal is very close to the CT value of high-density bone tissue, and is affected by radial artifacts, it is difficult to separate the metal using only a single threshold; furthermore, metal implants outside the field of view (FOV) are not visible in the reconstructed image, therefore, they cannot be segmented using image domain methods, but metal outside the FOV still causes severe radial artifacts.
[0004] In contrast, projection domain-based methods hold greater potential. The main idea is to identify and segment metal trajectories in projection data, then treat these trajectories as missing data regions, filling them in using interpolation or more complex methods to finally reconstruct an artifact-free image. However, the biggest challenge with these methods lies in the accurate segmentation of metal trajectories. Traditional projection domain segmentation relies on simple thresholding or edge detection, but for some low-contrast or noisy projection images, it's difficult to distinguish metal trajectories from bone trajectories. In recent years, deep learning-based segmentation methods (such as U-Net) have demonstrated superior performance, but these supervised learning algorithms typically require large amounts of paired, pixel-level precisely labeled datasets for training. In clinical practice, manually labeling metal trajectories in the projection domain is not only time-consuming and laborious but also requires specialized knowledge from the labelers, making it difficult to obtain high-quality, large-scale labeled data and limiting the widespread clinical application of deep learning models.
[0005] In recent years, large-scale visual models, represented by the Segment Anything Model (SAM), have demonstrated excellent zero-shot and few-shot capabilities in natural image segmentation tasks. However, directly applying pre-trained SAMs to medical CBCT projection data yields less than ideal results. This is because the texture features and grayscale distribution of projected images differ significantly from those of natural images. Furthermore, the complex shapes of metal trajectories, coupled with the lack of prompts, prevent SAMs from focusing on the target region and accurately segmenting metal objects. Therefore, effectively fine-tuning the SAM model using low-cost sparse labeled data, combined with prior knowledge of medical imaging (such as coarse segmentation in the image domain), to adapt it to metal segmentation tasks in the CBCT projection domain and achieve high-quality metal artifact removal is a critical challenge that urgently needs to be addressed in the field of medical image processing. Summary of the Invention
[0006] To address the challenge of segmenting metallic trajectories in the CBCT projection domain, this invention proposes a CBCT projection metallic trajectory segmentation and artifact removal method based on sparsely labeled fine-tuned SAM. The procedure is as follows: Figure 1 As shown, by combining projection frames with multiple contrasts, threshold Prompt, and SAM model fine-tuning techniques, high-precision metal trajectory segmentation was achieved with a small amount of data annotation. Furthermore, the projection data repair and reconstruction strategies effectively reduced metal artifacts and truncation artifacts in CBCT images.
[0007] To achieve the above objectives, the technical solution of the present invention is as follows: The present invention provides a method for CBCT projection metal trajectory segmentation and artifact removal using sparsely labeled fine-tuned SAM, the specific steps of which are as follows:
[0008] Step 1: Collect CBCT data from real-world scenarios and construct the dataset at a low cost using sparse annotation. The data collection should include metal objects of all shapes and sizes. Each sample in the dataset contains paired original projections. and the corresponding metal trajectory Step 2: Use the above dataset to test the SAM-based metal artifact segmentation model. Training is performed by stitching together projection frames with different contrasts. As input, the metal trajectory is obtained by thresholding and then projecting the image domain. As a prompt; Step 3, the trained SAM-based metal artifact segmentation model Applied to actual projection data, it predicts the precise projection domain metal trajectory angle by angle. Step 4: Analyze the predicted metal trajectory in the projection domain. Morphological processing is performed, and the original projection is triangulated and interpolated using the processed mask. In step 5, padding is applied to the interpolated projection to suppress truncation artifacts, and a weighted FDK algorithm is used for reconstruction to suppress incomplete sampling artifacts, resulting in the final CBCT image after metal artifact removal.
[0009] Furthermore, in step 1, CBCT data from actual scenes is collected, and a dataset is constructed at a low cost using sparse annotation. The sparsity ratio is unlimited, and the collected data should include metal objects of all shapes as much as possible. After completion, each group of samples in the dataset contains paired original projections. and the corresponding metal trajectory .
[0010] Furthermore, in step 2, a metal artifact segmentation model based on SAM is trained using the dataset constructed in step 1. During training, projected frames with different contrasts are stitched together. As input, the metal trajectory obtained by image domain thresholding and reprojection As a Prompt, the specific process is as follows:
[0011] (1)
[0012] in, It is a metal artifact segmentation model based on SAM. These are its corresponding trainable parameters. It is the segmentation loss function. spliced projection frames As input, information on different contrasts is provided to the segmentation network as much as possible to improve the accuracy of metal trajectory segmentation. and By projecting the original image Histogram equalization and histogram-based enhancements can be performed, but are not limited to the methods described above. The metal trajectory is obtained by thresholding and then projecting the image domain. As a prompt, to prevent the network from deviating from the convergence direction during training, the fusion of prompts can be adopted. Figure 1 The direct fusion shown in fusion method one can also be used, as shown in fusion method two, but it is not limited to these two methods. When fine-tuning SAM, some parameters or all parameters can be adjusted. The above steps enable rapid fine-tuning of SAM, allowing the segmentation model trained on massive amounts of data to be applied to the highly challenging task of metal artifact segmentation in the medical field.
[0013] Furthermore, in step 3, the trained SAM-based metal artifact segmentation model is... Applying this to actual projection data, it predicts the metal trajectory in the projection domain for each angle. This allows for rapid and precise removal of metal artifacts in clinical surgical settings, improving surgical accuracy and reducing surgical risks. The specific process is as follows:
[0014] (2)
[0015] Furthermore, in step 4, the predicted projection domain metal trajectory is... Perform morphological processing, and then use the processed mask to project the original image. Triangulation interpolation is performed, which involves using the projected values of the surrounding non-metallic regions to fill the values of the metallic trajectory region. The specific process is as follows:
[0016] (3)
[0017] (4)
[0018] in, This is a morphological processing operation. This indicates element-wise multiplication. This represents the interpolation function based on triangulation. The above steps can achieve more accurate interpolation while ensuring that the measurement data without metal contamination is completely accurate, thus achieving a better balance between the consistency of physical information and the accuracy of missing area estimation.
[0019] Furthermore, in step 5, the interpolated projection... Padding is applied to the detector boundaries to suppress truncation artifacts. Then, a weighted FDK algorithm is used to perform 3D reconstruction on the padded projection to suppress incomplete sampling artifacts, resulting in the final CBCT image with artifacts removed. This step can simultaneously suppress both metal artifacts and truncation artifacts that commonly occur in clinical surgical scenarios, accelerating the practical application of the algorithm. The specific process is as follows:
[0020] (5)
[0021] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the CBCT projection metal trajectory segmentation and artifact removal method using sparsely labeled fine-tuned SAM.
[0022] A computer-readable storage medium having stored thereon computer instructions, which are executed by a processor to implement the CBCT projection metal trajectory segmentation and artifact removal method for sparsely labeled fine-tuned SAM.
[0023] Compared with the prior art, the advantages of the present invention are as follows:
[0024] (1) Low-cost SAM fine-tuning strategy based on sparse annotation: Currently, most deep learning methods require large-scale pixel-level accurate annotation information as training data, resulting in high data annotation costs. However, the sparse annotation strategy adopted in this invention only requires annotation of some projection angles to construct an effective dataset. By fine-tuning the pre-trained SAM model, its powerful zero-shot generalization ability can be fully utilized, reducing data annotation costs while achieving high-precision segmentation of complex metal trajectories.
[0025] (2) Feature enhancement combining multi-contrast input and threshold cues: To address the issues of large dynamic range in CBCT projection data and low contrast in some metal trajectories, this invention uses multi-contrast projection frame stitching as network input to provide the model with more edge information. In addition, this invention uses image domain threshold segmentation results as "cues" to guide the SAM model, minimizing convergence and divergence issues during training and enhancing the SAM model's ability to focus on target regions and its segmentation stability.
[0026] (3) High robustness achieved through complementary advantages of the projection domain and the image domain: The method of this invention overcomes the limitations of single-domain methods. The image domain cue information can be used to obtain the accurate prior location of metal within the FOV; for metal outside the FOV, the SAM model can accurately detect the trajectory in the projection domain. This approach enables the algorithm to achieve robust segmentation results when facing metal implants of different types, shapes, and locations inside and outside the FOV.
[0027] (4) Weighted Reconstruction Method Against Truncation and Incomplete Sampling: This invention is not limited to metal segmentation, but also constructs a complete artifact removal and image reconstruction process. After obtaining an accurate metal mask, the edges are first optimized using morphological methods, then the missing data is filled using triangulation interpolation, and reconstruction is performed by combining detector edge padding and weighted FDK algorithm, thereby removing metal artifacts and effectively suppressing truncation artifacts and incomplete sampling artifacts, and finally obtaining high-quality CBCT images. Attached Figure Description
[0028] Figure 1 : Schematic diagram of the overall process of this invention. Figure 2 Comparison of the metal trajectory segmentation effect of this invention on real CBCT projection data. Figure 3 Comparison of reconstruction results after correcting for metal artifacts in real CBCT data according to this invention. Detailed Implementation
[0029] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.
[0030] Example: Figure 1 As shown, this invention provides a CBCT projection metal trajectory segmentation algorithm based on threshold Prompt and SAM fine-tuning using sparse annotation. The final test results are as follows. Figure 2 , Figure 3 As shown, the specific steps are as follows:
[0031] Step 1: Collect CBCT data from real-world scenarios and construct a dataset at a low cost using sparse annotation. The sparsity ratio is unlimited. When collecting data, include metal objects of all shapes as much as possible. After completion, each sample in the dataset contains paired original projections. and the corresponding metal trajectory .
[0032] Step 2: Train a SAM-based metal artifact segmentation model based on the dataset constructed in Step 1. During training, projected frames with different contrasts are stitched together. As input, the metal trajectory obtained by image domain thresholding and reprojection As a Prompt, the specific process is as follows:
[0033] (1)
[0034] in, It is a metal artifact segmentation model based on SAM. These are its corresponding trainable parameters. It is the segmentation loss function. spliced projection frames As input, information on different contrasts is provided to the segmentation network as much as possible to improve the accuracy of metal trajectory segmentation. and By projecting the original image Histogram equalization and histogram-based enhancements can be performed, but are not limited to the methods described above. The metal trajectory is obtained by thresholding and then projecting the image domain. As a prompt, to prevent the network from deviating from the convergence direction during training, the fusion of prompts can be adopted. Figure 1 The direct fusion shown in fusion method one can also be used, as shown in fusion method two, but it is not limited to these two methods. When fine-tuning SAM, some parameters or all parameters can be fine-tuned.
[0035] Step 3: Apply the trained SAM-based metal artifact segmentation model Applied to actual projection data, it predicts the precise projection domain metal trajectory angle by angle. The specific process is as follows:
[0036] (2)
[0037] Step 4: Analyze the predicted metal trajectory in the projection domain. Perform morphological processing, and then use the processed mask to project the original image. Triangulation interpolation is performed, which involves using the projected values of the surrounding non-metallic regions to fill the values of the metallic trajectory region. The specific process is as follows:
[0038] (3)
[0039] (4)
[0040] in, This is a morphological processing operation. This indicates element-wise multiplication. This represents an interpolation function based on triangulation.
[0041] Step 5, interpolate the projection Padding is applied to the detector boundaries to suppress truncation artifacts; then, a weighted FDK algorithm is used to perform 3D reconstruction on the padded projection to suppress incomplete sampling artifacts, resulting in the final CBCT image with artifacts removed. The specific process is as follows:
[0042] (5)
[0043] Effectiveness evaluation:
[0044] This invention discloses a method for CBCT projection metal trajectory segmentation and artifact removal using sparsely labeled fine-tuned SAM. The method was tested on real CBCT data, and the results are as follows: Figure 2 , Figure 3 As shown, the comparison results of projection domain metal trajectory segmentation of real data and the comparison results of reconstructed images after metal artifact removal are presented respectively. The test was performed in the CMD command line under the Windows system. The test procedure includes: collecting real scene data and performing sparse annotation to construct a dataset; fine-tuning and training the SAM model using stitched multi-contrast projection frames and image domain threshold Prompt; using the trained model to predict the projection domain metal mask of real data, and performing morphological processing and triangulation interpolation on it; finally, performing padding and weighted FDK reconstruction on the interpolated projection to obtain CBCT images without metal artifacts. The advantages of this method are mainly reflected in the following points:
[0045] (1) Low-cost SAM fine-tuning strategy based on sparse annotation: Currently, most deep learning methods require large-scale pixel-level accurate annotation information as training data, resulting in high data annotation costs. However, the sparse annotation strategy adopted in this invention only requires annotation of some projection angles to construct an effective dataset. By fine-tuning the pre-trained SAM model, its powerful zero-shot generalization ability can be fully utilized, reducing data annotation costs while achieving high-precision segmentation of complex metal trajectories.
[0046] (2) Feature enhancement combining multi-contrast input and threshold cues: To address the issues of large dynamic range in CBCT projection data and low contrast in some metal trajectories, this invention uses multi-contrast projection frame stitching as network input to provide the model with more edge information. In addition, this invention uses image domain threshold segmentation results as "cues" to guide the SAM model, minimizing convergence and divergence issues during training and enhancing the SAM model's ability to focus on target regions and its segmentation stability.
[0047] (3) High robustness achieved through complementary advantages of the projection domain and the image domain: The method of this invention overcomes the limitations of single-domain methods. The image domain cue information can be used to obtain the accurate prior location of metal within the FOV; for metal outside the FOV, the SAM model can accurately detect the trajectory in the projection domain. This approach enables the algorithm to achieve robust segmentation results when facing metal implants of different types, shapes, and locations inside and outside the FOV.
[0048] (4) Weighted Reconstruction Method Against Truncation and Incomplete Sampling: This invention is not limited to metal segmentation, but also constructs a complete artifact removal and image reconstruction process. After obtaining an accurate metal mask, the edges are first optimized using morphological methods, then the missing data is filled using triangulation interpolation, and reconstruction is performed by combining detector edge padding and weighted FDK algorithm, thereby removing metal artifacts and effectively suppressing truncation artifacts and incomplete sampling artifacts, and finally obtaining high-quality CBCT images.
[0049] like Figure 2 As shown, the first column is the original projection, the second column is the Mask generated by image domain thresholding and reprojection. It can be observed that due to the proximity of the metal and bone CT values or the metal being outside the FOV, the segmentation results are fragmented and incomplete, failing to completely cover the entire metal trajectory. The third column shows the segmentation results of the method proposed in this invention. It can be seen that the Mask obtained by the method of this invention has smooth edges and a complete structure, accurately extracting the complete trajectory of the metal in the projection domain, which is superior to traditional thresholding segmentation methods.
[0050] like Figure 3 As shown, the first column is the result of direct reconstruction from the original projection. It can be seen that the image contains severe radial metal artifacts and dark streaks, obscuring the vertebral body and surrounding soft tissue structures, affecting clinical diagnosis. The second column is the result after interpolation using an image domain threshold segmentation mask. Due to inaccurate segmentation, obvious metal artifacts remain in the reconstructed image, and the edges of some bone structures are blurred. The third column is the reconstruction result using the method of this invention. It can be seen that the method of this invention effectively removes most of the metal artifacts while well preserving bone structures and soft tissue details. Image contrast and clarity are also improved, verifying the effectiveness of the method of this invention in clinical applications.
[0051] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any transformations or substitutions that can be conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A CBCT projection metal track segmentation and artifact removal method for sparsely labeled fine-tuned SAM, characterized in that, The specific steps are as follows: Step 1: Collect CBCT data from real-world scenarios and construct the dataset at a low cost using sparse annotation. The data collection should include metal objects of all shapes and sizes. Each sample in the dataset contains paired original projections. and the corresponding metal trajectory Step 2: Use the above dataset to test the SAM-based metal artifact segmentation model. Training is performed by stitching together projection frames with different contrasts. As input, the metal trajectory obtained by thresholding and then projecting the image domain is... As a prompt; Step 3, the trained SAM-based metal artifact segmentation model Applied to actual projection data, it predicts the precise projection domain metal trajectory angle by angle. Step 4: Analyze the predicted metal trajectory in the projection domain. Morphological processing is performed, and the original projection is triangulated and interpolated using the processed Mask. In step 5, padding is applied to the interpolated projection to suppress truncation artifacts, and weighted FDK algorithm is used for reconstruction to suppress incomplete sampling artifacts, resulting in the final CBCT image after metal artifact removal.
2. The CBCT projection metal track segmentation and artifact removal method of sparse-labeled fine-tuned SAM according to claim 1, wherein, The actual scene CBCT data is collected in step 1, and a sparse annotation is used to construct the dataset at a low cost, and each group of samples in the dataset after this contains a pair of original projections and corresponding metal tracks .
3. The CBCT projection metal track segmentation and artifact removal method of sparse-labeled fine-tuned SAM of claim 1, wherein, In step 2, the SAM-based metal artifact segmentation model is trained based on the dataset constructed in step 1 The stitched projection frames of different contrasts are input during training Metal trajectories obtained by re-projection with image domain thresholding As Prompt, the specific process is as follows: (1) wherein, is a SAM-based metal artifact segmentation model, is a corresponding trainable parameter, is a segmentation loss function, the concatenated projection frames are inputted, and can be obtained by histogram equalization and histogram statistics-based enhancement on the original projection ; the metal trajectories obtained by image domain threshold segmentation and re-projection are also inputted as Prompt, and the SAM fine-tuning can fine-tune partial parameters or all parameters.
4. The CBCT projection metal track segmentation and artifact removal method of sparse-labeled fine-tuned SAM of claim 1, wherein, In step 3, the trained SAM-based metal artifact segmentation model is applied to the actual projection data, predicting the precise projection domain metal trajectory per angle The specific process is as follows: (2)。 5. The CBCT projection metal track segmentation and artifact removal method of sparse-labeled fine-tuned SAM of claim 1, wherein, In step 4, the predicted projected domain metal track is morphologically processed and then the original projection is triangulation interpolated, i.e. the values of the metal track area are filled with the values of the surrounding non-metal area, in the following way: (3) (4) wherein, is a morphological processing operation, denotes element-wise multiplication, denotes an interpolation function based on a triangulation.
6. The CBCT projection metal track segmentation and artifact removal method of sparse-labeled fine-tuned SAM of claim 1, wherein, In step 5, the interpolated projection Padding is performed at the detector boundary to suppress truncation artifacts, and a weighted FDK algorithm is used to reconstruct the padded projection to suppress incomplete sampling artifacts, to obtain the final CBCT image after metal artifact removal. The specific process is as follows: (5)。 7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: The processor implements the CBCT projection metal track segmentation and artifact removal method for sparse label fine-tuning SAM as claimed in any one of claims 1-6 when executing the program.
8. A computer readable storage medium having stored thereon computer instructions, characterized in that: The computer instructions are run by the processor to implement the CBCT projection metal track segmentation and artifact removal method for sparse label fine-tuning SAM as claimed in any one of claims 1-6.