Methods and Systems for Metal Artifact Correction and Ablation Visualization in CT Images
By using a two-stage generative adversarial network for metal artifact correction and ablation needle reconstruction, the problem of inaccurate tumor and ablation needle localization in CT-guided liver cancer ablation treatment was solved, achieving precise intraoperative visualization and real-time calculation.
Patent Information
- Application Number
- CN202310110416.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-13
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-02-13
AI Technical Summary
In CT-guided ablation therapy for liver cancer, it is difficult to accurately locate the tumor and the relative position of the ablation needle during the procedure. Furthermore, existing technologies lack sufficient precision in metal artifact correction and registration, resulting in blurred images and structural inconsistencies.
A two-stage generative adversarial network was used for metal artifact correction. The model was optimized by adversarial loss, reconstruction loss and perceptual loss to remove metal and artifacts from intraoperative CT images and register them with preoperative CT images to reconstruct the position of the ablation needle.
It improves the accuracy of intraoperative positioning, significantly reduces the amount of computation, ensures the real-time nature of calculations, and achieves clear visualization of the tumor and ablation needle in three-dimensional space.
Smart Images

Figure CN116258784B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical image processing, artificial intelligence, and three-dimensional visualization, and particularly to the fields of CT image metal artifact correction and ablation visualization for CT-guided liver cancer ablation therapy. Background Technology
[0002] Ablation therapy involves targeting the tumor under the guidance of medical imaging such as CT and ultrasound, inserting an ablation needle into the tumor location, and then using radiofrequency waves or microwaves to coagulate and inactivate soft tissue and the tumor. Due to its minimally invasive nature, precise targeting, high safety, and minimal damage to normal tissues, ablation therapy has become an important method for treating liver cancer.
[0003] In ablation therapy, image guidance plays a particularly important role in tumor localization. CT images are widely used in ablation therapy due to their advantages such as clarity, no blind spots, simple operation, and no gas or bone interference.
[0004] However, the use of CT images for ablation therapy still has some problems, such as: difficulty in observing the location and shape of the tumor through images during the operation; low registration accuracy of tumor localization through preoperative and intraoperative image registration; easy to cause problems such as image blurring and structural inconsistency when calibrating based on two-dimensional images or three-dimensional image blocks; and the inability to obtain the relative position of the tumor and ablation needle after removing strong information to complete the registration, etc. Summary of the Invention
[0005] The purpose of this application is to provide a method and system for correcting metal artifacts in CT images and visualizing ablation, which can not only display the relative position of the ablation needle and the tumor more intuitively and effectively during intraoperative treatment, improving the accuracy of intraoperative localization, but also significantly reduce the amount of computation and ensure the real-time nature of the calculation.
[0006] This application discloses a method for correcting and visualizing the ablation of metal artifacts in CT images, including: Preprocessing of preoperative and intraoperative CT images; A two-stage generative adversarial network was constructed to remove metal and artifacts from the intraoperative CT images. The preoperative CT image and the intraoperative CT image after removing metal artifacts are registered, and the intraoperative linked image of the tumor in the preoperative CT image is output. Intraoperative CT images after removing metal artifacts were reconstructed to recreate the ablation needle, providing a three-dimensional view of the tumor and the ablation needle.
[0007] In a preferred embodiment, the method is used for metal artifact correction and visualization of ablation therapy in CT-guided liver cancer ablation treatment, wherein the tumor is a liver tumor, and the step of preprocessing preoperative and intraoperative CT images includes the following sub-steps: Preoperative and intraoperative CT images were resampled to the same physical size. The resampled preoperative CT images and intraoperative CT images were rigidly registered to make their physical positions the same. The preoperative and intraoperative CT images after registration are cropped to the same size, including both the liver tumor and the area where the ablation needle is located. The CT values of the cropped preoperative and intraoperative CT images were adjusted to display the liver and other tissues and normalized to [-1,1].
[0008] In a preferred embodiment, the step of constructing the two-stage generative adversarial network includes the following sub-steps: Construct a two-stage generative adversarial network for metal artifact correction; Define the loss function; The model is optimized by iteratively minimizing the loss function.
[0009] In a preferred embodiment, the step of constructing the two-stage generative adversarial network for metal artifact correction includes the following sub-steps: A two-stage generator subnetwork is constructed, including a first-stage decoder-encoder network, which connects the decoder and encoder through a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network, which consists of a feature extraction layer, a residual module, and an image reconstruction module. A discriminator subnetwork is constructed. The input is a stitched image of the intraoperative image with artifacts and the preoperative image, or a stitched image of the intraoperative image with artifacts and the corrected intraoperative image. The output is a matrix. The discriminator subnetwork consists of 5 downsampled convolutional blocks.
[0010] In a preferred embodiment, the step of defining the loss function includes the following sub-steps: Define adversarial loss, which is used to improve the generator's ability to remove metal artifacts and the discriminator's classification ability. To define the reconstruction loss, in order to preserve the intraoperative structure while removing artifacts, and to make the output image consistent with the preoperative image in the areas where metal and artifacts are located, and consistent with the intraoperative image in the areas where non-metal and artifacts are located, the L1 distance of the corresponding regions is calculated and added according to a certain weight to obtain the reconstruction loss. To define perceptual loss, in order to minimize the similarity between the output image and the artifact-free image in features such as texture and uniformity, the L1 distance between the feature maps of the corrected intraoperative CT image and the artifact-free preoperative CT image is calculated. Define the total loss by adding the adversarial loss, reconstruction loss, and perception loss according to certain weights.
[0011] In a preferred embodiment, the step of optimizing the model by iteratively minimizing the loss function includes the following sub-steps: Combine preoperative CT images and intraoperative CT images into image pairs; The image pairs are input into a two-stage generative adversarial network, and the loss function is calculated. By selecting an optimizer, iterating a certain number of times, and minimizing the loss function, the optimized metal artifact correction model is obtained.
[0012] In a preferred embodiment, the step of registering the preoperative CT image and the intraoperative CT image after removing metal artifacts, and outputting the intraoperative linked image of the tumor fused from the preoperative CT image to the intraoperative CT image, includes the following sub-steps: The intraoperative CT image with artifacts is input into the generator subnetwork of the optimized metal artifact correction model, and the corrected intraoperative CT image without artifacts is output. Non-rigid registration was performed on the preoperative CT images and the corrected intraoperative CT images to fuse the tumor in the preoperative CT images and display it on the intraoperative CT images.
[0013] In a preferred embodiment, the step of reconstructing the ablation needle from the intraoperative CT image after removing metal artifacts includes the following sub-steps: The difference image is obtained by subtracting the intraoperative image with metal artifacts from the intraoperative image with metal artifacts. Binarize the difference image to segment out the ablation needles; The segmented ablation needles are constructed into a three-dimensional point cloud, and the skeleton of the point cloud is extracted. The axis of the ablation needle is located by calculating the line where the point cloud is located using 3D Hough line detection. Record all voxels that pass through the axis on the difference image to obtain the voxel value variation curve along the axis; Locate the point where the curve drops sharply to pinpoint the endpoint of the ablation needle; The ablation needle was reconstructed using the obtained axis and endpoints in the corrected intraoperative images; The tumor is delineated before surgery, and the relative positions of the tumor and the reconstructed electrode needles are displayed in three dimensions during surgery.
[0014] This application also discloses a CT image metal artifact correction and ablation visualization system, including: The image and processing module is used to preprocess preoperative and intraoperative CT images; The metal artifact correction model building module is used to build a two-stage generative adversarial network to remove metal and artifacts from intraoperative CT images. The intraoperative tumor localization module is used to register preoperative images and intraoperative CT images after removing metal artifacts, and outputs intraoperative linked images of tumor fused from preoperative images into intraoperative images. The ablation needle reconstruction module is used to reconstruct the ablation needle in intraoperative CT images after removing metal artifacts, providing a three-dimensional display of the tumor and the ablation needle.
[0015] In a preferred embodiment, the system is used for metal artifact correction and visualization of ablation therapy in CT-guided liver cancer ablation treatment, wherein the tumor is a liver tumor, and the image preprocessing module is further used for: Preoperative and intraoperative CT images were resampled to the same physical size. The resampled preoperative CT images and intraoperative CT images were rigidly registered to make their physical positions the same. The preoperative and intraoperative CT images after registration are cropped to the same size, and must include the liver tumor and the area where the ablation needle is located. The CT values of the cropped preoperative and intraoperative CT images were adjusted to display the liver and other tissues and normalized to [-1,1].
[0016] In a preferred embodiment, the metal artifact correction model building module is further used for: Construct a two-stage generative adversarial network for metal artifact correction; Define the loss function; The model is optimized by iteratively minimizing the loss function.
[0017] In a preferred embodiment, the construction of the two-stage generative adversarial network for metal artifact correction includes: A two-stage generator subnetwork is constructed, including a first-stage decoder-encoder network, which connects the decoder and encoder through a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network, which consists of a feature extraction layer, a residual module, and an image reconstruction module. A discriminator subnetwork is constructed, taking as input either a stitched image of the intraoperative image with artifacts and a preoperative image, or a stitched image of the intraoperative image with artifacts and a corrected intraoperative image, and outputting as a matrix. The discriminator subnetwork consists of 5 downsampled convolutional blocks.
[0018] In a preferred embodiment, the loss function is defined as follows: Define adversarial loss, which is used to improve the generator's ability to remove metal artifacts and the discriminator's classification ability. To define the reconstruction loss, in order to preserve the intraoperative structure while removing artifacts, and to make the output image consistent with the preoperative image in the areas where metal and artifacts are located, and consistent with the intraoperative image in the areas where non-metal and artifacts are located, the L1 distance of the corresponding regions is calculated and added according to a certain weight to obtain the reconstruction loss. To define perceptual loss, in order to minimize the similarity between the output image and the artifact-free image in features such as texture and uniformity, the L1 distance between the feature maps of the corrected intraoperative CT image and the artifact-free preoperative CT image is calculated. Define the total loss by adding the adversarial loss, reconstruction loss, and perception loss according to certain weights.
[0019] In a preferred embodiment, the optimization model includes: Combine preoperative CT images and intraoperative CT images into image pairs; The image pairs are input into a two-stage generative adversarial network, and the loss function is calculated. By selecting an optimizer, iterating a certain number of times, and minimizing the loss function, the optimized metal artifact correction model is obtained.
[0020] In a preferred embodiment, the intraoperative tumor localization module is further used for: The intraoperative CT image with artifacts is input into the generator subnetwork of the optimized metal artifact correction model, and the corrected intraoperative CT image without artifacts is output. Non-rigid registration was performed on the preoperative CT images and the corrected intraoperative CT images to fuse the tumor in the preoperative CT images and display it on the intraoperative CT images.
[0021] In a preferred embodiment, the ablation needle reconstruction module is further configured to: The difference image is obtained by subtracting the intraoperative image with metal artifacts from the intraoperative image with metal artifacts. Binarize the difference image to segment out the ablation needles; The segmented ablation needles are constructed into a three-dimensional point cloud, and the skeleton of the point cloud is extracted. The axis of the ablation needle is located by calculating the line where the point cloud is located using 3D Hough line detection. Record all voxels that pass through the axis on the difference image to obtain the voxel value variation curve along the axis; Locate the point where the curve drops sharply to pinpoint the endpoint of the ablation needle; The ablation needle was reconstructed using the obtained axis and endpoints in the corrected intraoperative images; The tumor is delineated before surgery, and the relative positions of the tumor and the reconstructed electrode needles are displayed in three dimensions during surgery.
[0022] This application also discloses a CT image metal artifact correction and ablation visualization system, including: Memory, used to store computer-executable instructions; and, A processor for implementing the steps of the method as described above when executing the computer-executable instructions.
[0023] This application also discloses a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method described above.
[0024] The embodiments of this application have the following technical effects: 1. Register preoperative and intraoperative images to locate the tumor during surgery, making the shape and location of the tumor clearly visible.
[0025] 2. By removing metal and artifacts from intraoperative images through metal artifact correction methods, the information of intraoperative images and preoperative images are made symmetrical, and then registration is performed, which effectively improves the registration accuracy.
[0026] 3. By designing a metal artifact correction model based on a two-stage generative adversarial network suitable for full-size 3D images, image blurring is effectively avoided.
[0027] 4. By designing a reconstruction process for the ablation needle, the location information of the ablation needle can be effectively visualized in three-dimensional space.
[0028] In summary, this application not only enables a more intuitive and effective display of the relative positions of the ablation needle and the tumor during intraoperative treatment, improving the accuracy of intraoperative localization, but also greatly reduces the amount of computation and ensures the real-time nature of the calculation.
[0029] The specification of this application contains numerous technical features distributed across various technical solutions. Listing all possible combinations of these technical features (i.e., technical solutions) would make the specification excessively lengthy. To avoid this problem, the various technical features disclosed in the above-described invention, the various technical features disclosed in the following embodiments and examples, and the various technical features disclosed in the accompanying drawings can be freely combined to form various new technical solutions (all of which are considered to have been described in this specification), unless such a combination of technical features is technically infeasible. For example, one example discloses feature A+B+C, and another example discloses feature A+B+D+E. Features C and D are equivalent technical means that serve the same function, and technically only one needs to be used; they cannot be used simultaneously. Feature E can technically be combined with feature C. Therefore, the solution A+B+C+D should not be considered as described because it is technically infeasible, while the solution A+B+C+E should be considered as described. Attached Figure Description
[0030] Figure 1 This is a flowchart illustrating a method for visualizing and ablation metal artifacts in CT images according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the framework of a CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention; Figure 3 This is a diagram showing intraoperative CT images and preoperative CT images in a CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention. Figure 4 A two-stage generative adversarial network structure diagram for metal artifact correction in a CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention; Figure 5 This is an ablation visualization framework diagram of relevant steps in a CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention; Figure 6 (a) Figure 6 (b) and Figure 6 Figure (c) shows the result of metal artifact correction in the CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention, wherein Figure (a) is an intraoperative CT image with artifacts, Figure (b) is a preoperative CT image, and Figure (c) is a corrected intraoperative CT image. Figure 7 In a CT image metal artifact correction and ablation visualization method according to an embodiment of the invention, the variation curve of voxel values along the axis of the differential image is shown. Figure 8 In a CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention, the reconstruction results of the ablation needle on three slices in an intraoperative CT image are shown. Figure 9 This is a three-dimensional visualization result of an intraoperative CT image in a CT image metal artifact correction and ablation visualization method according to an embodiment of the present invention.
[0031] Figure 10 This is a schematic diagram of a CT image metal artifact correction and ablation visualization system according to an embodiment of the present invention. Detailed Implementation
[0032] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.
[0033] The following is a brief summary of some of the innovative aspects of this application: The inventors of this application, through long-term research, discovered that during CT-guided ablation therapy, the intraoperative CT images typically contain metal artifacts because the ablation needle is made of metal. Strong metal artifacts can partially or completely cover the tumor tissue, and the anatomical structures around the ablation needle will also be blurred, making it impossible for doctors to accurately determine the relative position of the tumor and the ablation needle based on the intraoperative CT scan. Preoperative CT can provide complete tumor information; by registering the preoperative and intraoperative CT scans, the complete tumor and ablation needle can be displayed simultaneously during the procedure. However, because metal and artifacts are strong information, significant errors can occur during registration. Therefore, it is necessary to correct for metal artifacts on the intraoperative CT scan, remove metal and artifacts, then register it with the preoperative CT scan, and finally reconstruct the ablation needle to complete the visualization of the ablation.
[0034] Most existing deep learning solutions for metal artifact correction in CT images employ generative adversarial networks (GANs) or image inpainting. For example, Wang et al. (WANG J, NOBLE JH, DAWAN TBM. Metal artifact reduction for the segmentation of the intracortical articulation in CT images of the ear with 3D-conditional GANs[J]. Medical image analysis, 2019, 58: 101-553.) used a three-dimensional conditional generative adversarial network for metal correction in CT images of cochlear implants.
[0035] Most existing methods for visualizing ablation needles employ three-dimensional Hough line detection to locate the axis and tip of the ablation needle. For example, Qiu et al. (QIUW, YUCHIM, DINGM, et al. Need lesegmentation using 3D Hough transform in 3D TRUS guided prostate transperineal therapy[J]. Medical Physics, 2013, 40(4): 042902.) used Hough line detection to locate the ablation needle in three-dimensional ultrasound for guiding interventional treatment of prostate adenocarcinoma.
[0036] Based on actual clinical needs, this study aims to locate tumors intraoperatively and visualize the relative positions of the ablation needle and the tumor. To locate the tumor intraoperatively, preoperative and intraoperative images need to be registered. For the same modality, when the patient's posture remains unchanged, the main registration challenge lies in the strong information provided by metal artifacts. Metal artifacts need to be removed first; therefore, this invention focuses on metal artifact correction. Wei et al. (WEID, AHMADS, HUOJ, et al. SLIR: Synthesis, localization, inpainting, and registration for image-guided thermolablation of liver tumors[J]. Medical image analysis, 2020, 65:101763.) segmented the metal ablation needle in CT image-guided liver cancer ablation therapy and filled in the needle's area using partial convolution, thereby removing the ablation needle before registering and locating the tumor. However, this method only considers the area where the ablation needle is located, without taking into account the metal artifacts caused by the ablation needle. Radial metal artifacts can also cause asymmetry in preoperative and intraoperative images, leading to decreased registration accuracy. In addition, after removing the ablation needle from the intraoperative CT images, this method does not reconstruct the ablation needle, resulting in the loss of ablation needle information and making it impossible for doctors to observe the relative position of the ablation needle and the tumor.
[0037] Based on the above research and the problems found, the inventors of this application have creatively proposed a technical solution for CT image metal artifact correction and ablation visualization, which can be used, for example, in a visualization system for CT-guided liver cancer ablation. It includes a metal artifact correction module and an ablation visualization module. First, the metal ablation needle and artifacts in the intraoperative image are corrected. Then, the tumor is located by registration with the preoperative image. Finally, the ablation needle is reconstructed to display the relative position of the ablation needle and the tumor in three-dimensional space.
[0038] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0039] The first embodiment of this application relates to a method for correcting metal artifacts in CT images and visualizing ablation, which enables intuitive and effective display of the relative positions of the tumor and the ablation needle during intraoperative treatment. The process is as follows: Figure 1 As shown, the model consists of a metal artifact correction model and an ablation visualization model. Intraoperative CT images with artifacts are used to generate corrected artifact-free intraoperative CT images. Preoperative CT images are registered with the corrected intraoperative CT images to locate the tumor. The ablation needle is segmented from the intraoperative CT images and reconstructed. Finally, the tumor and ablation needle are displayed in three dimensions.
[0040] The method in this embodiment includes the following steps: Step 110, Image preprocessing: Preprocess the preoperative CT images and intraoperative CT images.
[0041] Preferably, step 110 may further include the following sub-steps: Step 111: Obtain CT images of the ablation patient before and during the procedure using CT scans, and resample them to the same size; Step 112: Perform rigid registration on the preoperative and intraoperative CT images. The preoperative images are floating images, and the intraoperative images are fixed images.
[0042] Step 113: Crop the registered preoperative and intraoperative images to the same size, and adjust the window width and window level of the image CT values to display the shape and location of the tumor. The tumor may be, for example, a liver tumor, a lung tumor, a kidney tumor, a breast cancer tumor, etc. Step 114: Normalize the CT image I to [-1,1] according to formula (1), where min and max represent the maximum and minimum gray values of the CT image. Preferably, in step 111, the intraoperative and preoperative CT images are resampled to 1.2mm × 1.2mm × 1.5mm. Preferably, for step 113, in order to clearly display the tumor, the image size is cropped to 256×256×32 according to the center, and the CT value is adjusted to [-300, 300] HU; Step 120, Building a Metal Artifact Correction Model: A two-stage generative adversarial network is built to remove metal and artifacts from intraoperative CT images.
[0043] Preferably, step 120 may further include the following sub-steps: Step 121: Construct a two-stage generative adversarial network (GAN) for metal artifact correction. The GAN comprises a two-stage generator subnetwork and a discriminator subnetwork. The two-stage generator subnetwork includes a first-stage decoder-encoder, connected by a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network consisting of a feature extraction layer, a residual module, and an image reconstruction module. The discriminator subnetwork consists of five downsampling convolutional blocks.
[0044] More specifically, step 121 may further include the following sub-steps: Step 1211: The encoder-decoder of the first stage, as described in step 121, consists of four downsampling convolutional blocks, a dilated spatial convolutional pooling pyramid module, and four upsampling convolutional blocks connected in series. The final output image has the same size as the original image. The structure of each module is as follows: The downsampling convolutional block consists of a concatenated convolutional layer, a batch regularization layer, and a LeakyReLu activation function. After one convolution operation, the length, width, and height of the image become half of their original values. The dilated spatial convolutional pooling pyramid module consists of three dilated convolutions and a global average pooling layer connected in parallel. The three dilated convolutions have a kernel size of 3, a stride of 1, and dilation rates of 1, 6, and 12, respectively. The feature layers obtained from the encoder output are then processed through the three dilated convolutions and the global pooling layer, and concatenated along the channels. The upsampling convolutional block consists of a transposed convolutional layer, batch regularization, and the LeakyReLu activation function. The transposed convolution has a kernel size of 4 and a stride of 2. After one deconvolution operation, the image's length, width, and height are doubled.
[0045] Step 1212: The second-stage image enhancement network, as described in step 121, is used to improve the image quality of the output image. The final output image has the same size as the input image. The image enhancement network consists of a feature extraction layer, a residual module, and a reconstruction layer. The structure of each module is as follows: The feature extraction layer consists of a convolutional layer with a kernel size of 3 and a stride of 2, and a LeakyReLu activation function; The residual module consists of 3 residual blocks and 1 convolutional block, which are connected by jumpers; The reconstruction layer consists of one transposed convolution and one convolution.
[0046] Step 1213: As described in step 121, the discriminator sub-network consists of five downsampling feature extraction layers connected in series. After each feature extraction layer, the image length, width, and height are reduced to half their original values. The discriminator finally outputs a matrix, where the value of each point in the matrix represents the probability that the corresponding image patch in the input image contains artifacts.
[0047] Step 122: Define the loss function. The loss function consists of adversarial loss, reconstruction loss, and perceptual loss. The adversarial loss, reconstruction loss, and perceptual loss may further include the following sub-steps: Step 1221: Adversarial Loss: Adversarial loss is used to improve the generator's ability to remove metal artifacts and the discriminator's classification ability. The calculation formula is shown in equation (2). Intraoperative images with artifacts are considered. Input generator Intraoperative images after artifact correction were obtained. Discriminator judge Artifact-free preoperative CT images The authenticity; Step 1222: Reconstruction Loss: In order to preserve intraoperative structures while removing artifacts, the output image... In the area where metal and artifacts are located Compared with preoperative images Consistent, non-metallic areas and artifacts. With intraoperative images Consistent, calculate the L1 distance of the corresponding regions and sum them according to certain weights to obtain the reconstruction loss. , Indicates the number of voxels in the image; Step 1223: Perceptual Loss: In order to minimize the similarity between the output image and the artifact-free image in features such as texture and uniformity, the perceptual loss between the two is calculated. Indicates the number of slices in the image. The image represents the first Layer slice, This represents the feature extractor, which calculates the L1 distance between the feature maps of the output image and the artifact-free image. Step 1224: Overall Loss: The adversarial loss, reconstruction loss, and perception loss are weighted according to certain criteria. and Add; (7) Step 123: Optimize the model by iteratively minimizing the loss function, which may include the following sub-steps: Step 1231: Combine the preoperative CT images and intraoperative CT images into image pairs; Step 1232: Input the intraoperative CT image into the generator subnetwork of the model, output the corrected intraoperative CT image, calculate the loss function, minimize the loss function through the optimizer, and update the network parameters of the generator; Step 1233: Stitch the intraoperative CT image and the preoperative CT image along the channel and input them into the discriminator subnetwork of the model; or stitch the corrected intraoperative CT image and the preoperative CT image along the channel and input them into the discriminator subnetwork of the model, minimize the loss function through the optimizer, and update the network parameters of the discriminator. Step 1234: Repeat steps 1232 and 1233, i.e., iterate a certain number of times, update the network parameters of the generator and discriminator, and obtain the final optimized metal artifact correction model.
[0048] Preferably, for the convolutional layer of the downsampling module described in step 1211, the kernel size is 3 and the stride is 2; Preferably, for the transposed convolutional layer of the upsampling module described in step 1211, the transposed convolutional kernel size is 4 and the stride is 2; Preferably, the residual block of the residual module described in step 1212 consists of a convolutional layer, a batch regularization layer, a LeakyReLu activation function, a convolutional layer, and a batch regularization layer. The convolutional kernel size of the convolutional layer is 3, and the stride is 1. Preferably, for the reconstruction layer described in step 1212, the transposed convolution consists of a transposed convolutional layer with a kernel size of 4 and a stride of 2 and a LeakyReLu activation function; the convolution consists of a convolutional layer with a kernel size of 3 and a stride of 1 and a Tanh activation function. Preferably, the downsampling feature extraction layer of the discriminator described in step 1213 consists of a convolutional layer with a kernel size of 5 and a stride of 2, a LeakyReLu activation function dropout layer, and the last layer does not contain a dropout layer. The activation function is the Tanh function. Preferably, for the reconstruction loss described in step 1222, the hyperparameter Set to 0.2; Preferably, the feature extraction network described in step 1223 is a VGG16 pre-trained using the ImageNet dataset, and the output feature layer consists of a first pooling layer, a second pooling layer, and a third pooling layer. Preferably, for the overall loss described in step 1224, the hyperparameter and Set them to 10 and 1 respectively; Preferably, for the optimization process described in step 123, the process is iterated 300 times, with the generator using the Adam optimizer to optimize the model and the discriminator using the SGD optimizer to optimize the model. Step 130, Intraoperative tumor localization: Register the preoperative image containing a clear tumor and the intraoperative CT image after removing metal artifacts, and output the intraoperative linked image in which the tumor from the preoperative image is fused into the intraoperative image.
[0049] Preferably, step 130 may further include the following sub-steps: Step 131: Input the intraoperative images with metal and artifacts into the optimized metal artifact correction model, and obtain the intraoperative images without metal artifacts through the generator of the metal artifact correction model.
[0050] Step 132: Register the preoperative image with the intraoperative image after metal artifact correction to display the complete tumor in the preoperative image during the operation.
[0051] Preferably, for step 132, the Elastic registration method, including affine transformation and deformable transformation, is used to register the preoperative CT and the corrected intraoperative CT. Steps 140-150, Ablation Needle Reconstruction: The ablation needle is reconstructed from the intraoperative CT image after removing metal artifacts, and the tumor and the ablation needle are displayed in three dimensions.
[0052] Preferably, step 140 may further include the following sub-steps: Step 141: Subtract the intraoperative image after correcting for metal artifacts from the intraoperative image with metal artifacts to obtain the difference image; Step 142: Binarize the difference image to segment out the ablation needles; Step 143: Construct a three-dimensional point cloud from the segmented ablation needles and extract the skeleton from the point cloud; Step 144: Use 3D Hough line detection to calculate the line where the point cloud is located, and locate the axis of the ablation needle; Step 145: Record all voxels passing through the axis on the difference image to obtain the voxel value variation curve along the axis; Step 146: Locate the point where the curve drops sharply and pinpoint the endpoint of the ablation needle; Step 147: Reconstruct the ablation needle using the obtained axis and endpoints in the corrected intraoperative images; Step 148: Delineate the tumor before surgery and display the relative positions of the tumor and the reconstructed electrode needles in three dimensions during surgery.
[0053] Preferably, in step 142, a threshold of 0.7 is selected to binarize the difference image; Preferably, for step 144, a four-parameter point-direction representation of a straight line in three-dimensional space is used, and the four parameters of the line containing the point cloud are calculated by Hough transformation, thereby locating the axis of the ablation needle. Preferably, in step 146, the position where the change curve drops rapidly is determined by a threshold T, and the coordinates corresponding to the first point where the change curve is less than the threshold T are the endpoints. Preferably, the threshold T is calculated as follows: all voxels are divided into ablation needle and background regions based on their voxel values I, and the probability densities of the ablation needle and background regions are calculated respectively. and Threshold T makes .
[0054] The CT image metal artifact correction and ablation visualization method described in the above embodiments has the following technical effects: 1. It can effectively help doctors clearly observe tumors during surgery. By registering preoperative CT images with intraoperative CT images, it solves the problem of tumors being obscured by metal artifacts. 2. It can intuitively and effectively display the puncture process of liver cancer ablation treatment, and display the relative position of the tumor and the ablation needle in three dimensions during the operation, thereby improving the accuracy of ablation treatment; 3. Effectively solves the problem of low registration accuracy caused by metal and artifacts. First, correct the metal and artifacts in the intraoperative CT image, and then perform registration. 4. A de-artifact model applicable to full-size 3D images is proposed, which effectively solves the problem of discontinuous image structure caused by using 2D and 3D image blocks as input, as well as the problem of image blurring caused by full-size image input in existing methods; 5. An artifact removal model was proposed, which not only considers the removal of local ablation needles, but also the removal of radial artifacts caused by ablation needles. 6. An artifact removal model was proposed to solve the problem of changes in the anatomical structure of intraoperative CT images during the removal of metal artifacts. By designing a weighted reconstruction loss, metal artifacts in intraoperative CT images are removed while ensuring that the anatomical structure of intraoperative CT images remains unchanged, and the position and shape information of intraoperative anatomical structures can be reflected in a true and effective manner. 7. A reconstruction process for ablation needles was proposed, which greatly reduced the amount of computation by extracting the skeleton and ensured the real-time performance of the calculation.
[0055] To better understand the technical solution of this application, a specific example is provided below. The details listed in this example are mainly for ease of understanding and are not intended to limit the scope of protection of this application.
[0056] This example illustrates a method for metal artifact correction and ablation visualization in CT image-guided liver cancer ablation surgery. The method comprises a metal artifact correction model and an ablation visualization model. Intraoperative CT images with artifacts are generated to produce corrected artifact-free intraoperative CT images. Preoperative CT images are registered with the corrected intraoperative CT images to locate the tumor. The ablation needle is segmented from the intraoperative CT images and reconstructed. Finally, the tumor and ablation needle are displayed in three dimensions. The method in this embodiment includes the following steps: Step 210, Image preprocessing: Preprocess the intraoperative CT images and intraoperative CT images.
[0057] Step 220, Metal artifact correction model construction: Construct a two-stage generative adversarial network to remove metal and artifacts from intraoperative CT images; Step 230, Intraoperative tumor localization: Register the preoperative images and the intraoperative CT images after removing metal artifacts, and output the intraoperative linked images in which the tumor from the preoperative images is fused with the intraoperative images. Steps 240-250, Ablation Needle Reconstruction: The ablation needle is reconstructed from the intraoperative CT image after removing metal artifacts, and the tumor and the ablation needle are displayed in three dimensions.
[0058] The implementation methods for each step are described in detail below.
[0059] First, step 210 further includes the following sub-steps: Step 211: Obtain preoperative and intraoperative CT images of the patient undergoing ablation via CT scan. Figure 3 The images show intraoperative and preoperative CT images. Due to the presence of metal artifacts, the tumor could not be observed intraoperatively, as shown in the rectangular box. Furthermore, when the preoperative CT image was used as a floating image for registration, significant registration errors occurred due to the presence of metal and artifacts, as shown in the rectangular and yellow boxes. Figure 3 As shown, the preoperative CT images and intraoperative images were resampled to 1.2×1.2×1.5mm3; Step 212: Perform rigid registration on the resampled preoperative CT image and intraoperative CT image, including translation and rotation transformations. The preoperative CT image is a floating image. Rigid registration makes the physical positions of the preoperative CT image and the intraoperative CT image the same. Step 213: Crop the registered preoperative CT image and intraoperative CT image to a size of 256×256×32. Step 214: Adjust the CT values of the cropped preoperative CT image and intraoperative CT image to [-300, 300] HU. Normalize the image to [-1, 1] according to formula (1). In the formula, min and max represent the maximum and minimum values of the image voxel values, respectively.
[0060] (1) After step 210, the result of image preprocessing is as follows: Figure 3 As shown, Figure 3 The images show intraoperative and preoperative CT images. Due to the presence of metal artifacts, the tumor could not be observed intraoperatively, as shown in the rectangular boxes. In addition, the preoperative CT images were registered as floating images. Due to the presence of metal and artifacts, there were large registration errors, as shown in the rectangular and square boxes.
[0061] Step 220 further includes the following sub-steps: Step 221: Build a two-stage generative adversarial network for metal artifact correction based on Python.
[0062] like Figure 4 As shown, the network comprises a two-stage generator subnetwork and a discriminator subnetwork. The two-stage generator subnetwork includes a first-stage decoder-encoder, which connects the decoder and encoder via a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network, consisting of a feature extraction layer, a residual module, and an image reconstruction module. The discriminator subnetwork consists of five downsampling convolutional blocks.
[0063] Step 222: Define the loss function. Define the adversarial loss (Equation 2), reconstruction loss (Equation 3), and perception loss (Equation 6), where the reconstruction loss (Equation 3) includes... =0.2; Adversarial loss, reconstruction loss, and perception loss are calculated as follows: The sum of these values yields the total loss (Equation 7). Step 223 optimizes the model by iteratively minimizing the loss function, and may further include the following sub-steps: Step 2231: Combine the preoperative CT images and intraoperative CT images into image pairs; Step 2232: Input the intraoperative CT image into the generator subnetwork of the model, output the corrected intraoperative CT image, calculate the loss function, minimize the loss function through the Adam optimizer, and update the network parameters of the generator; Step 2233: Stitch the intraoperative CT image and the preoperative CT image along the channel and input them into the discriminator subnetwork of the model; or stitch the corrected intraoperative CT image and the preoperative CT image along the channel and input them into the discriminator subnetwork of the model, minimize the loss function through the SGD optimizer, and update the network parameters of the discriminator. Step 2234: Repeat steps 2232 and 2233 300 times, updating the network parameters of the generator and discriminator once in each iteration to obtain the final optimized metal artifact correction model.
[0064] See Figure 5 This diagram is a visualization framework for ablation, in which the tumor is located during the operation by registering preoperative CT images with corrected intraoperative CT images; the ablation needle is reconstructed by segmenting and locating the ablation in the intraoperative CT images, and the relative position of the ablation needle and the tumor is displayed in three-dimensional space.
[0065] like Figure 5 As shown, step 230 further includes the following sub-steps: Step 231: Input the intraoperative CT image with artifacts into the generator subnetwork of the optimized metal artifact correction model, and output the corrected intraoperative CT image without artifacts, such as... Figure 6 As shown, this figure illustrates the results of metal artifact correction and ablation visualization methods in CT images. Figure (a) shows the intraoperative CT image with artifacts, Figure (b) shows the preoperative CT image, and Figure (c) shows the corrected intraoperative CT image. After metal artifact correction, the metal and artifacts in the intraoperative CT image are removed, while preserving the original anatomical structures. The yellow box indicates the area where registration errors occur due to metal artifacts. Step 232: Perform non-rigid registration between the preoperative CT image and the corrected intraoperative CT image, so that the tumor in the preoperative CT image is fused and displayed on the intraoperative CT image, such as... Figure 9As shown in the figure, this is a three-dimensional visualization result of an intraoperative CT image in the method of CT image metal artifact correction and ablation visualization. The positions of the tumor and ablation needle are clearly visible.
[0066] like Figure 5 As shown, step 240 further includes the following sub-steps: Step 241: Subtract the intraoperative image after correcting for metal artifacts from the intraoperative image with metal artifacts to obtain the difference image; Step 242: Based on experience, use a threshold of 0.7 to binarize the difference image and segment out the ablation needles; Step 243: Construct a three-dimensional point cloud from the segmented ablation needles and extract the skeleton from the point cloud; Step 244: Use a four-parameter point-direction expression to represent a straight line in three-dimensional space, and calculate the straight line where the point cloud is located through three-dimensional Hough transformation to locate the axis of the ablation needle; Step 245: Record all voxels passing through the axis on the difference image to obtain the voxel value variation curve along the axis, such as... Figure 7 As shown in the figure, this figure is a curve showing the change of voxel values along the axis of the differential image in the method of metal artifact correction and ablation visualization of CT images. Small breaks in the curve are skipped, and the endpoints of the ablation needle are determined by the distribution of voxel values. Step 246: Using a threshold T=154, locate the point where the change curve drops sharply, and pinpoint the endpoint of the ablation needle, such as... Figure 7 As shown; Step 247: Using the obtained axis and endpoints of the ablation needle, reconstruct the ablation needle in the corrected intraoperative image, such as... Figure 8 As shown in the figure, this is an example of ablation needle reconstruction on three slices in an intraoperative CT image, which matches the actual position of the ablation needle in a CT image of CT image metal artifact correction and ablation visualization method.
[0067] Step 248: The tumor is delineated preoperatively, and the relative positions of the tumor and the reconstructed electrode needles are displayed in three dimensions during the operation, such as... Figure 9 As shown.
[0068] The above-described method for metal artifact correction and ablation visualization in CT image-guided liver cancer ablation surgery addresses the problem of tumor obscuring caused by intraoperative metal artifacts. By using a metal artifact correction model to remove metal and artifacts from intraoperative CT images, it resolves the information asymmetry between preoperative and intraoperative CT images, reducing registration difficulty. Through registration of preoperative and intraoperative CT images, the tumor from the preoperative CT image is fused and displayed on the intraoperative CT image. By reconstructing the ablation needle and displaying it in three dimensions, the relative positions of the ablation needle and tumor are shown intuitively and effectively, improving the accuracy of intraoperative localization.
[0069] It should be noted that this embodiment uses CT-guided liver cancer ablation visualization as an example, but this application is not limited to this. In other embodiments of this application, it can also be used for tumor ablation visualization scenarios such as lung tumors, kidney tumors, and breast cancer tumors. The second embodiment of this application relates to a CT image metal artifact correction and ablation visualization system, the structure of which is as follows: Figure 10 As shown, this CT image metal artifact correction and ablation visualization system includes: The image and processing module is used to preprocess preoperative and intraoperative CT images; The metal artifact correction model building module is used to build a two-stage generative adversarial network to remove metal and artifacts from intraoperative CT images. The intraoperative tumor localization module is used to register preoperative images and intraoperative CT images after removing metal artifacts, and outputs intraoperative linked images of tumor fused from preoperative images into intraoperative images. The ablation needle reconstruction module is used to reconstruct the ablation needle in intraoperative CT images after removing metal artifacts, providing a three-dimensional display of the tumor and the ablation needle.
[0070] Preferably, the system is used for metal artifact correction and visualization of ablation therapy in CT-guided liver cancer ablation treatment, wherein the tumor is a liver tumor, and the image preprocessing module is further used for: Preoperative and intraoperative CT images were resampled to the same physical size. The resampled preoperative CT images and intraoperative CT images were rigidly registered to make their physical positions the same. The preoperative and intraoperative CT images after registration are cropped to the same size, and must include the liver tumor and the area where the ablation needle is located. The CT values of the cropped preoperative and intraoperative CT images were adjusted to display the liver and other tissues and normalized to [-1,1].
[0071] Preferably, the metal artifact correction model building module is further used for: Construct a two-stage generative adversarial network for metal artifact correction; Define the loss function; The model is optimized by iteratively minimizing the loss function.
[0072] Preferably, the construction of the two-stage generative adversarial network for metal artifact correction includes: A two-stage generator subnetwork is constructed, including a first-stage decoder-encoder network, which connects the decoder and encoder through a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network, which consists of a feature extraction layer, a residual module, and an image reconstruction module. A discriminator subnetwork is constructed, taking as input either a stitched image of the intraoperative image with artifacts and a preoperative image, or a stitched image of the intraoperative image with artifacts and a corrected intraoperative image, and outputting as a matrix. The discriminator subnetwork consists of 5 downsampled convolutional blocks.
[0073] Preferably, the loss function is defined as follows: Define adversarial loss, which is used to improve the generator's ability to remove metal artifacts and the discriminator's classification ability. To define the reconstruction loss, in order to preserve the intraoperative structure while removing artifacts, and to make the output image consistent with the preoperative image in the areas where metal and artifacts are located, and consistent with the intraoperative image in the areas where non-metal and artifacts are located, the L1 distance of the corresponding regions is calculated and added according to a certain weight to obtain the reconstruction loss. To define perceptual loss, in order to minimize the similarity between the output image and the artifact-free image in features such as texture and uniformity, the L1 distance between the feature maps of the corrected intraoperative CT image and the artifact-free preoperative CT image is calculated. Define the total loss by adding the adversarial loss, reconstruction loss, and perception loss according to certain weights.
[0074] Preferably, the optimization model includes: Combine preoperative CT images and intraoperative CT images into image pairs; The image pairs are input into a two-stage generative adversarial network, and the loss function is calculated. By selecting an optimizer, iterating a certain number of times, and minimizing the loss function, the optimized metal artifact correction model is obtained.
[0075] Preferably, the intraoperative tumor localization module is further used for: The intraoperative CT image with artifacts is input into the generator subnetwork of the optimized metal artifact correction model, and the corrected intraoperative CT image without artifacts is output. Non-rigid registration was performed on the preoperative CT images and the corrected intraoperative CT images to fuse the tumor in the preoperative CT images and display it on the intraoperative CT images.
[0076] Preferably, the ablation needle reconstruction module is further used for: The difference image is obtained by subtracting the intraoperative image with metal artifacts from the intraoperative image with metal artifacts. Binarize the difference image to segment out the ablation needles; The segmented ablation needles are constructed into a three-dimensional point cloud, and the skeleton of the point cloud is extracted. The axis of the ablation needle is located by calculating the line where the point cloud is located using 3D Hough line detection. Record all voxels that pass through the axis on the difference image to obtain the voxel value variation curve along the axis; Locate the point where the curve drops sharply to pinpoint the endpoint of the ablation needle; The ablation needle was reconstructed using the obtained axis and endpoints in the corrected intraoperative images; The tumor is delineated before surgery, and the relative positions of the tumor and the reconstructed electrode needles are displayed in three dimensions during surgery.
[0077] The CT image metal artifact correction and ablation visualization system described in the above embodiments has the following technical effects: 1. It can effectively help doctors clearly observe tumors during surgery. By registering preoperative CT images with intraoperative CT images, it solves the problem of tumors being obscured by metal artifacts. 2. It can intuitively and effectively display the puncture process of liver cancer ablation treatment, and display the relative position of the tumor and the ablation needle in three dimensions during the operation, thereby improving the accuracy of ablation treatment; 3. Effectively solves the problem of low registration accuracy caused by metal and artifacts. First, correct the metal and artifacts in the intraoperative CT image, and then perform registration. 4. A de-artifact model applicable to full-size 3D images is proposed, which effectively solves the problem of discontinuous image structure caused by using 2D and 3D image blocks as input, as well as the problem of image blurring caused by full-size image input in existing methods; 5. An artifact removal model was proposed, which not only considers the removal of local ablation needles, but also the removal of radial artifacts caused by ablation needles. 6. An artifact removal model was proposed to solve the problem of changes in the anatomical structure of intraoperative CT images during the removal of metal artifacts. By designing a weighted reconstruction loss, metal artifacts in intraoperative CT images are removed while ensuring that the anatomical structure of intraoperative CT images remains unchanged, and the position and shape information of intraoperative anatomical structures can be reflected in a true and effective manner. 7. A reconstruction process for ablation needles was proposed, which greatly reduced the amount of computation by extracting the skeleton and ensured the real-time performance of the calculation.
[0078] The first embodiment is a method embodiment corresponding to this embodiment. The technical details in the first embodiment can be applied to this embodiment, and the technical details in this embodiment can also be applied to the first embodiment.
[0079] It should be noted that those skilled in the art should understand that the functions of each module shown in the embodiments of the above-described CT image metal artifact correction and ablation visualization system can be understood with reference to the relevant descriptions of the aforementioned CT image metal artifact correction and ablation visualization methods. The functions of each module shown in the embodiments of the above-described CT image metal artifact correction and ablation visualization system can be implemented by a program (executable instructions) running on a processor, or by specific logic circuits. If the above-described CT image metal artifact correction and ablation visualization system is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.
[0080] Accordingly, this application also provides a computer storage medium storing computer-executable instructions, which, when executed by a processor, implement the various method implementations of this application.
[0081] Furthermore, this application also provides a CT image metal artifact correction and ablation visualization system, including a memory for storing computer-executable instructions and a processor; the processor is used to implement the steps in the above-described method embodiments when executing the computer-executable instructions in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The aforementioned memory may be read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or solid-state drive, etc. The steps of the methods disclosed in the various embodiments of this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
[0082] It should be noted that in this patent application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this patent application, if it refers to performing an action according to an element, it means performing the action at least according to that element, including two cases: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "repeatedly," and "various" include two, two times, two kinds, and more than two, more than two times, and more than two kinds.
[0083] All documents mentioned in this application are considered to be incorporated in their entirety into the disclosure of this application so that they can serve as a basis for modifications if necessary. Furthermore, it should be understood that after reading the foregoing disclosure of this application, those skilled in the art can make various alterations or modifications to this application, and these equivalent forms also fall within the scope of protection claimed in this application.
Claims
1. A method of CT image metal artifact correction and ablation visualization, characterized in that, include: Preprocessing of preoperative and intraoperative CT images; A two-stage generative adversarial network was constructed to remove metal and artifacts from the intraoperative CT images. The preoperative CT image and the intraoperative CT image after removing metal artifacts are registered, and the intraoperative linked image of the tumor in the preoperative CT image is output. Intraoperative CT images after removing metal artifacts are reconstructed to recreate the ablation needle, providing a three-dimensional display of the tumor and the ablation needle. The step of reconstructing the ablation needle from the intraoperative CT image after removing metal artifacts includes the following sub-steps: The difference image is obtained by subtracting the intraoperative CT image with metal artifacts from the intraoperative CT image after correction by the two-stage generative adversarial network. Binarize the difference image to segment out the ablation needles; The segmented ablation needles are constructed into a three-dimensional point cloud, and the skeleton of the point cloud is extracted. The axis of the ablation needle is located by calculating the line containing the point cloud using three-dimensional Hough line detection. Record all voxels on the difference image that pass through the axis to obtain the voxel value variation curve along the axis; Locate the point where the curve drops sharply and pinpoint the endpoint of the ablation needle; The ablation needle was reconstructed using the obtained axis and endpoints of the ablation needle in the corrected intraoperative CT images; The tumor is delineated before the operation, and the relative position of the tumor and the reconstructed ablation needle is displayed in three dimensions during the operation.
2. The method of claim 1, wherein, The method is used for metal artifact correction and visualization of CT-guided liver cancer ablation therapy, where the tumor is a liver tumor. The preprocessing steps for preoperative and intraoperative CT images include the following sub-steps: Preoperative and intraoperative CT images were resampled to the same physical size. The resampled preoperative CT images and intraoperative CT images were rigidly registered to make their physical positions the same. The registered preoperative CT images and intraoperative CT images are cropped to the same size, while including the area where the tumor and ablation needle are located. The CT values of the cropped preoperative and intraoperative CT images were adjusted to show the tumor and other tissues, and normalized to [-1,1].
3. The method of claim 1, wherein, The steps for constructing a two-stage generative adversarial network include the following sub-steps: Construct a two-stage generative adversarial network for metal artifact correction; Define the loss function; The model is optimized by iteratively minimizing the loss function.
4. The method as described in claim 3, characterized in that, The steps for constructing a two-stage generative adversarial network for metal artifact correction include the following sub-steps: A two-stage generator subnetwork is constructed, including a first-stage encoder-decoder network, which connects the encoder and decoder through a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network, which consists of a feature extraction layer, a residual module, and an image reconstruction module. A discriminator subnetwork is constructed. The input is a stitched image of the intraoperative image with artifacts and the preoperative image, or a stitched image of the intraoperative image with artifacts and the corrected intraoperative image. The output is a matrix. The discriminator subnetwork consists of 5 downsampled convolutional blocks.
5. The method as described in claim 3, characterized in that, The step of defining the loss function includes the following sub-steps: Define adversarial loss, which is used to improve the generator's ability to remove metal artifacts and the discriminator's classification ability. To define the reconstruction loss, in order to preserve the anatomical structure of the intraoperative image while removing artifacts, and to make the output image consistent with the preoperative image in the areas where metal and artifacts are located, and consistent with the intraoperative image in the areas where non-metal and artifacts are located, the L1 distance of the corresponding regions is calculated and added according to weights to obtain the reconstruction loss. To define perceptual loss, in order to minimize the similarity of the output image with the artifact-free image in terms of texture and uniformity features, the L1 distance between the feature maps of the corrected intraoperative CT image and the artifact-free preoperative CT image is calculated. Define the overall loss by adding the adversarial loss, reconstruction loss, and perception loss according to their weights.
6. The method as described in claim 3, characterized in that, The step of optimizing the model by iteratively minimizing the loss function includes the following sub-steps: Combine preoperative CT images and intraoperative CT images into image pairs; The image pairs are input into a two-stage generative adversarial network, and the loss function is calculated. By selecting an optimizer and iterating multiple times to minimize the loss function, an optimized metal artifact correction model is obtained.
7. The method as described in claim 1, characterized in that, The step of registering the preoperative CT image and the intraoperative CT image after removing metal artifacts, and outputting the intraoperative linked image of the tumor fused from the preoperative CT image to the intraoperative CT image, includes the following sub-steps: The intraoperative CT image with artifacts is input into the generator subnetwork of the optimized metal artifact correction model, and the corrected intraoperative CT image without artifacts is output. Non-rigid registration was performed on preoperative CT images containing clear tumors and corrected intraoperative CT images, so that the tumor in the preoperative CT images was fused and displayed on the intraoperative CT images.
8. A CT image metal artifact correction and ablation visualization system, characterized in that, include: The image preprocessing module is used to preprocess preoperative and intraoperative CT images; The metal artifact correction model building module is used to build a two-stage generative adversarial network to remove metal and artifacts from intraoperative CT images. The intraoperative tumor localization module is used to register preoperative images and intraoperative CT images after removing metal artifacts, and outputs intraoperative linked images of tumor fused from preoperative images into intraoperative images. The ablation needle reconstruction module is used to reconstruct the ablation needle in intraoperative CT images after removing metal artifacts, and to display the tumor and the ablation needle in three dimensions. The ablation needle reconstruction module is further used for: The difference image is obtained by subtracting the intraoperative CT image with metal artifacts from the intraoperative CT image after correction by the two-stage generative adversarial network. Binarize the difference image to segment out the ablation needles; The segmented ablation needles are constructed into a three-dimensional point cloud, and the skeleton of the point cloud is extracted. The axis of the ablation needle is located by calculating the line containing the point cloud using three-dimensional Hough line detection. Record all voxels on the difference image that pass through the axis to obtain the voxel value variation curve along the axis; Locate the point where the curve drops sharply and pinpoint the endpoint of the ablation needle; The ablation needle was reconstructed using the obtained axis and endpoints of the ablation needle in the corrected intraoperative CT images; The tumor is delineated before the operation, and the relative position of the tumor and the reconstructed ablation needle is displayed in three dimensions during the operation.
9. The system as described in claim 8, characterized in that, The system is used for metal artifact correction and visualization of ablation therapy in CT-guided liver cancer ablation treatment, wherein the tumor is a liver tumor, and the image preprocessing module is further used for: Preoperative and intraoperative CT images were resampled to the same physical size. The resampled preoperative CT images and intraoperative CT images were rigidly registered to make their physical positions the same. The preoperative and intraoperative CT images after registration are cropped to the same size, including both the liver tumor and the area where the ablation needle is located. The CT values of the cropped preoperative and intraoperative CT images were adjusted to display the liver and other tissues and normalized to [-1,1].
10. The system as described in claim 8, characterized in that, The metal artifact correction model building module is further used for: Construct a two-stage generative adversarial network for metal artifact correction; Define the loss function; The model is optimized by iteratively minimizing the loss function.
11. The system as claimed in claim 10, characterized in that, The construction of the two-stage generative adversarial network for metal artifact correction includes: A two-stage generator subnetwork is constructed, including a first-stage encoder-decoder network, which connects the encoder and decoder through a dilated spatial convolutional pooling pyramid module; and a second-stage image enhancement network, which consists of a feature extraction layer, a residual module, and an image reconstruction module. A discriminator subnetwork is constructed. The input is a stitched image of the intraoperative image with artifacts and the preoperative image, or a stitched image of the intraoperative image with artifacts and the corrected intraoperative image. The output is a matrix. The discriminator subnetwork consists of 5 downsampled convolutional blocks.
12. The system as claimed in claim 10, characterized in that, The defined loss function includes: Define adversarial loss, which is used to improve the generator's ability to remove metal artifacts and the discriminator's classification ability. To define the reconstruction loss, in order to preserve the intraoperative structure while removing artifacts, and to make the output image consistent with the preoperative image in the areas where metal and artifacts are located, and consistent with the intraoperative image in the areas where non-metal and artifacts are located, the L1 distance of the corresponding regions is calculated and added according to the weights to obtain the reconstruction loss. To define perceptual loss, in order to minimize the similarity of the output image with the artifact-free image in terms of texture and uniformity features, the L1 distance between the feature maps of the corrected intraoperative CT image and the artifact-free preoperative CT image is calculated. Define the overall loss by adding the adversarial loss, reconstruction loss, and perception loss according to their weights.
13. The system as described in claim 10, characterized in that, The optimization model includes: Combine preoperative CT images and intraoperative CT images into image pairs; The image pairs are input into a two-stage generative adversarial network, and the loss function is calculated. By selecting an optimizer and iterating multiple times to minimize the loss function, an optimized metal artifact correction model is obtained.
14. The system as described in claim 8, characterized in that, The intraoperative tumor localization module is further used for: The intraoperative CT image with artifacts is input into the generator subnetwork of the optimized metal artifact correction model, and the corrected intraoperative CT image without artifacts is output. Non-rigid registration was performed on the preoperative CT images and the corrected intraoperative CT images to fuse the tumor in the preoperative CT images onto the intraoperative CT images.
15. A CT image metal artifact correction and ablation visualization system, characterized in that, include: Memory is used to store executable instructions for a computer; as well as, A processor configured to implement the steps of the method as described in any one of claims 1 to 7 when executing the computer-executable instructions.
16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 7.
