Cone beam computed tomography image processing method, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the technical field of image processing, and disclose a cone-beam computed tomography image processing method, an electronic device and a storage medium, which comprise the following steps: obtaining a target CT image and a target CBCT (cone-beam computed tomography) image containing bone structures and soft tissues; using a pre-trained elastic registration network to perform elastic registration on the target CT image and the target CBCT image to obtain a vCT image; inputting the target CBCT image into a pre-trained image generation model to generate a pCT image; and fusing the pCT image and the vCT image to obtain an image processing result. The method can automatically optimize a non-rigid transformation from a CT image to a CBCT image, generate a pCT image according to a vCT image obtained through elastic registration and the target CBCT image, realize transfer learning of a CT image style under the premise of keeping the spatial structure of the CBCT image unchanged, complete a cross-modal image generation task, fuse the pCT image and the vCT image to obtain an image processing result with high fidelity, and solve the problem of improving the quality of a CBCT image.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, in particular to a cone beam computed tomography image processing method, an electronic device and a storage medium. BACKGROUND

[0002] In modern radiotherapy, CBCT (Cone Beam Computer Tomography) can reflect the current tissue structure of the patient and provide real-time positioning information, which is an indispensable image tool in the process of radiotherapy. However, CBCT images have large noise, low contrast and artifacts, which affect the recognition of anatomical structures and the accuracy of dose calculation. In contrast, CT (Computer Tomography) images have high resolution and accurate tissue density information, but their images are usually acquired before treatment, so there is a time lag and they cannot fully reflect the patient's anatomical state at the time of treatment.

[0003] To solve the problems of low CBCT image quality and CT image time lag, there are currently two technical approaches: one is to register historical CT images to the current CBCT images, and fuse the information of the two modalities through registration technology to realize high-quality, time-accurate images for clinical application. The second is to generate CT image style pseudo-CT (pCT) images based on CBCT images, and improve the CBCT image quality and gray consistency through deep learning or image conversion methods.

[0004] However, the existing methods still have obvious deficiencies: traditional registration or enhancement-based image processing methods require a large number of manual parameter adjustments, have low computational efficiency, and are difficult to apply quickly in clinical practice; while the existing deep learning generated pCT images are faster in generation speed, but still have limitations in detail fidelity, gray consistency and bone structure recovery, and are difficult to fully meet the requirements of high-precision positioning and dose calculation.

[0005] Therefore, there is an urgent need in the art for an image processing scheme that can improve the quality of CBCT images and thus improve the clinical usability of CBCT images. SUMMARY

[0006] The purpose of the present application is to at least provide a cone beam computed tomography image processing method, an electronic device and a storage medium, which can at least solve the problem of improving the quality of CBCT (Cone Beam Computer Tomography) images and thus improving the clinical usability of CBCT images, and at least achieve the effect of improving the clinical usability of CBCT images.

[0007] To solve the above technical problems, at least one embodiment of the present application provides a cone-beam computed tomography image processing method, comprising: acquiring a target CT image and a target CBCT image containing bone structures and soft tissues; performing elastic registration on the target CT image and the target CBCT image by using a pre-trained elastic registration network to obtain a vCT image; inputting the target CBCT image into a pre-trained attention generative adversarial network to generate a pCT image; and fusing the pCT image and the vCT image to obtain an image processing result.

[0008] At least one embodiment of the present application also provides a cone-beam computed tomography image processing device, comprising an image acquisition module, an elastic registration module, an image generation module and an image fusion module. The image acquisition module is configured to acquire a target CT image and a target CBCT image containing bone structures and soft tissues. The elastic registration module is configured to perform elastic registration on the target CT image and the target CBCT image by using a pre-trained elastic registration network to obtain a vCT image. The image generation module is configured to input the target CBCT image into a pre-trained image generation model to generate a pCT image. The image fusion module is configured to fuse the pCT image and the vCT image to obtain an image processing result.

[0009] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned cone-beam computed tomography image processing method.

[0010] At least one embodiment of the present application also provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the above-mentioned cone-beam computed tomography image processing method.

[0011] The cone beam computed tomography image processing method provided by the embodiment of the application realizes automatic optimization of the non-rigid transformation of the CT image to the CBCT image through an elastic registration network, realizes alignment of the CT image to the CBCT image, realizes generation of a pCT image based on the target CBCT image through an attention generative adversarial network, realizes transfer learning of the CT image style under the premise of keeping the spatial structure of the CBCT image unchanged, completes the cross-modal image generation task, and finally obtains an image processing result with high fidelity by fusing the pCT image and the vCT image. Under the premise of keeping the spatial resolution of the original CBCT image, the embodiment effectively fuses the fine structure information of the CT image through a deep learning method, thereby generating a CT style image with high fidelity from the CBCT image which is low in quality and susceptible to interference of artifacts and noise, to meet the requirements of clinical structure accuracy and gray consistency. The fused image can be applied to subsequent radiotherapy accurate positioning, dose calculation and clinical diagnosis, significantly improves the application effect of the prior art, and improves the clinical usability of the CBCT image.

[0012] In some optional embodiments, the elastic registration network comprises: a stitching channel configured to stitch a target CT image as a floating image and a target CBCT image as a fixed image; a deep neural network configured to obtain a deformation field representing an elastic registration relationship based on a stitching result; and an elastic transformation module configured to perform elastic transformation on the floating image based on the deformation field to obtain a vCT image.

[0013] In some optional embodiments, the cone beam computed tomography image processing method further comprises: training the elastic registration network through a first loss function in an unsupervised training manner; the first loss function is a weighted sum result of a mean square error loss, a smoothness loss, a first HU distribution loss and a mutual information loss; wherein the mean square error loss is a weighted sum result of a bone structure loss and a soft tissue loss, the bone structure loss is configured to constrain alignment of bone structures between the floating image and the fixed image, the soft tissue loss is configured to constrain alignment of soft tissues between the floating image and the fixed image, and the weight of the bone structure loss is greater than the weight of the soft tissue loss; the smoothness loss is configured to constrain continuity and smoothness of the deformation field; the HU distribution loss is configured to constrain matching degree of tissue densities between the floating image and the fixed image; and the mutual information loss is configured to constrain similarity between the floating image and the fixed image.

[0014] In some optional embodiments, the cone beam computed tomography image processing method further comprises: training the attention generative adversarial network for generating the pCT image by using the historical vCT images and the historical CBCT images through a second loss function training to obtain an image generation model, wherein an input of the image generation model comprises the CBCT image, and an output of the image generation model comprises the pCT image; the second loss function is a weighted sum result of an adversarial loss, a perception loss, a second HU distribution loss, and an edge loss, wherein the adversarial loss is used to constrain a matching degree between the generated pCT image and the vCT image, the perception loss is a weighted sum result of a first perception loss and a second perception loss, the first perception loss is used to constrain a consistency between the generated pCT image and the CBCT image at a feature level, the second perception loss is used to constrain a consistency between the generated pCT image and the vCT image at the feature level, a weight of the first perception loss is greater than a weight of the second perception loss, the second HU distribution loss is used to constrain a consistency of a gray histogram distribution between the generated pCT image and the vCT image, and the edge loss is used to constrain a reality of an organ boundary and a structure in the generated pCT image.

[0015] In some optional embodiments, a calculation formula of the adversarial loss is as follows:

[0016] In the formula, denotes the adversarial loss, denotes an expectation operation, x denotes the CBCT image, y denotes the vCT image, denotes a discrimination result of a discriminator on a real image pair, denotes the pCT image, denotes a discrimination result of the discriminator on a generated image pair, the real image pair is an image pair obtained by splicing the CBCT image and the vCT image, and the generated image pair is an image pair obtained by splicing the CBCT image and the pCT image.

[0017] In some optional embodiments, VGG network is used to extract features of each layer of the vCT image and the pCT image; and a calculation formula of the perception loss is as follows:

[0018]

[0019] In the formula, denotes the perception loss, x denotes the CBCT image, y denotes the vCT image, ϕ denotes a feature map of any intermediate layer in the VGG network, denotes the pCT image, denotes the first perception loss, This indicates the loss of second perception. and Let represent the weights of the first perceptual loss and the second perceptual loss, respectively. .

[0020] In some optional embodiments, acquiring a target CT image and a target CBCT image containing bony structures and soft tissue structures includes: acquiring an original CBCT image and an original CT image; rigidly registering the original CT image to the position of the original CBCT image; resampling the rigid registration result with reference to the voxel spacing of the original CBCT image; and extracting the region of interest from the resampling result and the original CBCT image to obtain the target CT image and the target CBCT image, wherein the region of interest includes bony structures and soft tissue.

[0021] In some optional embodiments, the pCT image and the vCT image are fused to obtain an image processing result, including: fusing the pCT image and the vCT image using a linear weighting principle to obtain an image processing result, wherein the weight of the pCT image is set to be greater the lower the gray value, and the weight of the vCT image is set to be greater the higher the gray value. Attached Figure Description

[0022] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0023] Figure 1 This is a flowchart of a cone-beam computed tomography (CBCT) image processing method provided in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram of the flexible registration network provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the generator of the attention generative adversarial network provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of a discriminator for an attention-based generative adversarial network provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of image fusion based on grayscale distribution provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of the execution flow of cone-beam computed tomography image processing provided in an embodiment of this application;

[0029] Figure 7This is a schematic diagram of a cone-beam computed tomography (CBCT) image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0031] To facilitate understanding of the embodiments of this application, relevant content regarding CBCT images will be introduced first.

[0032] In modern radiotherapy, CBCT images can reflect the patient's current tissue structure and provide real-time localization information, making them an indispensable imaging tool during radiotherapy. However, CBCT images suffer from interference such as high noise, low contrast, and artifacts, affecting the accuracy of anatomical structure identification and dose calculation. In contrast, CT images have high resolution and accurate tissue density information, but these images are usually acquired before treatment, resulting in a time lag and failing to fully reflect the patient's anatomical state at the moment of treatment.

[0033] To address the issues of low CBCT image quality and time lag in CT images, there are currently two main technical approaches: First, registering historical CT images to current CBCT images, fusing information from both modalities through registration techniques to achieve high-quality, time-accurate images for clinical applications. Second, generating pseudo-CT images with a CT image style based on CBCT images, using deep learning or image conversion methods to improve CBCT image quality and grayscale consistency for clinical use.

[0034] However, the methods in the relevant technologies still have significant shortcomings:

[0035] (1) The traditional CT→CBCT elastic registration (DIR, Deformation Image Registration) image processing method includes the following steps: First, the CT image and CBCT image are preprocessed, such as intensity normalization and artifact removal, to improve the stability of the registration; then, the deformation field is generated by the DIR algorithm based on the B-spline free deformation model to realize the non-rigid alignment of the CT image to the CBCT image, thereby characterizing the complex deformation of the anatomical structure; at the same time, the DVF (displacement vector field) is constrained by smoothing regularization to ensure the continuity and rationality of the deformation field and avoid deformation that does not conform to physiological laws.

[0036] However, traditional intensity-based registration methods rely on iterative optimization, which incurs significant computational overhead when processing three-dimensional volume data, making it difficult to meet the real-time requirements of online adaptive radiotherapy. Furthermore, they lack robustness to low-quality CBCT images; artifacts, low contrast, and significant anatomical distortions often cause registration errors. For example, respiratory motion artifacts can significantly reduce registration accuracy.

[0037] Furthermore, the algorithm's performance is highly dependent on the selection of regularization weights and similarity metrics. Different anatomical sites often require manual parameter tuning, resulting in low computational efficiency and making it difficult to apply quickly in clinical settings, further limiting its clinical application.

[0038] Overall, the main bottleneck of CT→CBCT flexible registration lies in the limited integration of the application process and the insufficient computational efficiency of the methods in related technologies, making it difficult to meet the real-time requirements of subsequent online adaptive radiotherapy.

[0039] (2) Methods for synthesizing pCT images from CBCT images based on deep learning are mainly divided into supervised and unsupervised categories. Supervised methods, represented by Pix2Pix, rely on pixel-level strictly paired CBCT–CT datasets. Under the adversarial training framework of generator and discriminator, pixel-level L1 loss and adversarial loss are combined to accurately map CBCT images to CT images. Unsupervised methods are mostly based on CycleGAN. Through the structure of dual generators (CBCT→CT and CT→CBCT) and dual discriminators, cycle consistency loss and adversarial loss are introduced to achieve cross-modal mapping without the need for strictly paired data.

[0040] However, while existing pCT images generated based on deep learning are faster in terms of generation speed, they still have limitations in terms of detail fidelity, grayscale consistency, and bone tissue structure restoration. They cannot fully meet the requirements of high-precision localization and dose calculation, and therefore cannot be directly used in clinical applications.

[0041] Despite the potential of deep learning methods in pCT image generation, several limitations remain. First, image quality is limited: CBCT images typically contain scattering artifacts and noise, leading to density deviations, blurred boundaries, and insufficient soft tissue contrast in generated pCT images, thus affecting the reliability of dose calculation and target delineation. Second, the algorithms lack generalization ability: related techniques are highly dependent on specific equipment or patient data, and performance tends to degrade across equipment, scan parameters, and anatomical sites, significantly affected by domain shift and data dependence. Furthermore, research has largely focused on the head, neck, and pelvis, with insufficient validation for higher-motion organs such as the chest.

[0042] At the methodological level, supervised learning relies on high-quality paired data, but such data is difficult to obtain in clinical practice due to limitations in organ motion, registration errors, and imaging costs. The presence of misaligned samples can easily lead to error accumulation and affect the fidelity of anatomical structures. While unsupervised methods can utilize unmatched data for training, they fall short in preserving anatomical details, often exhibiting learning bias, pattern collapse, and training instability. Cyclic consistency constraints can improve the mismatch problem to some extent, but the optimal solution is not unique and may still introduce anatomical shifts or distortions.

[0043] In summary, the technology of synthesizing pCT from CBCT based on deep learning has significant application potential in radiotherapy. However, this method only utilizes CBCT images and does not have effective information from reference CT images. Its development is still limited by the quality of the original CBCT images and faces bottlenecks such as insufficient accuracy and robustness of the synthesis results, limited algorithm generalization ability, and insufficient validation of high-motion areas, making it unable to be directly applied in clinical practice.

[0044] To address the aforementioned technical problem of improving CBCT image quality and thus enhancing its clinical usability, this invention proposes a cone-beam computed tomography (CBCT) image processing method. This method fuses effective information from CT and CBCT images to generate a final image that meets clinical needs. The implementation details of this CBCT image processing method are described below. These details are provided for ease of understanding and are not essential for implementing this solution.

[0045] Example 1:

[0046] The cone-beam computed tomography (CBCT) image processing method of this embodiment can be applied to electronic devices with communication, computing, and data storage capabilities. Its specific process can be as follows: Figure 1 As shown, it includes:

[0047] Step 101: Obtain the target CT image and target CBCT image containing bony structures and soft tissues.

[0048] In some cases, the target CT images and target CBCT images may include target CT images and target CBCT images of areas such as the head and neck, lungs, abdomen, pelvis, and breasts, which contain bony structures and soft tissues. The bony structures include bones, and the soft tissues include key soft tissues.

[0049] Step 102: Using a pre-trained elastic registration network, the target CT image and the target CBCT image are elastically registered to obtain a vCT image.

[0050] Specifically, the elastic registration network elastically registers the target CT image to the spatial location of the target CBCT image, achieving alignment between the CT image and the CBCT image.

[0051] Step 103: Input the target CBCT image into the pre-trained image generation model to generate a pCT image.

[0052] Specifically, by taking the target CBCT image as input and using a pre-trained image generation model to generate pCT images, it is possible to achieve transfer learning of CT image style while keeping the spatial structure of the CBCT image unchanged, thus completing the cross-modal image generation task.

[0053] Step 104: Fuse the pCT image with the vCT image to obtain the image processing result.

[0054] This embodiment utilizes an elastic registration network to automatically optimize the non-rigid transformation from CT images to CBCT images, achieving alignment between them. The target CBCT image is input into a pre-trained image generation model to generate a pCT image. This achieves CT image style transfer learning while preserving the spatial structure of the CBCT image, completing the cross-modal image generation task. Finally, the pCT image is fused with the vCT image to obtain a high-fidelity image processing result. This embodiment effectively fuses the fine structural information of the CT image using deep learning methods while maintaining the spatial resolution of the original CBCT image. This allows for the high-fidelity generation of CT-style images from low-quality, artifact-prone, and noise-affected CBCT images, meeting clinical requirements for structural accuracy and grayscale consistency. The resulting images can be used for subsequent radiotherapy precise positioning, dose calculation, and clinical diagnosis, significantly improving the application effect of existing technologies and enhancing the clinical usability of CBCT images.

[0055] In some embodiments, the cone-beam computed tomography (CBCT) image processing method described above, acquiring target CT images and target CBCT images containing bony structures and soft tissue structures, includes:

[0056] Step 1011: Obtain the original CBCT image and the original CT image.

[0057] Step 1012: Rigidly register the original CT image to the position of the original CBCT image.

[0058] Step 1013: Resample the rigid registration result using the voxel spacing of the original CBCT image as a reference.

[0059] Step 1014: Extract the region of interest from the resampling results and the original CBCT image to obtain the target CT image and the target CBCT image. The region of interest includes bony structures and soft tissues.

[0060] In this specific implementation, the original CBCT image and the original CT image undergo three preprocessing steps, including:

[0061] (1) Rigid registration (Transform): Rigidly register the CT image to the spatial position of the CBCT image to achieve consistency of the overall coordinate system of the image.

[0062] Specifically, during the training phase, a dataset is acquired, including original CBCT images and original CT images. Based on the rigid registration of the original CBCT images and original CT images in the dataset, a transformation matrix for matching between the CBCT images and CT images is obtained, which is used to perform rigid registration of the original CBCT images and original CT images in actual image processing.

[0063] (2) Resampling: Using the voxel spacing of the CBCT image as a reference, the CT image is linearly interpolated to achieve resampling, ensuring that the two sets of images are consistent in voxel resolution, and avoiding spatial scale differences from interfering with training, as well as subsequent elastic registration and cross-modal image generation.

[0064] (3) Mask generation: A uniform mask is generated by combining threshold segmentation and morphological processing to remove irrelevant background and retain only the region of interest containing bony structures and soft tissues as the effective region, thereby obtaining the target CBCT image containing the effective region (masked CBCT) and the target CT image containing the effective region (masked CT).

[0065] Specifically, threshold segmentation extracts the outer contours containing bones and soft tissues. On one hand, this effectively removes redundant parts from CT images. For example, due to the different scanning methods of CBCT and CT, the sizes of CBCT and CT images of the head and neck differ. CT images may capture redundant images of areas such as the shoulder and neck, which are irrelevant information in subsequent head and neck image processing. On the other hand, it removes irrelevant environmental factors such as the bed, avoiding interference. A uniform mask ensures that the regions of interest (effective regions) of the target CBCT and CT images are consistent.

[0066] It should be understood that the above preprocessing is performed in both the training and actual image processing stages. That is, in the stage of training the elastic registration network and image generation model using historical CT images and historical CBCT images, it is also necessary to perform the above-mentioned rigid registration, resampling and mask generation preprocessing on the historical CT images and historical CBCT images to obtain preprocessed CT images and CBCT images, and then train the elastic registration network and image generation model based on the preprocessed CT images and CBCT images.

[0067] The aforementioned integrated cross-modal preprocessing workflow ensures the consistency of input data in terms of spatial location, resolution, and effective area through rigid registration, resampling, and unified mask generation, thus avoiding the impact of background interference and spatial scale differences on modeling training or image processing.

[0068] In some embodiments, the above-described flexible registration network includes:

[0069] The stitching channel is used to stitch together a target CT image as a floating image with a target CBCT image as a fixed image.

[0070] Deep neural networks are used to obtain the deformation field representing the elastic registration relationship based on the stitching results;

[0071] The elastic transformation module is used to elastically transform a floating image to obtain a vCT image using a deformation field.

[0072] In the specific implementation, the preprocessed target CT image is used as input to the elastic registration network. The elastic registration network, trained in an unsupervised manner, aligns the target CT image with the target CBCT image. This elastic registration network learns the spatial deformation field (representing the spatial continuous deformation relationship from the target CT image to the target CBCT image) through a deep neural network (e.g., a 3D U-Net network). It does not rely on manually designed energy functions or parameter tuning processes, but automatically optimizes the non-rigid transformation from CT image to CBCT image to obtain the registered vCT image. In this process, the deep neural network pays particular attention to the registration of bony structures and soft tissues (region of interest / effective region), ensuring accurate matching of anatomical positions and reducing the high computational cost and manual intervention required by traditional optimization-based elastic registration.

[0073] The above-mentioned cone-beam computed tomography image processing method further includes:

[0074] Step 1001: Train the elastic registration network using an unsupervised training method with the first loss function;

[0075] The first loss function is a weighted sum of mean squared error loss, smoothness loss, first HU distribution loss, and mutual information loss. The mean squared error loss is a weighted sum of skeletal structure loss and soft tissue loss. The skeletal structure loss constrains the alignment of skeletal structures between the floating and fixed images, while the soft tissue loss constrains the alignment of soft tissues between the floating and fixed images. The weight of the skeletal structure loss is greater than that of the soft tissue loss. The smoothness loss constrains the continuity and smoothness of the deformation field. The HU distribution loss constrains the matching degree of tissue density between the floating and fixed images. The mutual information loss constrains the similarity between the floating and fixed images.

[0076] This embodiment employs an unsupervised registration network (ElasReg, ElasticRegistration) with 3D U-Net as its core structure. It directly learns the elastic transformation relationship between CT images and CBCT images, without relying on manual parameter tuning or traditional energy function design. This network combines bony structure loss, soft tissue loss, smoothness loss, first HU distribution loss, and mutual information loss to achieve accurate alignment of bony structures and soft tissue regions under cross-modal conditions.

[0077] In some examples, such as Figure 2 As shown, the specific design of the flexible registration network is as follows:

[0078] (1) Network structure: A 3D U-Net structure is used as the registration network. The input of the 3D U-Net is the stitched channel output of the floating image (CT image) and the fixed image (CBCT image), and the output of the 3D U-Net is a three-dimensional deformation field (flowfield). This deformation field is applied to the floating image through three-dimensional mesh sampling and bilinear interpolation to obtain the elastically transformed CT image (denoted as vCT image). While the CT image and CBCT image are input into the 3D U-Net, threshold segmentation based on the fixed image is also performed to extract the bony tissue, and the rest is soft tissue.

[0079] (2) Loss function design: The loss function takes into account multiple factors:

[0080] ① MSE (mean-square error) constraint

[0081] Bony structure constraint: Based on the bony tissue regions extracted using a fixed image threshold, a mean square error constraint is introduced to the bony regions to ensure accurate alignment of the bones; Soft tissue constraint: Weak constraints are applied to non-bone structures, also using mean square error, to ensure overall alignment of the soft tissue regions; Since the elastic registration network focuses more on the features of bony structures, when weighting the sum of bony structure loss and soft tissue loss, the weight of the bony structure loss is set to be greater than the weight of the soft tissue loss.

[0082] ② Flow field smoothness constraint: The continuity and smoothness of the deformation field are constrained by the gradient penalty term to avoid non-physical deformation.

[0083] ③HU Distribution Loss: Based on the histogram differences in soft tissue regions, it constrains the distribution of HU values ​​and improves the matching degree of tissue density between CT images and CBCT images.

[0084] ④ Mutual Information Loss: Used to constrain the similarity between floating and fixed images to enhance the statistical consistency of grayscale distribution between cross-modal CT images and CBCT images.

[0085] (3) Training method: Unsupervised training method is adopted, that is, there is no need to use the actual deformation field of the pair as the supervision signal. Instead, the elastic registration network is guided to adaptively learn the optimal elastic registration relationship through the above loss function and represented by the deformation field.

[0086] In some embodiments, the cone-beam computed tomography (CBCT) image processing method described above further includes:

[0087] Step 1002: Using the second loss function, an attention-generative adversarial network (Attention-GAN) is trained to generate pCT images using historical vCT images and historical CBCT images, resulting in an image generation model. The input of the image generation model includes CBCT images, and the output includes pCT images.

[0088] The second loss function is a weighted sum of adversarial loss, perceptual loss, second HU distribution loss, and edge loss. The adversarial loss is used to constrain the matching degree between the generated pCT image and the vCT image. The perceptual loss is a weighted sum of the first perceptual loss and the second perceptual loss. The first perceptual loss is used to constrain the consistency between the generated pCT image and the CBCT image at the feature level. The second perceptual loss is used to constrain the consistency between the generated pCT image and the vCT image at the feature level. The weight of the first perceptual loss is greater than the weight of the second perceptual loss. The second HU distribution loss is used to constrain the consistency of the gray-level histogram distribution between the generated pCT image and the vCT image. The edge loss is used to constrain the authenticity of organ boundaries and structures in the generated pCT image.

[0089] In the specific implementation, for the cross-modal image generation task from CBCT images to CT images, an Attention-GAN model based on conditional GAN ​​is introduced. The preprocessed target CBCT image is used as input, and a pre-trained image generation model is used to generate pCT images. This model focuses on bony structures and key tissue regions through an attention mechanism, learning and transferring the image style of CT while maintaining the spatial structure of the CBCT image, thereby generating pCT images.

[0090] In some examples, such as Figure 3 and Figure 4 As shown, the key design of attention-based generative adversarial networks is as follows:

[0091] (1) Model structure

[0092] ① Generator: Employs a U-Net-style encoder-decoder architecture, consisting of five downsampling layers and five upsampling layers, with skip connections between corresponding layers to pass multi-scale features. The generator embeds a self-attention module in the deepest layer to calculate the global spatial dependencies of the feature maps. It obtains attention weights through a Query / Key / Value mechanism and Softmax, thereby enhancing the representation of the Region of Interest (ROI) and suppressing irrelevant background. The final output is activated by Tanh to generate a predicted CT image (pCT image).

[0093] ② Discriminator: The PatchGAN structure is adopted, which concatenates the input CBCT image and vCT image (as an image pair) as a conditional input. Through multi-layer convolution and LeakyReLU, the feature matrix containing multiple local patch features is output, which improves the ability to distinguish details and structural consistency.

[0094] (2) Loss function design:

[0095] ① Adversarial Loss (GAN Loss): The MSE loss form of LSGAN (Least Squares GAN) is adopted to ensure that the discriminator judges the generated image to be close to the real CT image.

[0096] In some embodiments, the adversarial loss is calculated as follows:

[0097] In the formula, Indicating resistance to loss, This represents the expectation operation. x Represents CBCT images, y Represents vCT images, This represents the discriminator's judgment result on the real image pairs. Represents pCT images, This indicates the discriminator's judgment result on the generated image pair. The real image pair is an image pair formed by stitching together a CBCT image and a vCT image, while the generated image pair is an image pair formed by stitching together a CBCT image and a pCT image.

[0098] ②Perceptual Loss:

[0099] In some embodiments, features of each layer of the vCT image and the generated pCT image are extracted based on the VGG network; the calculation formula for the perceptual loss is as follows;

[0100]

[0101] In the formula, Indicates perceived loss. x Represents CBCT images, y Represents vCT images, ϕ (•) represents the feature map of any intermediate layer in the VGG network. Represents pCT images, Indicates the loss of first perception. This indicates the loss of second perception. and Let represent the weights of the first perceptual loss and the second perceptual loss, respectively. , ||1 represents the L1 norm, which is used to measure differences at the feature level.

[0102] In some examples, based on VGG16 (Visual Geometry Group 16) feature extraction, the above dual perceptual constraints were designed: a large proportion maintains the consistency with the structure of the input CBCT image, supplemented by a small proportion to approximate the modality of the real CT image. In the calculation formula of perceptual loss, a=8 and b=2.

[0103] ③HU Distribution Loss: By comparing the grayscale histogram distribution of the generated CT (pCT image) and the real CT (vCT) image, the HU value is made to move closer to the CT style.

[0104]

[0105] In the formula, p i ( G ( x ))and p i ( y ) represent the pCT image and vCT image at the 1st and 2nd respectively. i The normalized frequencies over a grayscale range, where N represents the total number of ranges in the grayscale histogram.

[0106] ④ Edge Loss: Gradient information is extracted using the Sobel operator, and L1 loss is used to enhance the organ boundaries and detailed structures of the generated CT (pCT image).

[0107]

[0108] In the formula, ▽ represents the Sobel gradient operator, and ▽(G(x)) and ▽(y) represent the gradient map of the generated CT and the gradient map of the vCT image, respectively.

[0109] The overall loss function is a weighted sum of the above loss functions. Through the collaborative optimization of multiple constraints, the generated CT (pCT image) is close to the real CT in terms of grayscale distribution, detail restoration and structural consistency.

[0110]

[0111] In the formula, This represents the second loss function. , , , These represent the weights of each loss function.

[0112] The Attention-GAN generator incorporates a self-attention mechanism, enabling it to automatically capture global dependencies between bony structures and key soft tissues, thus completing cross-modal generation tasks. It achieves style transfer of CT images while preserving the spatial structure of CBCT images. This model utilizes a combination of adversarial loss, perceptual loss, HU distribution loss, and edge loss to ensure grayscale consistency, detail clarity, and structural fidelity in the generated images.

[0113] In some embodiments, pCT images are fused with vCT images to obtain image processing results, including:

[0114] A linear weighting principle is used to fuse pCT and vCT images to obtain the image processing result. The pCT image is weighted such that lower grayscale values ​​have higher weights, while the vCT image is weighted such that higher grayscale values ​​have higher weights. By synthesizing the pCT and vCT images at the pixel level, the final image result is obtained, achieving an organic unity of structure fidelity and style transfer.

[0115] In some examples, such as Figure 5 As shown, the image fusion strategy based on grayscale distribution fuses pCT images and vCT images. That is, between the predicted CT (pCT) and the elastic registration result (vCT), a differential weighting principle is adopted: pCT enhances dark areas (areas with low grayscale values) to highlight soft tissues, while vCT enhances bright areas (areas with high grayscale values) to highlight bony structures. This fusion strategy achieves complementary enhancement of soft tissues and bony structures, ensuring that the final generated image (Final CT) has high fidelity in both soft tissue details and bony anatomical structures.

[0116] The pCT images generated by the Attention-GAN described above can preserve the structure of CBCT images, but areas with significant artifacts may become blurred. To improve the clarity of the final result, pCT and vCT images are fused, making key parts such as bones more prominent. The fused result (Final CT) has the style of a real CT image and a structure consistent with the CBCT image, preserving both the real-time performance and anatomical accuracy of CBCT images and the quantitative reliability of CT images. It can be directly applied to radiotherapy planning, dosimetry calculation, image guidance, and clinical diagnosis. The image processing workflow in this embodiment achieves automated, end-to-end cross-modal image generation and optimization, significantly reducing manual intervention and parameter adjustments, and improving processing efficiency and result stability.

[0117] In some examples, elastic registration networks and attention-based generative adversarial networks are trained using CT and CBCT image data from multiple body sites, including the head and neck, lungs, abdomen, pelvis, and breast (each site involves CT and CBCT images from multiple patients). The execution flow of cone-beam computed tomography (CBCT) image processing is as follows: Figure 6 As shown:

[0118] (1) Perform three-step preprocessing on the original CBCT and CT to obtain masked CBCT (target CBCT image) and masked CT (target CT image):

[0119] transform: Use the transformation matrix in the dataset to rigidly register the CT to the CBCT position;

[0120] resample: Referencing the spacing of CBCT, perform linear interpolation resampling on CT.

[0121] mask: Create a uniform mask through quick selection and other methods to remove irrelevant backgrounds.

[0122] (2) Based on the above preprocessing results, an unsupervised elastic registration network is used to perform elastic registration from CT to CBCT, mainly focusing on the alignment of bony structures and effective soft tissue regions, to obtain the registered result vCT;

[0123] (3) The masked CBCT is fed into Attention GAN (Attention GAN, Attention Generative Adversarial Network) to calculate and generate pCT. Attention GAN achieves the learning of CT style while preserving the CBCT structure.

[0124] (4) Fusion of pCT and vCT, mainly taking the dark part (soft tissue) of pCT and the bright part (bone structure) of vCT to obtain Final CT.

[0125] The method in this embodiment aims to improve the clinical usability of CBCT images while fully utilizing high-quality anatomical information from historical CT scans. It proposes a high-quality image generation scheme based on CBCT and CT scans. Through deep learning, it achieves intelligent denoising and contrast enhancement while maintaining the original CBCT spatial resolution, and effectively integrates fine structural information from CT scans to generate high-quality images for precise radiotherapy localization, dose calculation, and clinical diagnosis, significantly improving the application effects of existing technologies. The beneficial effects of this method are mainly reflected in the following aspects:

[0126] (1) The generated images show a significant improvement in structural detail fidelity, with clear distinction of bony structures, soft tissue shapes and edges, while background noise and artifacts are significantly reduced. The quantitative indicators MAE (Mean Absolute Error), PSNR (Peak signal-to-noise ratio), and SSIM (structural similarity index) all perform well.

[0127] (2) Multiple loss constraints ensure the consistency of grayscale, texture and structure, making the generated images close to real CT in grayscale distribution and HU value, thus improving the usability of cross-modal images.

[0128] (3) Even with small sample size and multiple site training, it can maintain strong generalization ability, adapt to different anatomical sites and individual patient differences, focus on the core features and key structures of the image, and has high fidelity.

[0129] (4) The training and inference difficulty is moderate, and the efficiency is significantly improved compared with traditional algorithms. At the same time, compared with other deep learning methods, the computational cost is much lower than that of the Diffusion method, while the generation effect is better than that of Cycle-GAN and Pix2Pix methods.

[0130] Example 2:

[0131] Another embodiment of this application relates to a cone-beam computed tomography (CBCT) image processing apparatus. The implementation details of this embodiment's CBCT image processing apparatus are described below. The following details are provided for ease of understanding and are not essential for implementing this solution. A schematic diagram of this embodiment's CBCT image processing apparatus can be seen as follows: Figure 7 As shown, it includes an image acquisition module 201, an elastic registration module 202, an image generation module 203, and an image fusion module 204.

[0132] Image acquisition module 201 is used to acquire target CT images and target CBCT images containing bony structures and soft tissues. In some examples, the target CT images and target CBCT images may include target CT images and target CBCT images of areas such as the head and neck, lungs, abdomen, pelvis, and breast, which contain bony structures and soft tissues. The bony structures include bones, and the soft tissues include key soft tissues.

[0133] The elastic registration module 202 is used to elastically register a target CT image with a target CBCT image using a pre-trained elastic registration network to obtain a vCT image. Specifically, the elastic registration network elastically registers the target CT image to the spatial position of the target CBCT image, thus achieving alignment between the CT image and the CBCT image.

[0134] The image generation module 203 is used to input the target CBCT image into the pre-trained image generation model to generate pCT images, thereby achieving transfer learning of CT image style while keeping the spatial structure of the CBCT image unchanged, and completing the cross-modal image generation task.

[0135] The image fusion module 204 is used to fuse pCT images and vCT images to obtain image processing results.

[0136] This embodiment utilizes an elastic registration network to automatically optimize the non-rigid transformation from CT images to CBCT images, achieving alignment between them. The target CBCT image is input into a pre-trained image generation model to generate a pCT image. This achieves CT image style transfer learning while preserving the spatial structure of the CBCT image, completing the cross-modal image generation task. Finally, the pCT image is fused with the vCT image to obtain a high-fidelity image processing result. This embodiment effectively fuses the fine structural information of the CT image using deep learning methods while maintaining the spatial resolution of the original CBCT image. This allows for the high-fidelity generation of CT-style images from low-quality, artifact-prone, and noise-affected CBCT images, meeting clinical requirements for structural accuracy and grayscale consistency. The resulting images can be used for subsequent radiotherapy precise positioning, dose calculation, and clinical diagnosis, significantly improving the application effect of existing technologies and enhancing the clinical usability of CBCT images.

[0137] In some embodiments, acquiring a target CT image and a target CBCT image containing bony structures and soft tissue structures includes: acquiring an original CBCT image and an original CT image; rigidly registering the original CT image to the position of the original CBCT image; resampling the rigid registration result with reference to the voxel spacing of the original CBCT image; and extracting the region of interest from the resampling result and the original CBCT image to obtain the target CT image and the target CBCT image, wherein the region of interest includes bony structures and soft tissue.

[0138] In this specific implementation, the original CBCT image and the original CT image undergo three preprocessing steps, including:

[0139] (1) Rigid registration (Transform): Rigidly register the CT image to the spatial position of the CBCT image to achieve consistency of the overall image coordinate system.

[0140] Specifically, during the training phase, a dataset is acquired, including original CBCT images and original CT images. Based on the rigid registration of the original CBCT images and original CT images in the dataset, a transformation matrix for matching between the CBCT images and CT images is obtained, which is used to perform rigid registration of the original CBCT images and original CT images in actual image processing.

[0141] (2) Resampling: Using the voxel spacing of the CBCT image as a reference, the CT image is linearly interpolated to achieve resampling, ensuring that the two sets of images are consistent in voxel resolution, and avoiding spatial scale differences from interfering with training, as well as subsequent elastic registration and cross-modal image generation.

[0142] (3) Mask generation: A uniform mask is generated by combining threshold segmentation and morphological processing to remove irrelevant background and retain only the region of interest containing bony structures and soft tissues as the effective region, thereby obtaining the target CBCT image containing the effective region (masked CBCT) and the target CT image containing the effective region (masked CT).

[0143] Specifically, threshold segmentation extracts the outer contours containing bones and soft tissues. On one hand, this effectively removes redundant parts from CT images. For example, due to the different scanning methods of CBCT and CT, the sizes of CBCT and CT images of the head and neck differ. CT images may capture redundant images of areas such as the shoulder and neck, which are irrelevant information in subsequent head and neck image processing. On the other hand, it removes irrelevant environmental factors such as the bed, avoiding interference. A uniform mask ensures that the region of interest (effective region) of the target CBCT and CT images are consistent.

[0144] It should be understood that the above preprocessing is performed in both the training and actual image processing stages.

[0145] The aforementioned integrated cross-modal preprocessing workflow ensures the consistency of input data in terms of spatial location, resolution, and effective area through rigid registration, resampling, and unified mask generation, thus avoiding the impact of background interference and spatial scale differences on modeling training or image processing.

[0146] In some embodiments, the above-described flexible registration network includes:

[0147] The stitching channel is used to stitch together a target CT image as a floating image with a target CBCT image as a fixed image.

[0148] Deep neural networks are used to obtain the deformation field representing the elastic registration relationship based on the stitching results;

[0149] The elastic transformation module is used to elastically transform a floating image to obtain a vCT image using a deformation field.

[0150] In the specific implementation, the preprocessed target CT image is used as input to the elastic registration network. The elastic registration network, trained in an unsupervised manner, aligns the target CT image with the target CBCT image. This elastic registration network learns the spatial deformation field (representing the spatial continuous deformation relationship from the target CT image to the target CBCT image) through a deep neural network (e.g., a 3D U-Net network). It does not rely on manually designed energy functions or parameter tuning processes, but automatically optimizes the non-rigid transformation from CT image to CBCT image to obtain the registered vCT image. In this process, the deep neural network pays particular attention to the registration of bony structures and soft tissues (region of interest / effective region), ensuring accurate matching of anatomical positions and reducing the high computational cost and manual intervention required by traditional optimization-based elastic registration.

[0151] In the aforementioned cone-beam computed tomography image processing apparatus, the elastic registration module 202 is further used to train the elastic registration network using an unsupervised training method through a first loss function.

[0152] The first loss function is a weighted sum of mean squared error loss, smoothness loss, first HU distribution loss, and mutual information loss. The mean squared error loss is a weighted sum of skeletal structure loss and soft tissue loss. The skeletal structure loss constrains the alignment of skeletal structures between the floating and fixed images, while the soft tissue loss constrains the alignment of soft tissues between the floating and fixed images. The weight of the skeletal structure loss is greater than that of the soft tissue loss. The smoothness loss constrains the continuity and smoothness of the deformation field. The first HU distribution loss constrains the matching degree of tissue density between the floating and fixed images. The mutual information loss constrains the similarity between the floating and fixed images.

[0153] This embodiment employs an unsupervised registration network (ElasReg, ElasticRegistration) with 3D U-Net as its core structure. It directly learns the elastic transformation relationship between CT images and CBCT images, without relying on manual parameter tuning or traditional energy function design. This network combines bony structure loss, soft tissue loss, smoothness loss, first HU distribution loss, and mutual information loss to achieve accurate alignment of bony structures and soft tissue regions under cross-modal conditions.

[0154] In some examples, such as Figure 2 As shown, the specific design of the flexible registration network is as follows:

[0155] (1) Network structure: A 3D U-Net structure is used as the registration network. The input of the 3D U-Net is the stitched channel output of the floating image (CT image) and the fixed image (CBCT image), and the output of the 3D U-Net is a three-dimensional deformation field (flowfield). This deformation field is applied to the floating image through three-dimensional mesh sampling and bilinear interpolation to obtain the elastically transformed CT image (denoted as vCT image). While the CT image and CBCT image are input into the 3D U-Net, threshold segmentation based on the fixed image is also performed to extract the bony tissue, and the rest is soft tissue.

[0156] (2) Loss function design: The loss function takes into account multiple factors:

[0157] ① MSE (mean-square error) constraint

[0158] Bony structure constraint: Based on the bony tissue regions extracted using a fixed image threshold, a mean square error constraint is introduced to the bony regions to ensure accurate alignment of the bones; Soft tissue constraint: Weak constraints are applied to non-bone structures, also using mean square error, to ensure overall alignment of the soft tissue regions; Since the elastic registration network focuses more on the features of bony structures, when weighting the sum of bony structure loss and soft tissue loss, the weight of the bony structure loss is set to be greater than the weight of the soft tissue loss.

[0159] ② Flow field smoothness constraint: The continuity and smoothness of the deformation field are constrained by the gradient penalty term to avoid non-physical deformation.

[0160] ③HU Distribution Loss: Based on the histogram differences in soft tissue regions, it constrains the distribution of HU values ​​and improves the matching degree of tissue density between CT images and CBCT images.

[0161] ④ Mutual Information Loss: Used to enhance the statistical consistency of grayscale distribution in cross-modal CT images / CBCT images.

[0162] (3) Training method: Unsupervised training method is adopted, that is, there is no need to use the actual deformation field of the pair as the supervision signal. Instead, the elastic registration network is guided to adaptively learn the optimal elastic registration relationship through the above loss function and represented by the deformation field.

[0163] In some embodiments, the image generation module 203 is further configured to: train an attention-based generative adversarial network for generating pCT images using historical vCT images and historical CBCT images to obtain an image generation model, wherein the input of the image generation model includes CBCT images and the output includes pCT images.

[0164] The second loss function is a weighted sum of adversarial loss, perceptual loss, second HU distribution loss, and edge loss. The adversarial loss is used to constrain the matching degree between the generated pCT image and the vCT image. The perceptual loss is a weighted sum of the first perceptual loss and the second perceptual loss. The first perceptual loss is used to constrain the consistency between the generated pCT image and the CBCT image at the feature level. The second perceptual loss is used to constrain the consistency between the generated pCT image and the vCT image at the feature level. The weight of the first perceptual loss is greater than the weight of the second perceptual loss. The second HU distribution loss is used to constrain the consistency of the gray-level histogram distribution between the generated pCT image and the vCT image. The edge loss is used to constrain the authenticity of organ boundaries and structures in the generated pCT image.

[0165] In the specific implementation, for the cross-modal image generation task from CBCT images to CT images, an Attention-GAN model based on conditional GAN ​​is introduced. The preprocessed target CBCT image is used as input, and a pre-trained image generation model is used to generate pCT images. This model focuses on bony structures and key tissue regions through an attention mechanism, learning and transferring the image style of CT while maintaining the spatial structure of the CBCT image, thereby generating pCT images.

[0166] In some examples, such as Figure 3 and Figure 4 As shown, the key design of attention-based generative adversarial networks is as follows:

[0167] (1) Model structure

[0168] ① Generator: Employs a U-Net-style encoder-decoder architecture, consisting of five downsampling layers and five upsampling layers, with skip connections between corresponding layers to pass multi-scale features. The generator embeds a self-attention module in the deepest layer to calculate the global spatial dependencies of the feature maps. It obtains attention weights through a Query / Key / Value mechanism and Softmax, thereby enhancing the representation of the Region of Interest (ROI) and suppressing irrelevant background. The final output is activated by Tanh to generate a predicted CT image (pCT image).

[0169] ② Discriminator: The PatchGAN structure is adopted, which concatenates the input CBCT image and vCT image (as an image pair) as a conditional input. Through multi-layer convolution and LeakyReLU, the feature matrix containing multiple local patch features is output, which improves the ability to distinguish details and structural consistency.

[0170] (2) Loss function design:

[0171] ① Adversarial Loss (GAN Loss): The MSE loss form of LSGAN (Least Squares GAN) is adopted to ensure that the discriminator judges the generated image to be close to the real CT image.

[0172] In some embodiments, the adversarial loss is calculated as follows:

[0173] In the formula, Indicating resistance to loss, This represents the expectation operation. x Represents CBCT images, y Represents vCT images, This represents the discriminator's judgment result on the real image pairs. Represents pCT images, This indicates the discriminator's judgment result on the generated image pair. The real image pair is an image pair formed by stitching together a CBCT image and a vCT image, while the generated image pair is an image pair formed by stitching together a CBCT image and a pCT image.

[0174] ②Perceptual Loss:

[0175] In some embodiments, features of each layer of vCT and pCT images are extracted based on the VGG network; the calculation formula for perceptual loss is as follows;

[0176]

[0177] In the formula, Indicates perceived loss. x Represents CBCT images, y Represents vCT images, ϕ (•) represents the feature map of any intermediate layer in the VGG network. Represents pCT images, Indicates the loss of first perception. This indicates the loss of second perception. and Let represent the weights of the first perceptual loss and the second perceptual loss, respectively. , ||1 represents the L1 norm, which is used to measure differences at the feature level.

[0178] In some examples, based on VGG16 (Visual Geometry Group 16) feature extraction, the above dual perceptual constraints were designed: a large proportion maintains the consistency with the structure of the input CBCT image, supplemented by a small proportion to approximate the modality of the real CT image, a=8, b=2.

[0179] ③HU Distribution Loss: By comparing the grayscale histogram distribution of the generated CT (pCT image) and the real CT (vCT) image, the HU value is made to move closer to the CT style.

[0180]

[0181] In the formula, p i ( G ( x ))and p i ( y ) represent the pCT image and vCT image at the 1st and 2nd respectively. i The normalized frequencies over a grayscale range, where N represents the total number of ranges in the grayscale histogram.

[0182] ④ Edge Loss: Gradient information is extracted using the Sobel operator, and L1 loss is used to enhance the organ boundaries and detailed structures of the generated CT (pCT image).

[0183]

[0184] In the formula, ▽ represents the Sobel gradient operator, and ▽(G(x)) and ▽(y) represent the gradient map of the generated CT and the gradient map of the vCT image, respectively.

[0185] The overall loss function is a weighted sum of the above loss functions. Through the collaborative optimization of multiple constraints, the generated CT (pCT image) is close to the real CT in terms of grayscale distribution, detail restoration and structural consistency.

[0186]

[0187] In the formula, This represents the second loss function. , , , These represent the weights of each loss function.

[0188] The Attention-GAN generator incorporates a self-attention mechanism, enabling it to automatically capture global dependencies between bony structures and key soft tissues, thus completing cross-modal generation tasks. It achieves style transfer of CT images while preserving the spatial structure of CBCT images. This model utilizes a combination of adversarial loss, perceptual loss, HU distribution loss, and edge loss to ensure grayscale consistency, detail clarity, and structural fidelity in the generated images.

[0189] In some embodiments, fusing pCT images and vCT images to obtain an image processing result includes: fusing pCT images and vCT images using a linear weighting principle, wherein the weight of the pCT image is set to increase as the grayscale value decreases, and the weight of the vCT image is set to increase as the grayscale value increases. By synthesizing pCT images and vCT images at the pixel level, the final image result is obtained, achieving an organic unity of structure fidelity and style transfer.

[0190] In some examples, such as Figure 5 As shown, the image fusion strategy based on grayscale distribution fuses pCT and vCT images. Specifically, a differential weighting principle is adopted between the predicted CT (pCT) and the elastic registration result (vCT): pCT enhances dark areas (areas with low grayscale values) to highlight soft tissues, while vCT enhances bright areas (areas with high grayscale values) to highlight bony structures. This fusion strategy achieves complementary enhancement of soft tissues and bony structures, ensuring that the final generated image (Final CT) has high fidelity in both soft tissue details and bony anatomical structures.

[0191] The pCT images generated by the Attention-GAN described above can preserve the structure of CBCT images, but areas with significant artifacts may become blurred. To improve the clarity of the final result, pCT and vCT images are fused, making key parts such as bones more prominent. The fused result (Final CT) has the style of a real CT image and a structure consistent with the CBCT image, preserving both the real-time performance and anatomical accuracy of CBCT images and the quantitative reliability of CT images. It can be directly applied to radiotherapy planning, dosimetry calculation, image guidance, and clinical diagnosis. The image processing workflow in this embodiment achieves automated, end-to-end cross-modal image generation and optimization, significantly reducing manual intervention and parameter adjustments, and improving processing efficiency and result stability.

[0192] This embodiment possesses at least all the beneficial effects of Embodiment 1, which will not be repeated here.

[0193] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0194] Example 3:

[0195] Another embodiment of this application relates to an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the cone-beam computed tomography image processing method of the above embodiments.

[0196] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0197] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0198] Example 4:

[0199] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0200] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0201] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A cone-beam computed tomography (CBCT) image processing method, characterized in that, include: Acquire target CT and target CBCT images containing bony structures and soft tissues; Using a pre-trained elastic registration network, the target CT image and the target CBCT image are elastically registered to obtain the vCT image; The target CBCT image is input into a pre-trained image generation model to generate a pCT image; The pCT image and the vCT image are fused to obtain the image processing result; The elastic registration network includes: The stitching channel is used to stitch together a target CT image as a floating image with a target CBCT image as a fixed image. A deep neural network is used to obtain a spatial deformation field representing the elastic registration relationship based on the stitching results. The spatial deformation field represents the spatial continuous deformation relationship from the target CT image to the target CBCT image. An elastic transformation module is used to elastically transform the floating image using the spatial deformation field to obtain a vCT image; The cone-beam computed tomography (CBCT) image processing method further includes: The elastic registration network is trained using an unsupervised training method with a first loss function. The first loss function is a weighted sum of mean squared error loss, smoothness loss, first HU distribution loss, and mutual information loss; wherein, the mean squared error loss is a weighted sum of skeletal structure loss and soft tissue loss, the skeletal structure loss is used to constrain the alignment of skeletal structures between the floating image and the fixed image, the soft tissue loss is used to constrain the alignment of soft tissues between the floating image and the fixed image, and the weight of the skeletal structure loss is greater than the weight of the soft tissue loss; the smoothness loss is used to constrain the continuity and smoothness of the spatial deformation field; the first HU distribution loss is used to constrain the matching degree of tissue density between the floating image and the fixed image; and the mutual information loss is used to constrain the similarity between the floating image and the fixed image.

2. The cone-beam computed tomography image processing method according to claim 1, characterized in that, Also includes: By using a second loss function, an attention-based generative adversarial network for generating pCT images is trained using historical vCT images and historical CBCT images to obtain an image generation model. The input of the image generation model includes CBCT images, and the output includes pCT images. The second loss function is a weighted sum of adversarial loss, perceptual loss, second HU distribution loss, and edge loss. The adversarial loss is used to constrain the matching degree between the generated pCT image and the vCT image. The perceptual loss is a weighted sum of the first perceptual loss and the second perceptual loss. The first perceptual loss is used to constrain the consistency between the generated pCT image and the CBCT image at the feature level. The second perceptual loss is used to constrain the consistency between the generated pCT image and the vCT image at the feature level. The weight of the first perceptual loss is greater than the weight of the second perceptual loss. The second HU distribution loss is used to constrain the consistency of the gray-level histogram distribution between the generated pCT image and the vCT image. The edge loss is used to constrain the authenticity of organ boundaries and structures in the generated pCT image.

3. The cone-beam computed tomography image processing method according to claim 2, characterized in that, The formula for calculating the adversarial loss is as follows: In the formula, Indicating resistance to loss, This represents the expectation operation. x Represents CBCT images, y Represents vCT images, This represents the discriminator's judgment result on the real image pairs. Represents pCT images, This indicates the discrimination result of the discriminator on the generated image pair. The real image pair is an image pair formed by stitching together a CBCT image and a vCT image, and the generated image pair is an image pair formed by stitching together a CBCT image and a pCT image.

4. The cone-beam computed tomography image processing method according to claim 2, characterized in that, Features of each layer in vCT and pCT images are extracted based on the VGG network; The formula for calculating the perception loss is as follows; In the formula, Indicates perceived loss. x Represents CBCT images, y Represents vCT images, ( ) represents the feature map of any intermediate layer in the VGG network. Represents pCT images, Indicates the loss of first perception. This indicates the loss of second perception. and Let represent the weights of the first perceptual loss and the second perceptual loss, respectively. .

5. The cone-beam computed tomography image processing method according to any one of claims 1 to 4, characterized in that, Acquire target CT and target CBCT images containing bony and soft tissue structures, including: Acquire raw CBCT images and raw CT images; The original CT image is rigidly registered to the position of the original CBCT image; Using the voxel spacing of the original CBCT image as a reference, the rigid registration result is resampled; The resampling results and the region of interest in the original CBCT image are extracted to obtain the target CT image and the target CBCT image. The region of interest includes bony structures and soft tissues.

6. The cone-beam computed tomography image processing method according to claim 1, characterized in that, The pCT image and the vCT image are fused to obtain the image processing result, including: The pCT image and the vCT image are fused using a linear weighting principle to obtain the image processing result. The weight of the pCT image is set such that the lower the gray value, the greater the weight, and the weight of the vCT image is set such that the higher the gray value, the greater the weight.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the cone-beam computed tomography image processing method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the cone-beam computed tomography image processing method according to any one of claims 1 to 6.

Images