Dual-energy material decomposition image synthesis method and system

Abstract

Provided in the present invention are a dual-energy material decomposition image synthesis method and system. The system comprises a model training unit, wherein the model training unit uses historical dual-energy material decomposition images of a patient and a low-dose or dose-free image of the patient after an anatomical structure changes to train a dual-energy material decomposition image model. The model inputs are the historical dual-energy material decomposition images and the low-dose or dose-free image of the patient after the anatomical structure changes, and the output is an dual-energy material decomposition image corresponding to the low-dose or dose-free image of the patient after the anatomical structure changes. The solution provided in the present invention has the following beneficial effects: by means of a dual-energy image synthesis technique, providing a low-dose and high-precision intelligent imaging method for scenarios such as image guidance and dose calculation required in online adaptive radiotherapy, while aiming to improve the quality of low-dose sparsely reconstructed cone-beam CT images. A quantitative image basis is provided for clinical application scenarios such as online adaptive radiotherapy and dose reconstruction.

Description

A method and system for synthesizing dual-energy material decomposition diagrams Technical Field

[0001] The present invention is applicable to, but not limited to, the field of image guidance technology required for online adaptive radiotherapy, and in particular relates to a dual-energy material decomposition image synthesis method and system. Background Art

[0002] With the widespread application of techniques such as intensity-modulated radiotherapy and stereotactic radiotherapy, dose gradients within and outside the tumor target volume have become steeper. Traditional image-guided radiation therapy (IGRT) significantly reduces setup errors. However, dose deviations caused by anatomical changes due to organ filling and tumor regression, as well as electron density errors in CT images, not only adversely affect treatment efficacy but may also damage normal organs. Addressing these issues requires the development and maturity of more advanced technologies such as online adaptive radiotherapy (ART). Simultaneously acquiring accurate anatomical and material decomposition information through innovative airborne imaging methods is a key technical bottleneck. Currently, Elekta's Unity and ViewRay's MRIdian Linac, United Imaging's uRT-linac 506c, and Varian's Ethos system have explored online adaptive radiotherapy using different imaging modalities, including magnetic resonance (MR), airborne CT, and high-resolution cone-beam CT (CBCT), respectively.

[0003] Although MR imaging has the advantages of high spatial resolution, high contrast and no radiation, its high cost, complex technology, low compatibility with accelerators and lack of electron density information required for dose calculation have limited its promotion and application in the field of radiotherapy.

[0004] As the most widely used IGRT image-guided modality in clinical practice, CBCT, in addition to its application in traditional positioning error correction, is also expected to provide image information reflecting the patient's true anatomical structure on the day of treatment for more complex scenarios such as dose calculation. However, before it can be implemented clinically, it still needs to overcome three technical bottlenecks: high imaging dose, poor image quality, and low material decomposition accuracy.

[0005] Currently, there is no commercial DECT (or DECBCT) specifically for accelerator-based online image guidance, and imaging technology based on repeated scanning may increase patients' additional radiation dose, time and economic costs. These unfavorable factors limit the application of imaging technology in online adaptive radiotherapy.

[0006] Summary of the Invention

[0007] The main purpose of the present invention is to overcome the above-mentioned defects of the prior art and provide a method and system for synthesizing a dual-energy material decomposition diagram.

[0008] The present invention provides a dual-energy material decomposition image synthesis method. This method uses a patient's previous dual-energy material decomposition images and low-dose or no-dose images of the patient's anatomically altered structures to train a dual-energy material decomposition image model. The model inputs the previous dual-energy material decomposition images and the low-dose or no-dose images of the patient's anatomically altered structures, and outputs a dual-energy material decomposition image corresponding to the low-dose or no-dose images of the patient's anatomically altered structures. This model can simplify treatment procedures, making it possible to obtain dual-energy material decomposition images from low-dose or no-dose images during treatment. Dual-energy image synthesis technology provides a low-dose, high-precision intelligent imaging method for scenarios such as image guidance and dose calculation required for online adaptive radiotherapy. More preferably, the dual-energy material decomposition image model is trained using previous dual-energy material decomposition images from multiple patients and low-dose or no-dose images of each patient's anatomically altered structures. Using these images for training can improve the quality and accuracy of the dual-energy material decomposition images output by the dual-energy material decomposition image model.

[0009] Preferably, the low-dose or zero-dose images of the patient's anatomically altered structures are CBCT images, CT images, or MRI images reconstructed from projection images generated at different gantry angles. Compared to the 700 to 900 projections required for a 200° or 360° scan in traditional CBCT full-fan mode, and the 150-250 projections required for a 360° scan in half-fan mode, this embodiment only uses approximately 10% of the projection images generated at different gantry angles for CBCT reconstruction, significantly reducing the patient's radiation exposure. Furthermore, MRI images are radiation-free.

[0010] More preferably, the low-dose images, specifically CT or CBCT images of anatomically altered structures, can be obtained by reducing the number of projections, lowering the tube voltage, reducing the exposure, or using specialized detector hardware. Alternatively, the dose can be lower than that obtained by rescanning dual-energy CT or dual-energy CBCT to obtain material decomposition images, which is also a low-dose imaging solution. Radiation is harmful to the human body, and this solution can reduce the risk of radiation damage to patients without sacrificing diagnostic and treatment accuracy.

[0011] More preferably, a test set consisting of multiple patient images is included. Dual-energy material decomposition images are generated from the patient images in the test set and used as ground truth to quantitatively evaluate the performance of dual-energy material decomposition images synthesized using the dual-energy material decomposition model for images with the same anatomical structure as the patient images in the test set. If the performance of the dual-energy material decomposition images synthesized by the dual-energy material decomposition model does not meet the predetermined standard, training is continued until the predetermined standard is met. Verification is performed during or after training of the dual-energy material decomposition model to ensure the accuracy of the dual-energy material decomposition images generated by the model.

[0012] More preferably, after the dual-energy material decomposition model is trained, a low-dose or no-dose image of the patient's anatomically altered structure is input into the dual-energy material decomposition model, and the output is the dual-energy material decomposition image of the patient's anatomically altered structure. During the training phase of the dual-energy material decomposition model, the model inputs are the previous dual-energy material decomposition image and the low-dose or no-dose image of the patient's anatomically altered structure. After the dual-energy material decomposition model is trained, when applied, only the low-dose or no-dose image of the patient's anatomically altered structure, such as a CBCT image, CT image, or MRI image taken on the day of treatment or closest to the day of treatment, needs to be input into the dual-energy material decomposition model to obtain the corresponding dual-energy material decomposition image of the patient's anatomically altered structure, without requiring the input of the previous dual-energy material decomposition image.

[0013] The present invention provides a dual-energy material decomposition image synthesis system, including a model training unit. The model training unit uses a patient's previous dual-energy material decomposition image and a low-dose or no-dose image of the patient after the anatomical structure changes to perform dual-energy material decomposition image model training. The model input is the previous dual-energy material decomposition image and the low-dose or no-dose image of the patient after the anatomical structure changes, and the output is a dual-energy material decomposition image corresponding to the low-dose or no-dose image of the patient after the anatomical structure changes.

[0014] More preferably, the dual-energy material decomposition model is trained using previous dual-energy material decomposition images of multiple patients and low-dose or no-dose images of each patient after anatomical changes.

[0015] More preferably, the low-dose or zero-dose image of the patient's anatomical structure after the change is a CBCT image or CT image, or a nuclear magnetic resonance image, reconstructed by projection images generated by different gantry angles.

[0016] More preferably, the low-dose image, specifically the CT image or CBCT image after the anatomical structure change, reduces the dose by reducing the number of projections, lowering the tube voltage, reducing the exposure, or using special detector hardware; or, the dose is lower than the method of obtaining a material decomposition image by rescanning the dual-energy CT or dual-energy CBCT, which is also a low-dose image solution.

[0017] More preferably, a test set consisting of multiple patient images is included, and a dual-energy material decomposition image is obtained based on the images of the patients in the test set. The dual-energy material decomposition image is used as the true value to quantitatively evaluate the performance of the dual-energy material decomposition image synthesized by the dual-energy material decomposition image model for images with the same anatomical structure as the images of the patients in the test set. If the performance of the dual-energy material decomposition image synthesized by the dual-energy material decomposition image model does not meet the predetermined standard, training continues until the predetermined standard is met.

[0018] More preferably, after the dual-energy material decomposition model is trained, the low-dose or no-dose image of the patient after the anatomical structure changes is input into the dual-energy material decomposition model, and the output is the dual-energy material decomposition image of the patient after the anatomical structure changes.

[0019] The beneficial effects of each part of a dual-energy material decomposition diagram synthesis system are the same as those of the above method and will not be repeated here.

[0020] The beneficial effects of the solution of the present invention are as follows:

[0021] This patent uses sparse reconstruction technology to obtain low-quality cone-beam CT images, CT, and MRI images that reflect the anatomical structure on the day of treatment at a lower radiation dose. Combining pre-treatment DECT as prior knowledge, deep learning technology is used to synthesize high-quality dual-energy material decomposition images (MDI) that are consistent with the anatomical structure on the day of treatment. This approach aims to improve the quality of low-dose sparsely reconstructed cone-beam CT images while providing a low-dose, high-precision intelligent imaging method for scenarios such as image guidance and dose calculation required for online adaptive radiotherapy through dual-energy image synthesis technology. This provides a quantitative imaging foundation for clinical application scenarios such as online adaptive radiotherapy and dose reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0023] Figure 1 is a schematic diagram of the overall technical roadmap of a dual-energy material decomposition diagram synthesis method;

[0024] Figure 2 Schematic diagram of the forward loop structure of CycleGAN;

[0025] Figure 3 Schematic diagram of the backward loop structure of CycleGAN;

[0026] Figure 4: An image of a male patient in the training set;

[0027] Figure 5: An image of a female patient in the test set. DETAILED DESCRIPTION

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of the present invention. The technical features of each embodiment can be combined with each other to form a practical solution for achieving the purpose of the invention. For ordinary technicians in this field, without paying any creative work, the present invention can also be applied to other similar scenarios based on these drawings. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0029] It should be understood that the terms "system" and "unit" as used herein are a method for distinguishing between different components, elements, parts, portions, or assemblies at different levels. However, other expressions may be substituted if they achieve the same purpose. Furthermore, a "system" and "unit" may be implemented by software or hardware and may refer to a physical or virtual component having such functionality.

[0030] Flowcharts are used in the present invention to illustrate the operations performed by the systems according to the embodiments of the present invention. It should be understood that the preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes. The technical solutions in the various embodiments may be combined to achieve the objectives of the present invention.

[0031] Example 1:

[0032] A dual-energy material decomposition image synthesis method, as shown in Figure 1, uses a patient's dual-energy material decomposition image and a low-dose or no-dose image of the patient after anatomical structure changes to train a dual-energy material decomposition image model. The model input is the dual-energy material decomposition image and the low-dose or no-dose image of the patient after anatomical structure changes, and the output is the dual-energy material decomposition image corresponding to the low-dose or no-dose image of the patient after anatomical structure changes.

[0033] Dual-energy CT (DECT) or dual-energy CBCT (DECBCT) technology decomposes measurement data and generates images of different base materials through X-ray spectra of different energies. It can not only improve tissue density information and dose calculation accuracy, but also provide more important data, such as accurately quantifying iodine concentration in tissue to determine the blood supply of tumors and evaluate the effectiveness of tumor treatment.

[0034] Besides using dual-energy CT or dual-energy CBCT, dual-layer detectors, counting detectors, and other methods can also be used to obtain dual-energy material decomposition images. Regardless of the method, as long as the dual-energy material decomposition images can be provided for dual-energy material decomposition model training, they are sufficient.

[0035] Regarding low-dose or no-dose imaging, low-dose can include various modalities such as low-dose CT and low-dose CBCT, while no-dose imaging includes MRI. Low-dose imaging, specifically CT or CBCT images of anatomical changes, can be achieved by reducing the number of projections, lowering the tube voltage, reducing the exposure, or using specialized detector hardware. Alternatively, a method that reduces the dose by rescanning dual-energy CT or dual-energy CBCT to obtain material decomposition images is also a low-dose imaging approach. CT and CBCT tube voltages range from approximately 60kV to 140kV in terms of kV energy. Alternatively, imaging with MV-level radiation, such as MVCT or MVCBCT, typically uses energies between 2.5MV and 10MV for commercial systems. The maximum tube current for CT machines is approximately 1000mA, while that for CBCT machines is approximately 500mA. Lower voltage, energy level, and current reduce radiation exposure.

[0036] Dual-energy material decomposition graph model. This embodiment uses a 2D CycleGAN network to implement dual-energy material decomposition graph model training. It is not limited to the 2D CycleGAN network, and other network models with similar functions can also be used to construct a dual-energy material decomposition graph model.

[0037] The low-dose or zero-dose image of the patient after the anatomical structure change can be input as the image of the patient on the day of treatment, and output as the dual-energy material decomposition image of the patient on the day of treatment.

[0038] For example, 30 groups of male human body data are constructed as training sets. Each group of data includes dual-energy CT before treatment and CBCT images after physiological deformation to reflect the changes in patients during radiotherapy. This embodiment uses an iterative algorithm to perform material decomposition on dual-energy CT to obtain dual-energy material decomposition images, such as bone decomposition images (Material decomposition image of bone, MDIB) and soft tissue decomposition images (Material decomposition image of soft tissue, MDIST). A 2D CycleGAN network based on tomographic images is constructed to achieve modal conversion from CBCT to MDI, and retain the true anatomical structure represented by CBCT on the day of radiotherapy. The 2D CycleGAN network takes the CBCT image, bone decomposition image MDIB and soft tissue decomposition image MDIST on the day of treatment after physiological deformation as input, and outputs the bone decomposition image MDIB and soft tissue decomposition image MDIST on the day of treatment after physiological deformation.

[0039] Based on low-dose sparse reconstruction of CBCT, the 2D CycleGAN network constructed in this paper can realize cross-modal, high-fidelity dual-energy material decomposition image conversion, and can provide a new intelligent imaging method for application scenarios such as online adaptive radiotherapy, ion radiotherapy plan design, dose reconstruction and monitoring on the existing clinical platform.

[0040] Example 2:

[0041] Creation of training and test sets

[0042] In order to simulate different human body conditions, this embodiment uses a hybrid phantom to construct various organs in the body to provide a realistic pixelated model. Specifically, by changing the 13 anatomical structure parameter variables such as body shape, chest and abdomen, and heart in the XCAT digital model tool, simulated digital images of 30 male training set patients and 5 female test set patients with different morphologies are generated respectively. In practice, the training set patients and test set patients do not need to be divided by gender, and both men and women are acceptable. The set variable parameters obey a normal distribution with an expectation of 1.0 and a variance of 0.04. For the 35 training set and test set patients, DECT images (DECT1) were reconstructed based on data with tube voltages of 80kV and 125kV, respectively, representing the anatomical structure data of the patient in the radiotherapy preparation stage. By further adjusting the physiological deformation parameters to simulate the before and after changes in the anatomical structure of the same patient during radiotherapy, the CBCT images in radiotherapy guidance were simulated and reconstructed based on the deformed data. The CBCT tube voltage was set to 80 kV, and the geometry simulation used the parameter settings of a Varian OBI 2000 kV CBCT system (Table 1). Projection images were generated from 87 different gantry angles for CBCT reconstruction. Compared to the approximately 630 projections required for a conventional CBCT 360° scan, this embodiment only used 87 projections, representing approximately 13.8%. However, the method of this patent is not limited to 87 projections. For example, in another embodiment, projection images were generated from 84 different gantry angles for CBCT reconstruction, with the number of projections being approximately 10% of that required for a conventional CBCT scan. For example, a conventional CBCT scan requires 841 projections.

[0043] The method of the present invention uses CBCT with sparse projection reconstruction of at least 10%, which greatly reduces the patient's radiation dose and scanning time, while achieving cross-modal, high-fidelity dual-energy material decomposition image conversion. It can provide a new intelligent imaging method for application scenarios such as online adaptive radiotherapy, ion radiotherapy plan design, dose reconstruction and monitoring on existing clinical platforms.

[0044] Table 1. Geometric parameter settings of the Varian OBI kV CBCT system used in this work to simulate projection images.

[0045] To evaluate the accuracy of the deep learning model's material decomposition images generated from DECT1 and CBCT, a test set of patients constructed DECT images (DECT2) with identical anatomical structures to the CBCT images. MDIB and MDIST were used as ground truth values ​​to quantitatively evaluate the model's performance in synthesizing dual-energy material decomposition images. Each patient's reconstructed DECT and CBCT images each contained 150 slices.

[0046] The present invention has been verified to be feasible using both hybrid phantom images and real patient images. In practice, it is recommended to use real patient cases to establish training and test sets. The DECT image acquisition process involves extracting two voltage images of the same anatomical structure for the same patient at the same treatment stage. The CBCT image acquisition process involves extracting a voltage image of the same patient at a different stage and with different anatomical deformation compared to the previous DECT image. CBCT images can also be CT images or MRI images.

[0047] The number of patients in the training and test sets is not limited to the aforementioned 30 and 5 cases, and can be selected based on practical conditions. The more data in the training and test sets, the better.

[0048] This embodiment reconstructs DECT images based on data at tube voltages of 80 kV and 125 kV. However, this patent is not limited to these two voltages or X-ray energies. DECT images can also be reconstructed using data from other voltages, or the same voltage can be used to obtain DECT images through a dual-layer detector. Other voltages are also possible, such as 85 kV and 125 kV, or 100 kV and 6 MV, which are not listed here.

[0049] Decomposition of matter

[0050] Based on 80 kV and 125 kV CT images of the same anatomical structure obtained in the training and test sets, this example uses an iterative decomposition method to obtain DECT dual-energy decomposition images. As a preparatory step, the image domain direct decomposition uses the inverse linear attenuation matrix operation of the basis material. The specific formula used is as follows:

[0051] Among them I M1 and I M2 Respectively represent the composition ratio of the two base materials; LAC H and LAC L Represent the linear attenuation coefficients obtained in high-energy and low-energy scans respectively; the decomposition matrix (v M1,H ,v M1,L ,v M2,H ,v M2,L ) is established by respectively collecting the CT values ​​of a homogeneous area containing two base materials in two scanning modes during calibration. M1,H ,V M1,L ,V M2,H V M2,L They represent the linear attenuation coefficients of base material 1 and base material 2 used for dual-energy decomposition at high energy and low energy, respectively. Specifically, in the form of a decomposition matrix, they represent the four coefficients multiplied by the same n-order unit matrix, where n is the number of pixels in the length and width directions of the image.

[0052] The results will be directly decomposed, that is, the proportion of the two base materials bone and soft tissue, as mentioned above The initial solution for the iterative decomposition is used as the basis for the optimal linear unbiased estimate. The inverse of the estimated variance matrix of the decomposed image is then used as the penalty weight for the least squares term. A regularization term is introduced to enhance image smoothness, namely, by calculating the sum of squares of the differences between adjacent pixel values ​​to promote image smoothness. Furthermore, to maintain clear boundaries in the decomposed image, image edges are first detected and this edge information is retained during the regularization process. This method uses the noise variance-covariance matrix of the decomposed image for noise reduction, avoiding the problem of base material noise amplification that may result from direct inversion. By using the electron density of bone and soft tissue as the base material for iterative decomposition, a bone decomposition image (MDIB) and a soft tissue decomposition image (MDIST) can be obtained, respectively, improving image quality while clearly presenting bone and soft tissue structures.

[0053] In addition to using the aforementioned two-voltage CT images, dual-energy material decomposition images can also be obtained using other methods, such as dual-layer detectors and counting detectors. The present invention does not limit the method for obtaining dual-energy material decomposition images. Regardless of the method, as long as the dual-energy material decomposition images can be provided for model training, they will suffice.

[0054] Synthesizing dual-energy material decomposition diagram based on deep learning

[0055] To achieve unpaired data style conversion from CBCT images to MDI, a 2D CycleGAN method based on tomographic images was used to train a dual-energy material decomposition model. CycleGAN consists of two generators and two discriminators. The generator attempts to achieve image conversion based on the input image, while the discriminator attempts to distinguish the generated image from the real image, thereby continuously improving the generator's generation effect. CycleGAN can achieve image conversion between the two domains based on unpaired training data. The training involves monoenergetic images and dual-channel material decomposition images, with the goal of converting monoenergetic images into dual-channel material decomposition images. This achieves the conversion of structurally modified images into dual-channel material decomposition images as described in this patent.

[0056] According to this work task, as shown in Figure 2, in the network forward cycle (FC), the generator-MDI (G MDI) realizes the synthesis of MDI dual-channel images through CBCT (Synthesis two-channel material decomposition image, s2C-MDI), the discriminator D MDI Used to evaluate the gap between s2C-MDI and the real input image, and the discriminant results constitute the L in the adversarial loss MDI ; Then through the generator-CBCT (Generator-CBCT, G CBCT ) Generate cycle CBCT (Cycle-CBCT, cCBCT) from s2C-MDI, compare cCBCT with the real input CBCT, and obtain L in the cycle consistency loss FC As shown in Figure 3, in the backward cycle (BC), through G CBCT Generate Synthesis CBCT (sCBCT) from 2C-MDI, discriminator D CBCT Used to evaluate the gap between sCBCT and the real input image, the discriminant results constitute the L in the adversarial loss CBCT , then G MDI Generate cycle 2C-MDI (c2C-MDI) from sCBCT, compare c2C-MDI with the 2C-MDI of the real input, and obtain L in the cycle consistency loss BC The loss function of CycleGAN consists of adversarial loss and cycle consistency loss. The adversarial loss formula of the two cycles is:

[0057] and L MDI =E MDI [(1-D MDI (MDI) 2 ]+E CBCT [(D MDI (G MDI (CBCT))) 2 ] L CBCT =E CBCT [(1-D CBCT (CBCT) 2 ]+E MDI [(D CBCT (G CBCT (CBCT))) 2 ]

[0058] The cycle consistency losses of the two cycles are: L FC =E CBCT [||CBCT-G CBCT (G MDI (CBCT))||1]

[0059] and L BC =E MDI [||MDI-G MDI (G CBCT (MDI))||1]

[0060] Combining the two losses, the complete loss function is: L cyclegan =L MDI +L CBCT +ω(L FC +L BC )

[0061] The overall technical route of this patent is shown in Figure 1, XCAT: digital model tool; CBCT: cone beam CT; DECT: dual-energy CT; MDI: material decomposition diagram; sMDI: synthetic material decomposition diagram.

[0062] The flowchart in Figure 1 illustrates the overall process. Figures 2 and 3 illustrate the forward and backward cycles of CycleGAN. For this flowchart, 30 patients were trained on a male phantom and 5 on a female phantom. The 30 patients were used for model training, and the 5 patients were used for model testing. The 125 / 80 kV CT scan represents pre-treatment dual-energy CT scans, while the physiologically deformed 80 kV CT-to-CBCT scan below represents deformed CBCT scans on the day of treatment. Combining CT and CBCT allows for both training and validation. The model is tested on paired data generated from the female phantom data. Specifically, the CBCT and DECT images in the test set are undistorted and have the same structure. This is for performance evaluation purposes. The undistorted DECT decomposition maps serve as the ground truth. If the performance of the dual-energy material decomposition maps generated by the dual-energy material decomposition model does not meet the predetermined standard, training continues until it does.

[0063] The forward and backward loop structures of CycleGAN are shown in Figures 2 and 3, respectively. The training process uses an NVIDIA (3070) GPU and uses the Adam optimizer for training, with a learning rate of 0.0002 and a batch size of 2. The training stopped after 130 rounds, at which time the verification loss began to stabilize. In order to evaluate the training effect, this patent uses three quantitative indicators: Structural Similarity Index (SSIM Index), Root Mean Square Error (RMSE), and Peak Signal-to-Noise Ratio (PSNR) to evaluate the similarity between the generated image and the real image, thereby reflecting the performance of the network. The definitions of RMSE, SSIM, and PSNR are as follows:

[0064] The structural similarity SSIM of two images is defined as follows: h1=(0.01L) 2 ,h2=(0.03L) 2

[0065] Among them, μ x and are images x and The mean of δ x and is x and The variance of δ xy is x and covariance; h1 and h2 are two constants; L is the range of pixel values ​​of the two images.

[0066] Among them MAX x is the maximum pixel value of image x, MSE is the sum of image x and The mean square error of .

[0067] According to the above definition, the closer the SSIM value is to 1, the better the consistency between the two images; the lower the RMSE, the smaller the image error; the larger the PSNR value, the better the image quality and consistency.

[0068] Figure 4 shows the various image sets for a male patient used as a training set. Figures a and b are the patient's pre-treatment planning CT images at 125 kV and 80 kV, respectively. The MDIB and MDIST images (Figures c and d) were subsequently derived using an iterative dual-energy image decomposition method. Figure e shows projections generated from an 80 kV XCAT phantom image after physiological deformation (gantry angle 81.9 degrees), representing CBCT projections based on the patient's true anatomy on the day of radiotherapy. Figure f shows a CBCT sparsely reconstructed from 87 projections obtained through a 360-degree rotation, also representing the true anatomy on the day of radiotherapy after physiological deformation.

[0069] The present invention successfully trained a 2D CycleGAN model using data from 30 male patients. The model synthesized sMDIB and sMDIST that were consistent with the anatomical structure on the day of radiotherapy based on the DECT images before radiotherapy and the sparsely reconstructed CBCT images of 5 female patients in the test set that underwent anatomical deformation on the day of radiotherapy. One example is shown in Figure 5. As the evaluation truth value of the synthetic image of the evaluation model, Figures 5(b) and (e) respectively show the MDIB and MDIST obtained by iterative decomposition of 125kV and 80kV dual-energy CT after physiological deformation of the test set cases. Their anatomical structures are exactly the same as those in Figures 5(c) and (f) generated by the model, while there are certain deformations in the anatomical structures of the radiotherapy planning stage represented by Figures 5(a) and (d), simulating the physiological changes of patients during radiotherapy. Figure 5. An image of a female patient in the test set. Figures a and d are the MDIB and MDIST images decomposed based on the pre-treatment planning CT scan, respectively; Figures b and e are the true reference values ​​of MDIB and MDIST on the day of treatment, respectively; and Figures c and f are the synthetic sMDIB and sMDIST representing the anatomical structure on the day of treatment predicted by the model based on the sparsely reconstructed CBCT images.

[0070] Table 2 shows the mean ± standard deviation of the SSIM, RMSE, and PSNR of the 150-layer synthetic images of each female patient as the test set compared with the true value, as well as the mean ± standard deviation of the results of the 750-layer images of the 5 test set patients.

[0071] As the most widely used IGRT image-guided modality in clinical practice, CBCT, in addition to its application in traditional setup error correction, is also expected to provide image information reflecting the patient's true anatomical structure on the day of treatment for more complex scenarios such as dose calculation. However, before it can be implemented clinically, it still needs to overcome three technical bottlenecks: high imaging dose, poor image quality, and low material decomposition accuracy. To address these issues, the present invention proposes a new method that uses the material decomposition information carried by pre-treatment DECT, combined with the anatomical structure on the day of treatment represented by low-dose, low-projection CBCT, and uses deep learning technology to synthesize a dual-energy material decomposition image that reflects the actual situation on the day of treatment. This method has the advantages of low patient radiation dose and no change in the clinical diagnosis and treatment process.

[0072] To validate the feasibility of this technology, the present invention conducted experiments using a digital simulation phantom and successfully achieved image modality conversion from CBCT to MDI based on an improved 2D CycleGAN network. Digital phantoms can generate multimodal imaging data of virtual patients using computer simulation technology. Compared to the ethical risks and radiation safety issues associated with using real patient data, or when patient data cannot be obtained experimentally, digital simulation has become an important alternative tool and is therefore widely used in the feasibility pre-study and clinical trial stages of new technologies. Digital simulation phantoms also offer unique advantages such as non-invasiveness, reproducibility, precise control of experimental conditions, accelerated research progress, and the generation of "ground truth" based on existing knowledge. For example, the digital phantom's anatomical and physiological data are known, providing a "gold standard" for research evaluating and improving imaging equipment and technologies. In this invention, the digital simulation phantom effectively addresses the technical challenge of constructing a ground truth test set. By generating dual-energy CT images that are fully consistent with the CBCT anatomy, it provides a standard for quantitative evaluation of model-synthesized images, effectively overcoming the "ground truth" issue commonly found in real-world research due to different scanning devices and the non-rigid structure of the human body.

[0073] Although a large number of experiments in the present invention are implemented through digital simulation technology, it has been verified that it is feasible to use real patient image data as training set and test set data, and the technical solution of the present invention can be applied to specific applications of real patients.

[0074] During network training, the patient's CBCT images, MDIB, and MDIST images from the day of treatment were used as input, and the output was the sMDIB and sMDIST images of the patient from the day of treatment. After the dual-energy material decomposition model was trained, low-dose or no-dose images of the patient's anatomically altered structure were input into the dual-energy material decomposition model. The output was the dual-energy material decomposition image of the patient's anatomically altered structure. This was the application phase, in which the model used the patient's CBCT images from the day of treatment as input, and the output was the sMDIB and sMDIST images of the patient from the day of treatment.

[0075] In order to better verify the generalization performance of the model, the present invention also constructed training sets and test sets based on male and female data respectively to maximize the anatomical structure differences between the two. As shown in Table 2, while reducing the number of projections by 86.2% and thus significantly reducing the imaging dose, the sMDIB and sMDIST of the test set patients output by the model based on sparsely reconstructed CBCT still maintained a high degree of consistency with the true values, with an average SSIM value ≥ 0.942, an average RMSE value ≤ 3.361, and an average PSNR value ≥ 28.063. The comparison images provided in Figure 5 also confirm the results of Table 2. The above data show that while retaining the correct anatomical information represented by CBCT, the model can successfully learn accurate material decomposition data from DECT with physiological deformation (as shown in Figures 5d-e), and shows good generalization performance between patients of different genders.

[0076] The model successfully converted five monoenergetic sparsely reconstructed CBCT images from the test set into anatomically consistent MDIB and MDIST images, using only approximately 13.8% of the projection number and radiation dose of conventional methods. The structural similarity index (SSIM) of the synthesized MDIB and MDIST images compared to the ground truth was 0.946±0.013 and 0.942±0.013, respectively. The root mean square error (RMSE) was 2.434±1.592 and 3.361±0.841, respectively. The peak signal-to-noise ratio (PSNR) was 29.850±1.082 and 28.063±0.957, respectively. Model training took approximately 44 hours and 47 minutes, and synthesizing MDI images took approximately 98 seconds.

[0077] Table 2 also shows that the model consistently outperforms MDIB images in predicting MDIB images. This may be due to the relatively small amount of detail in MDIB, making it easier to predict. Future research could further explore training MDIB and MDIST separately by increasing the number of network channels, but this would increase training time and place higher demands on the hardware environment. Furthermore, while this method achieves image conversion, the prediction results still suffer from a certain loss in image contrast. To further improve the model's predictive capabilities, subsequent research will explore the following optimization options: (a) adjusting the loss function used by CycleGAN, such as adjusting the contrast of the original loss function or the weighting of image boundaries, or introducing alternative loss functions such as perceptual loss or content loss, to improve image quality and contrast; (b) adjusting the CycleGAN model architecture, such as increasing or decreasing the number of layers, adding or reducing transition layers, and adjusting the convolution kernel size, to improve model performance and image quality; and (c) attempting to preprocess the input image and post-process the output image, such as adjusting brightness, contrast, and color balance, to improve image quality and contrast. The implementation of the above optimization scheme is expected to further improve the predictive ability of CycleGAN and provide higher quality images for potential clinical applications.

[0078] A dual-energy material decomposition diagram synthesis system of the present invention corresponds one-to-one to a dual-energy material decomposition diagram synthesis method, which will not be described in detail here. Please refer to the above explanation for details.

[0079] The beneficial effects that may be brought about by the embodiments of the present invention include but are not limited to:

[0080] This patent utilizes sparse projection reconstruction to significantly reduce CBCT radiation dose. Combining DECT prior data with an improved CycleGAN network preserves the true anatomical information reflected by CBCT while providing higher image quality and more accurate dual-energy material decomposition information. Prior to the advent of commercial airborne DECBCT, this work is expected to provide new low-dose, high-precision intelligent imaging methods and data for complex clinical applications such as online adaptive radiotherapy, online proton / heavy ion planning, 3D dose reconstruction, and beam monitoring.

[0081] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other possible beneficial effects.

[0082] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of the present invention.

[0083] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in the present invention are not intended to limit the order of the processes and methods of the present invention. Although the above disclosure discusses some embodiments of the invention that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of the present invention.

[0084] Similarly, it should be noted that in order to simplify the presentation of the present disclosure and thus facilitate understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the invention sometimes combines multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not mean that the subject matter of the invention requires more features than those mentioned in the claims.

[0085] Finally, it should be understood that the embodiments described herein are intended only to illustrate the principles of the present invention. Other variations may also fall within the scope of the present invention. Therefore, by way of example and not limitation, alternative configurations of the embodiments of the present invention may be considered consistent with the teachings of the present invention. Accordingly, the embodiments of the present invention are not limited to the embodiments explicitly described and illustrated herein.

Claims

1. A method for synthesizing a dual-energy material decomposition diagram, characterized in that, Using the previous dual-energy material decomposition images of the patient and the low-dose or no-dose images after the anatomical structure change of the patient, perform dual-energy material decomposition map model training. The input of the model is the previous dual-energy material decomposition images and the low-dose or no-dose images after the anatomical structure change of the patient, and the output is the dual-energy material decomposition images corresponding to the low-dose or no-dose images after the anatomical structure change of the patient.

2. The synthetic method of the dual-energy substance decomposition diagram according to claim 1, characterized in that, Using the previous dual-energy material decomposition images of multiple patients and the low-dose or no-dose images after the anatomical structure change of each patient, perform dual-energy material decomposition map model training.

3. The method for synthesizing a dual-energy substance decomposition diagram according to claim 1 or 2, characterized in that The low-dose or no-dose images after the anatomical structure change of the patient are CBCT images or CT images reconstructed by projection of the projection images generated at different gantry angles, or magnetic resonance images.

4. The method for synthesizing a dual-energy substance decomposition diagram according to claim 1, wherein The low-dose image is specifically a CT image or a CBCT image after the anatomical structure change, and the dose is reduced by reducing the number of projections, or reducing the tube voltage, or reducing the exposure, or using special detector hardware; or, a method with a dose lower than that of re-scanning dual-energy CT or dual-energy CBCT to obtain material decomposition images is also a low-dose image scheme.

5. The dual-energy material decomposition diagram synthesis method according to any one of claims 1-4, characterized in that It also includes a test set composed of multiple patient images. Obtain the dual-energy material decomposition images from the images of the patients in the test set as the ground truth, and quantitatively evaluate the performance of the dual-energy material decomposition maps synthesized by the dual-energy material decomposition map model for images with the same anatomical structure as the patients in the test set. If the performance of the dual-energy material decomposition maps synthesized by the dual-energy material decomposition map model does not meet the predetermined standard, continue training until the predetermined standard is reached.

6. The method for synthesizing a dual-energy substance decomposition diagram according to any one of claims 1-5, characterized in that, After the dual-energy material decomposition map model training is completed, input the low-dose or no-dose images after the anatomical structure change of the patient into the dual-energy material decomposition map model, and the output is the dual-energy material decomposition images after the anatomical structure change of the patient.

7. A dual-energy substance decomposition diagram synthesis system, characterized in that It includes a model training unit. The model training unit uses the previous dual-energy material decomposition images of the patient and the low-dose or no-dose images after the anatomical structure change of the patient to perform dual-energy material decomposition map model training. The input of the model is the previous dual-energy material decomposition images and the low-dose or no-dose images after the anatomical structure change of the patient, and the output is the dual-energy material decomposition images corresponding to the low-dose or no-dose images after the anatomical structure change of the patient.

8. The dual-energy substance decomposition diagram synthesis system according to claim 6, characterized in that, Using the previous dual-energy material decomposition images of multiple patients and the low-dose or no-dose images after the anatomical structure change of each patient, perform dual-energy material decomposition map model training.

9. The dual-energy material decomposition diagram synthesis system according to claim 6 or 7, characterized in that, The low-dose or no-dose images after the anatomical structure change of the patient are CBCT images or CT images reconstructed by projection of the projection images generated at different gantry angles, or magnetic resonance images.

10. The dual-energy material decomposition diagram synthesis system according to claim 7, characterized in that, The low-dose image is specifically a CT image or a CBCT image after the anatomical structure change, and the dose is reduced by reducing the number of projections, or reducing the tube voltage, or reducing the exposure, or using special detector hardware; or, a method with a dose lower than that of re-scanning dual-energy CT or dual-energy CBCT to obtain material decomposition images is also a low-dose image scheme.

11. The dual-energy substance decomposition diagram synthesis system according to any one of claims 7-10, characterized in that, It further includes a test set composed of multiple patient images. Based on the images of the patients in the test set, dual-energy material decomposition images are obtained as the ground truth, and the performance of the dual-energy material decomposition images synthesized by the dual-energy material decomposition map model for images with the same anatomical structure as the patients in the test set is quantitatively evaluated. If the performance of the dual-energy material decomposition images synthesized by the dual-energy material decomposition map model does not meet the predetermined standard, training continues until the predetermined standard is reached.

12. The dual-energy substance decomposition diagram synthesis system according to any one of claims 7-11, characterized in that, After the training of the dual-energy material decomposition map model is completed, low-dose or no-dose images with changed patient anatomical structures are input into the dual-energy material decomposition map model, and the output is the dual-energy material decomposition images with changed patient anatomical structures.

Images