Cyberknife real-time four-dimensional volume image generation method and system based on dual-channel deep learning and individual motion modeling

Abstract

The disclosure relates to a method and system for generating a real-time four-dimensional volume image of a CyberKnife based on dual-channel deep learning and individual motion modeling, which comprises: establishing an initial 4D-DVF by using 4D-CT, simulating and generating a plurality of two-dimensional forward projection image sequences with different respiratory amplitudes through the initial 4D-DVF, training a Unet network architecture based on a Transformer kernel model according to the plurality of two-dimensional forward projection image sequences, generating a target deep learning model, collecting a plurality of orthogonal dual-channel 2D projection images of a target patient's body corresponding to a to-be-detected region under a plurality of projection angle pairs, determining a target 4D-DVF under the corresponding projection angle pair through the 2D projection image, deforming the reference phase of the 4D-CT to generate a 3D volume image, fusing a plurality of 3D volume images to generate a four-dimensional volume image sequence. Thus, the two-dimensional projection is converted into a four-dimensional image in real time, improving the visual judgment conditions of doctors and improving the tracking accuracy of tumor treatment.

Description

Technical Field

[0001] This disclosure relates to the field of biomedical treatment technology, specifically to a method and system for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling. Background Technology

[0002] CyberKnife systems, with their high mechanical positioning accuracy, are widely used for stereotactic body radiotherapy of tumors such as lung cancer. Their image guidance relies on real-time acquisition of two-dimensional orthogonal X-ray projections (typically at 45° and 135°) during treatment to monitor the target area's location. However, this technology has inherent limitations: First, in the two-dimensional projection images, target movement is obstructed by the overlap and occlusion of high-contrast anatomical structures such as ribs and the heart, severely interfering with the physician's visual judgment and automatic tracking of target movement; second, the projection images lack depth dimension information, failing to intuitively and quantitatively display the real-time trajectory and morphological changes of the target area in three-dimensional space, thus limiting treatment accuracy and safety.

[0003] While some studies have attempted to establish respiratory motion models using 4D computed tomography (CT) scans, these methods largely rely on external respiratory signals or assume idealized periodic motion patterns. They fail to fully utilize the real-time 2D image information acquired during treatment and lack the ability to model individual motion variability. Furthermore, the relatively thick slices of routine clinical 4D-CT images (typically 2.5 mm) result in significant differences in spatial resolution between the digitally reconstructed images and the online X-ray projections. This "domain gap" limits the predictive accuracy and clinical transferability of data-driven models. Summary of the Invention

[0004] To overcome the technical problem of inaccurate four-dimensional volume image generation in related technologies, the purpose of this disclosure is to provide a method and system for real-time four-dimensional volume image generation of CyberKnife based on dual-channel deep learning and individual motion modeling.

[0005] To achieve the above objectives, the first aspect of this disclosure provides a method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling, the method comprising: Based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected, an initial 4D-DVF between each phase and the reference phase is constructed. Based on the initial 4D-DVF, multiple two-dimensional orthographic image sequences with different breathing amplitudes are simulated and generated at 45° and 135° based on the CyberKnife imaging geometry. Based on the multiple two-dimensional orthographic image sequences, a model is trained on the Unet network architecture based on the Transformer kernel to generate a target deep learning model. The target deep learning model includes the mapping relationship between multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. During CyberKnife treatment, multiple orthogonal dual-channel 2D projection images of the target area of ​​the patient's body under multiple projection angle pairs are acquired in real time. Based on the orthogonal dual-channel 2D projection images, the target deep learning model determines multiple spatial motion feature vectors of the current target area under different projection angle pairs. Based on the multiple motion information contained in each spatial motion feature vector, the target 4D-DVF under the corresponding projection angle is determined; and based on the target 4D-DVF, the reference phase of the 4D-CT is deformed to generate multiple 3D volume images of multiple orthogonal dual-channel 2D projection images under each projection angle. The multiple 3D volumetric images are fused to generate a four-dimensional volumetric image sequence during the CyberKnife treatment process.

[0006] Optionally, in some embodiments, the initial four-dimensional image deformation field (4D-DVF) between each phase and the reference phase is constructed based on the four-dimensional computed tomography (4D-CT) image sequence corresponding to the area to be detected on the target patient's body. Principal component analysis is used to analyze the motion of principal components in the 4D-CT image sequence, generating principal component motion information. The 4D-DVF is constructed based on the principal component motion information.

[0007] Optionally, in some embodiments, constructing the 4D-DVF based on the principal component motion information includes: Based on the principal component motion information, initial personalized reference model parameters characterizing respiratory motion features are constructed; The initial personalized reference model parameters are interpolated and expanded to generate the 4D-DVF with more variations in motion.

[0008] Optionally, in some embodiments, the Unet network architecture based on the Transformer kernel is a dual-channel network architecture, wherein each channel in the dual-channel network architecture corresponds to a two-dimensional orthographic projection image sequence at a certain angle.

[0009] Optionally, in some embodiments, the plurality of projection angles includes at least two orthogonal angles, 45° and 135°.

[0010] Optionally, in some embodiments, the step of training a model on a Unet network architecture based on a Transformer kernel according to the plurality of two-dimensional orthographic image sequences to generate a target deep learning model includes: Based on the multiple two-dimensional orthographic image sequences, the Unet network architecture based on the Transformer kernel is trained by setting a combined loss function to generate the target deep learning model. The set combined loss function is obtained by weighted summation of the mean squared error loss term and the smoothing loss term.

[0011] According to a second aspect of the present disclosure, a real-time four-dimensional volumetric image generation device for CyberKnife based on dual-channel deep learning and individual motion modeling is provided, the device comprising: The construction module is configured to construct an initial four-dimensional image deformation field (4D-DVF) between each phase and the reference phase based on the four-dimensional computed tomography (4D-CT) image sequence corresponding to the area to be detected on the target patient's body. The generation module is configured to simulate and generate multiple two-dimensional orthographic image sequences with different breathing amplitudes at 45° and 135° based on the initial 4D-DVF and the CyberKnife imaging geometry, and to train a model based on the Unet network architecture with Transformer kernel based on the multiple two-dimensional orthographic image sequences to generate a target deep learning model. The target deep learning model includes the mapping relationship between the multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. The first determining module is configured to, during the CyberKnife treatment process, acquire in real time multiple orthogonal dual-channel 2D projection images of the target patient's body corresponding to the area to be detected under multiple projection angle pairs, and determine multiple spatial motion feature vectors of the current area to be detected under different projection angle pairs based on the orthogonal dual-channel 2D projection images and the target deep learning model. The second determining module is configured to determine the target 4D-DVF under the corresponding projection angle pair based on the multiple motion information contained in each spatial motion feature vector; and deform the reference phase of the 4D-CT according to the target 4D-DVF to generate multiple 3D volume images of multiple orthogonal dual-channel 2D projection images under each projection angle pair. The execution module is configured to fuse the multiple 3D volumetric images to generate a four-dimensional volumetric image sequence during the CyberKnife treatment process.

[0012] According to a third aspect of the present disclosure, a CyberKnife system is provided, characterized in that it includes an image processing device configured to perform the CyberKnife real-time four-dimensional volumetric image generation method based on dual-channel deep learning and individual motion modeling as described in the first aspect of the present disclosure.

[0013] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling as described in any of the first aspects of the present disclosure.

[0014] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: A memory on which computer programs are stored; A processor is configured to execute the computer program in the memory to implement the steps of the method for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling as described in any of the first aspects of this disclosure.

[0015] Using the above technical solution, based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected, an initial 4D image deformation field (4D-DVF) between each phase and the reference phase is constructed. Based on the initial 4D-DVF, multiple two-dimensional orthographic image sequences with different respiratory amplitudes are simulated at 45° and 135° according to the CyberKnife imaging geometry. Based on these multiple two-dimensional orthographic image sequences, a Unet network architecture based on the Transformer kernel is trained to generate a target deep learning model. This target deep learning model includes the mapping relationship between multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. During CyberKnife treatment... The system acquires multiple orthogonal dual-channel 2D projection images of the target area on the patient's body under multiple projection angle pairs in real time. Based on these images, a target deep learning model determines multiple spatial motion feature vectors for the current target area under different projection angle pairs. Based on the motion information contained in each spatial motion feature vector, a target 4D-DVF is determined for the corresponding projection angle pair. Then, based on the target 4D-DVF, the reference phase of the 4D-CT is deformed to generate multiple 3D volume images from the multiple orthogonal dual-channel 2D projection images under various projection angle pairs. These multiple 3D volume images are then fused to generate a sequence of four-dimensional volume images during CyberKnife treatment. This transforms abstract, overlapping two-dimensional projections into intuitive and clear three-dimensional and four-dimensional images in real time, directly displaying the three-dimensional motion trajectory and morphological changes of the target area, greatly improving the visual judgment conditions for doctors and enhancing the tracking accuracy of tumor treatment.

[0016] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling, according to an exemplary embodiment.

[0018] Figure 2 This is a block diagram illustrating a real-time four-dimensional volumetric image generation device for CyberKnife based on dual-channel deep learning and individual motion modeling, according to an exemplary embodiment.

[0019] Figure 3 This is a block diagram illustrating an electronic device 300 according to an exemplary embodiment. Detailed Implementation

[0020] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0021] For example, this application aims to overcome the shortcomings of the prior art and provide a real-time four-dimensional volumetric image generation method and system based on a dual-channel deep learning network and patient-specific 4D-CT (Four-Dimensional Computed Tomography) respiratory motion feature extraction. This method can directly and quickly predict motion parameters characterizing the current respiratory phase from a pair of orthogonal two-dimensional X-ray projections acquired in real time during CyberKnife treatment. Then, through a pre-established personalized motion model, it reconstructs the three-dimensional image at that moment, continuously outputting a high-temporal-resolution four-dimensional dynamic image sequence.

[0022] Figure 1 This is a flowchart illustrating a method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling, according to an exemplary embodiment. Figure 1 As shown, the method includes: Step S101: Based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected, construct the initial 4D-DVF of the image deformation field between each phase and the reference phase.

[0023] Optionally, in some embodiments, step S101 above includes: Principal component analysis is used to analyze the motion of principal components in 4D-CT image sequences and generate principal component motion information. Based on the principal component motion information, a 4D-DVF (Four-Dimensional Deformation Vector Field) is constructed.

[0024] Optionally, in some embodiments, the above step of constructing a 4D-DVF based on the principal component motion information includes: Based on the principal component motion information, initial personalized reference model parameters representing respiratory motion characteristics are constructed; The parameters of the initial personalized reference model are extended by interpolation to generate 4D-DVF with more variations in motion.

[0025] Step S102: Based on the initial 4D-DVF, multiple two-dimensional orthographic image sequences with different breathing amplitudes are simulated and generated at 45° and 135° according to the CyberKnife imaging geometry. Based on the multiple two-dimensional orthographic image sequences, the Unet network architecture based on the Transformer kernel is trained to generate the target deep learning model.

[0026] In some embodiments, the target deep learning model includes a mapping relationship between multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors.

[0027] Optionally, in some embodiments, step S102 above includes: Based on multiple two-dimensional orthographic image sequences, a Unet network architecture based on the Transformer kernel is trained by setting a combined loss function to generate a target deep learning model. The loss function is obtained by weighted summation of mean squared error loss term and smoothing loss term, and the smoothing loss function includes a set threshold parameter.

[0028] For example, the deep learning model is set as a two-channel network structure model, where each channel in the two-channel network structure model corresponds to a two-dimensional projection image at a certain angle.

[0029] Optionally, in some embodiments, the Unet network architecture based on the Transformer kernel is a dual-channel network architecture, wherein each channel in the dual-channel network architecture corresponds to a two-dimensional orthographic projection image sequence at a certain angle.

[0030] Step S103: During CyberKnife treatment, multiple orthogonal dual-channel 2D projection images of the target patient's body corresponding to the area to be detected under multiple projection angle pairs are acquired in real time. Based on the orthogonal dual-channel 2D projection images, multiple spatial motion feature vectors corresponding to the current area to be detected under different projection angle pairs are determined through the target deep learning model.

[0031] Optionally, the multiple angles may include at least two orthogonal angles, 45° and 135°.

[0032] Step S104: Based on the multiple motion information contained in each spatial motion feature vector, determine the target 4D-DVF under the corresponding projection angle pair; and based on the target 4D-DVF, deform the reference phase of 4D-CT to generate multiple 3D volume images of multiple orthogonal dual-channel 2D projection images under each projection angle pair.

[0033] Step S105: Fuse multiple 3D volumetric images to generate a four-dimensional volumetric image sequence during the CyberKnife treatment process.

[0034] Using the above technical solution, based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected, an initial 4D image deformation field (4D-DVF) between each phase and the reference phase is constructed. Based on the initial 4D-DVF, multiple two-dimensional orthographic image sequences with different respiratory amplitudes are simulated at 45° and 135° according to the CyberKnife imaging geometry. Based on these multiple two-dimensional orthographic image sequences, a Unet network architecture based on the Transformer kernel is trained to generate a target deep learning model. This target deep learning model includes the mapping relationship between multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. During CyberKnife treatment... The system acquires multiple orthogonal dual-channel 2D projection images of the target area on the patient's body under multiple projection angle pairs in real time. Based on these images, a target deep learning model determines multiple spatial motion feature vectors for the current target area under different projection angle pairs. Based on the motion information contained in each spatial motion feature vector, a target 4D-DVF is determined for the corresponding projection angle pair. Then, based on the target 4D-DVF, the reference phase of the 4D-CT is deformed to generate multiple 3D volume images from the multiple orthogonal dual-channel 2D projection images under various projection angle pairs. These multiple 3D volume images are then fused to generate a sequence of four-dimensional volume images during CyberKnife treatment. This transforms abstract, overlapping two-dimensional projections into intuitive and clear three-dimensional and four-dimensional images in real time, directly displaying the three-dimensional motion trajectory and morphological changes of the target area, greatly improving the visual judgment conditions for doctors and enhancing the tracking accuracy of tumor treatment.

[0035] I. Offline Model Training Phase: (1) Construction of a personalized motion feature database: Obtain high-quality 4D-CT image sequences acquired before the patient's treatment.

[0036] By using a nonlinear image registration algorithm, the three-dimensional deformation field of each respiratory phase image relative to a reference phase (usually the 0% end-expiratory phase) is calculated, and then combined to obtain a three-dimensional deformation field sequence describing the motion of the entire respiratory cycle.

[0037] Principal component analysis (PCA) was performed on the three-dimensional deformation field sequence, and the first k principal components (e.g., k=3) were extracted as a set of basis vectors characterizing the specific respiratory movement pattern of the patient. The deformation field of each phase can be represented as a linear combination of this set of basis vectors, and the combination coefficients are the PCA coefficients.

[0038] By interpolating a limited set of original PCA coefficients (such as linear interpolation or cubic spline interpolation), a large number of PCA coefficient sets simulating different breathing amplitudes, rhythms, and morphologies are generated, thereby expanding the diversity of motion characteristics.

[0039] Using these extended PCA coefficients and PCA basis vectors, the corresponding deformation field is inverted and applied to the reference phase of the 3D CT image, thereby synthesizing a large number of 4D-CT image sequences containing continuous and variable respiratory motions, which are used to fully train the deep learning model.

[0040] 2. Training data generation: Under the condition of accurately simulating the geometry of the CyberKnife imaging system (including parameters such as X-ray source position, detector position, and pixel size), the above-synthesized extended 4D-CT sequence is subjected to orthographic projection calculations at two fixed projection angles of 45° and 135° to generate corresponding digital reconstructed image pairs.

[0041] Each pair of DRR (Digitally Reconstructed Radiograph) images (one at 45° and one at 135°) together with the PCA coefficient vectors used in the generation process constitutes a set of training samples. This constructs a large-scale, paired training dataset.

[0042] 3. Deep learning model construction and training: Model architecture: Construct a dual-channel deep learning network. Each channel is independently responsible for processing a projected image at a fixed angle (45° or 135°). The preferred input image size is [Batch Size, 1, 256, 256].

[0043] Network Core: The core feature extractor for each channel is a Transformer-based encoder, specifically a Vision Transformer or a hybrid UNet-Transformer architecture. Taking the Vision Transformer as an example, it first segments the input image into fixed-size (e.g., 16x16) image blocks and maps each block to an embedding vector through linear projection. Then, learnable positional encodings and classification tokens (CLS) are added. This encoder consists of L (e.g., L=6) cascaded Transformer Blocks.

[0044] Transformer Block: Each block contains layer normalization, multi-head self-attention mechanism, residual connection, another layer normalization, multilayer perceptron, and a second residual connection. MLP (Multi-Leaf Collimator) is usually composed of two linear layers and GELU activation function, with dropout layers added in between to prevent overfitting.

[0045] Feature Fusion and Output: The two-channel Transformer encoders extract global motion features from their respective projections. The integrated motion information is contained in the [CLS] token features output by each encoder. The two [CLS] feature vectors can be fused by concatenation or weighted averaging, and finally, a lightweight fully connected layer (MLP head) is used to regress and predict a PCA coefficient vector of dimension [B, k].

[0046] Loss function: A dynamically weighted combined loss function is used to supervise model training. This function aims to simultaneously ensure prediction accuracy and robustness. In a preferred embodiment, the loss function is calculated as follows:

[0047] in: (yc,h,w) represent the values ​​of the model's predicted output and the true label at channel c and spatial location (h, w), respectively.

[0048] (W) C ) is the weighting coefficient for the c-th channel, used to balance the importance of different channels (or different PCA coefficient components).

[0049] (N=C×H×W) represents the total number of feature elements.

[0050] (α) is a dynamic adjustment coefficient used to control the ratio between the mean square error term and the smoothing L1 loss term.

[0051] The smoothL1 function is defined as follows:

[0052] Wherein, (β) is the smoothing threshold, which is preferably set to 0.1 in this invention.

[0053] This loss function is equivalent to the target vector ultimately predicted by the network. The sum of the weighted mean square error and the smoothed L1 loss calculated from the PCA coefficients and the true vector (P) can be summarized as follows:

[0054] II. Online Real-Time Prediction Phase This phase runs in real time during CyberKnife treatment, generating dynamic four-dimensional images.

[0055] 1. At the start of treatment, the system loads the personalized deep learning model and PCA motion basis vectors trained for the patient.

[0056] 2. During the treatment, the imaging system simultaneously acquires two two-dimensional X-ray projection images of the patient in the 45° and 135° directions in real time.

[0057] 3. After performing necessary preprocessing on the projected image (such as normalization and cropping), input it into the trained dual-channel deep learning model.

[0058] 4. The model performs forward inference and outputs the predicted PCA coefficients for the current time step in real time. .

[0059] 5. Utilize the patient's personalized PCA basis vectors obtained in the offline phase. Reconstruct the three-dimensional deformation field at the current time (t):

[0060] in, It is the prediction vector The (i)th component.

[0061] 6. Apply the reconstructed three-dimensional deformation field (DVFt) to the patient's reference phase three-dimensional CT image (CTref) (usually 0% phase), and generate the three-dimensional volumetric image (CT) corresponding to the current time (t) using an image deformation algorithm. t ).

[0062] 7. Continuously process each real-time acquired projection image pair, repeating steps 3-6, to generate and display a continuous, dynamic four-dimensional volumetric image sequence (CT) in real time. t |t=1,2,……), visually demonstrating the movement of the target area with respiration.

[0063] For example, in some embodiments, the deep learning model preferably employs a UNet-Transformer hybrid architecture, which utilizes the encoder-decoder structure of UNet to capture multi-scale local features while introducing a Transformer module to enhance global dependency modeling. Experiments on digital phantoms show that this hybrid architecture outperforms pure CNN or pure Transformer models in prediction accuracy.

[0064] To fundamentally improve the clinical applicability of the model, the 4D-CT data preferably employs a high-resolution scanning protocol, significantly increasing the image slice thickness from the conventional 2.5mm to approximately 0.6mm. This greatly reduces the resolution gap between the DRR used for training and the online measured X-ray projection, ensuring that the model learns motion features consistent with the real scene, thereby improving the accuracy and stability of predictions.

[0065] The reference phase CT can be selected from a relatively stable time period in the respiratory cycle with the best image quality, such as 0% (end of expiration) or 50% (end of inspiration) phase.

[0066] Compared with the prior art, the present invention has the following significant advantages: 1. Highly intuitive, enhancing clinical decision-making: It transforms abstract, overlapping two-dimensional projections into intuitive and clear three-dimensional / four-dimensional images in real time, directly displaying the three-dimensional motion trajectory and morphological changes of the target area, greatly improving doctors' visual judgment and potentially enhancing the accuracy of treatment tracking.

[0067] 2. High prediction accuracy: Utilizing the powerful global feature modeling capabilities of advanced architectures such as Transformer, deep motion information can be effectively extracted from severely occluded 2D projections; combined with a PCA motion model based on individual 4D-CT, high-precision personalized motion parameter prediction is achieved. Digital phantom experiments show that the images predicted by the method of this invention are almost indistinguishable from the gold standard.

[0068] 3. Excellent real-time performance and seamless integration: The deep learning model has a fast inference speed, which can meet the needs of real-time image generation during treatment (typically reaching several to tens of frames per second). The entire solution is based entirely on the existing CyberKnife imaging system, without increasing the patient's additional radiation dose or the hospital's hardware costs, and is easy to clinically translate and integrate.

[0069] 4. Balancing Personalization and Robustness: Model training based on the patient's own 4D-CT data captures their unique and non-ideal respiratory movement patterns, resulting in stronger generalization ability. The use of high-resolution 4D-CT further enhances the method's robustness to real-world clinical scenarios.

[0070] Example 1: Verification of Algorithm Principle Based on 4D-NCAT Digital Phantom This embodiment aims to verify the effectiveness and superiority of the core algorithm of the present invention under ideal conditions.

[0071] 1. Data Preparation: Using a 4D-NCAT digital human phantom, a 4D-CT image sequence containing 10 phases was generated (image matrix 512x512x120, voxel size 0.97x0.97x0.625 mm³). Using professional DRR simulation software, the imaging geometry parameters of the hospital's CyberKnife system were accurately simulated to generate high-quality DRR images (1024x1024 pixels) in the 45° and 135° directions as training and testing data.

[0072] 2. Model Training and Comparison: Following the method described in this invention, the 4D-DVF of the phantoms was extracted and PCA analysis was performed (retaining the first three principal components). 2000 sets of extended PCA coefficients and corresponding DRR image pairs were generated. CNN, ResNet, UNet, Vision Transformer, and UNetTransformer models were constructed and trained respectively. All models were trained end-to-end using the aforementioned dynamically weighted combined loss function (set α=0.7, β=0.1).

[0073] 3. Results Analysis: Evaluation was conducted on an independent test set. The example UNetTransformer model performed optimally on all quantitative metrics (such as mean absolute error and correlation coefficient R²), and its predicted 4D image sequences were visually highly consistent with the gold standard images, with a prediction error of less than 1.5 mm for the 3D motion trajectory of the target center point. These results fully demonstrate the effectiveness and advancement of the proposed method framework and network architecture.

[0074] Example 2: High-resolution data strategy for clinical applications This embodiment proposes solutions to key bottlenecks in current clinical application and anticipates their effects.

[0075] 1. Problem Analysis: When applying this invention to real patients, a major challenge lies in the "domain difference" between training and testing data. Current routine 4D-CT scans in clinical settings have a relatively thick slice thickness (approximately 2.5 mm), resulting in blurred edges and unclear details in the generated DRR images. This presents a significant quality gap compared to online-acquired, high-resolution X-ray projection images, affecting the generalization ability of the trained model.

[0076] 2. Solution: To overcome this obstacle, this invention proposes and plans to implement the following: During the patient simulation and positioning phase, spectral CT or other advanced CT scanning technologies will be used to acquire ultra-thin, high-resolution 4D-CT data with a slice thickness of approximately 0.6 mm.

[0077] 3. Expected Results: The DRR reconstructed based on this high-resolution 4D-CT data will possess clearer anatomical details and sharper edges, significantly reducing the image quality difference from the measured X-ray projection. Retraining the deep learning model of this invention using this high-quality data is expected to greatly improve the accuracy, stability, and reliability of the model in predicting PCA coefficients and subsequent 4D images from real patient projection images, thereby establishing the broad clinical applicability of this method.

[0078] Example 3: System Implementation; Image acquisition interface unit: Communicates with the CyberKnife X-ray imaging control system to acquire two-dimensional projection image streams in the 45° and 135° directions in real time and synchronously.

[0079] Data preprocessing unit: Performs standardization, contrast adjustment, and automatic or semi-automatic cropping of regions of interest on the input projected image.

[0080] Deep learning inference engine unit: Loads a pre-trained dual-channel deep learning model (such as UNetTransformer) for the current patient and performs efficient real-time forward inference computation.

[0081] Motion model library management unit: Stores and manages each patient's personalized PCA motion basis vectors ((Ui)) and reference phase CT images ((CTref)).

[0082] Real-time image reconstruction and visualization unit: Based on the PCA coefficients and motion model library output by the inference engine, this unit reconstructs the 3D deformation field and deforms the reference CT in real time to generate 3D images. This unit provides a graphical user interface to display dynamic 4D image sequences in real time using various modes such as movie looping, multi-plane reconstruction, and 4D volumetric rendering. It may also integrate auxiliary functions such as automatic target delineation and motion trajectory depiction.

[0083] Control and Logic Unit: Coordinates the workflow of each unit, manages the matching of patient data with the model, and can interact with the CyberKnife treatment control system to provide potential image guidance signals for dynamic tracking treatment.

[0084] Figure 2 This is a block diagram illustrating a real-time four-dimensional volumetric image generation device for CyberKnife based on dual-channel deep learning and individual motion modeling, according to an exemplary embodiment. Figure 2 As shown, the device 100 includes: The construction module 110 is configured to construct an initial four-dimensional image deformation field 4D-DVF between each phase and the reference phase based on the four-dimensional computed tomography (4D-CT) image sequence corresponding to the area to be detected on the target patient's body. The generation module 120 is configured to simulate and generate multiple two-dimensional orthographic image sequences with different breathing amplitudes at 45° and 135° based on the initial 4D-DVF and the imaging geometry of the CyberKnife. Based on the multiple two-dimensional orthographic image sequences, the Unet network architecture based on the Transformer kernel is trained to generate a target deep learning model. The target deep learning model includes the mapping relationship between the multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. The first determining module 130 is configured to, during the CyberKnife treatment process, acquire in real time multiple orthogonal dual-channel 2D projection images of the target patient's body corresponding to the area to be detected under multiple projection angle pairs, and determine multiple spatial motion feature vectors of the current area to be detected under different projection angle pairs through the target deep learning model based on the orthogonal dual-channel 2D projection images. The second determining module 140 is configured to determine the target 4D-DVF under the corresponding projection angle pair based on the multiple motion information contained in each spatial motion feature vector; and deform the reference phase of 4D-CT based on the target 4D-DVF to generate multiple 3D volume images of multiple orthogonal dual-channel 2D projection images under each projection angle pair. The execution module 150 is configured to fuse multiple 3D volumetric images to generate a sequence of four-dimensional volumetric images during the CyberKnife treatment process.

[0085] Using the above technical solution, based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected, an initial 4D image deformation field (4D-DVF) between each phase and the reference phase is constructed. Based on the initial 4D-DVF, multiple two-dimensional orthographic image sequences with different respiratory amplitudes are simulated at 45° and 135° according to the CyberKnife imaging geometry. Based on these multiple two-dimensional orthographic image sequences, a Unet network architecture based on the Transformer kernel is trained to generate a target deep learning model. This target deep learning model includes the mapping relationship between multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. During CyberKnife treatment... The system acquires multiple orthogonal dual-channel 2D projection images of the target area on the patient's body under multiple projection angle pairs in real time. Based on these images, a target deep learning model determines multiple spatial motion feature vectors for the current target area under different projection angle pairs. Based on the motion information contained in each spatial motion feature vector, a target 4D-DVF is determined for the corresponding projection angle pair. Then, based on the target 4D-DVF, the reference phase of the 4D-CT is deformed to generate multiple 3D volume images from the multiple orthogonal dual-channel 2D projection images under various projection angle pairs. These multiple 3D volume images are then fused to generate a sequence of four-dimensional volume images during CyberKnife treatment. This transforms abstract, overlapping two-dimensional projections into intuitive and clear three-dimensional and four-dimensional images in real time, directly displaying the three-dimensional motion trajectory and morphological changes of the target area, greatly improving the visual judgment conditions for doctors and enhancing the tracking accuracy of tumor treatment.

[0086] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0087] Figure 3 This is a block diagram illustrating an electronic device 300 according to an exemplary embodiment. Figure 3 As shown, the electronic device 300 may include a processor 301 and a memory 302. The electronic device 300 may also include one or more of a multimedia component 303, an input / output (I / O) interface 304, and a communication component 305.

[0088] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps in the aforementioned method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, as well as application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 302 or transmitted via the communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or a combination thereof. Therefore, the corresponding communication component 305 may include a Wi-Fi module, a Bluetooth module, and an NFC module.

[0089] In an exemplary embodiment, the electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-described method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling.

[0090] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the above-described method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling. For example, the computer-readable storage medium may be the memory 302 including the program instructions, which may be executed by the processor 301 of the electronic device 300 to complete the above-described method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling.

[0091] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described method for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling.

[0092] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described method for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling.

[0093] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction.

[0094] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling, characterized in that, The method includes: Based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected, an initial 4D-DVF between each phase and the reference phase is constructed. Based on the initial 4D-DVF, multiple two-dimensional orthographic image sequences with different breathing amplitudes are simulated and generated at 45° and 135° based on the CyberKnife imaging geometry. Based on the multiple two-dimensional orthographic image sequences, a model is trained on the Unet network architecture based on the Transformer kernel to generate a target deep learning model. The target deep learning model includes the mapping relationship between multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. During CyberKnife treatment, multiple orthogonal dual-channel 2D projection images of the target area of ​​the patient's body under multiple projection angle pairs are acquired in real time. Based on the orthogonal dual-channel 2D projection images, the target deep learning model determines multiple spatial motion feature vectors of the current target area under different projection angle pairs. Based on the multiple motion information contained in each spatial motion feature vector, the target 4D-DVF under the corresponding projection angle is determined; and based on the target 4D-DVF, the reference phase of the 4D-CT is deformed to generate multiple 3D volume images of multiple orthogonal dual-channel 2D projection images under each projection angle. The multiple 3D volumetric images are fused to generate a four-dimensional volumetric image sequence during the CyberKnife treatment process.

2. The method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling according to claim 1, characterized in that, The construction of the initial four-dimensional image deformation field (4D-DVF) between each phase and the reference phase, based on the four-dimensional computed tomography (4D-CT) image sequence corresponding to the area to be detected on the target patient's body, includes: Principal component analysis is used to analyze the motion of principal components in the 4D-CT image sequence, generating principal component motion information. The 4D-DVF is constructed based on the principal component motion information.

3. The method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling according to claim 2, characterized in that, The step of constructing the 4D-DVF based on the principal component motion information includes: Based on the principal component motion information, initial personalized reference model parameters characterizing respiratory motion features are constructed; The initial personalized reference model parameters are interpolated and expanded to generate the 4D-DVF with more variations in motion.

4. The method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling according to claim 1, characterized in that, The Unet network architecture based on the Transformer kernel is a dual-channel network architecture, wherein each channel in the dual-channel network architecture corresponds to a two-dimensional orthographic projection image sequence at a certain angle.

5. The method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling according to claim 1, characterized in that, The plurality of projection angles include at least two orthogonal angles, 45° and 135°.

6. The method for generating real-time four-dimensional volumetric images of a CyberKnife based on dual-channel deep learning and individual motion modeling according to claim 1, characterized in that, The step of training a Unet network architecture based on a Transformer kernel to generate a target deep learning model based on the multiple two-dimensional orthographic image sequences includes: Based on the multiple two-dimensional orthographic image sequences, the Unet network architecture based on the Transformer kernel is trained by setting a combined loss function to generate the target deep learning model. The set combined loss function is obtained by weighted summation of the mean squared error loss term and the smoothing loss term.

7. A real-time four-dimensional volumetric image generation device for CyberKnife based on dual-channel deep learning and individual motion modeling, characterized in that, The device includes: The construction module is configured to construct an initial 4D-DVF between each phase and the reference phase based on the 4D-CT image sequence of the target patient's body corresponding to the area to be detected; The generation module is configured to simulate and generate multiple two-dimensional orthographic image sequences with different breathing amplitudes at 45° and 135° based on the initial 4D-DVF and the CyberKnife imaging geometry, and to train a model based on the Unet network architecture with Transformer kernel based on the multiple two-dimensional orthographic image sequences to generate a target deep learning model. The target deep learning model includes the mapping relationship between the multiple two-dimensional orthographic image sequences and multiple spatial motion feature vectors. The first determining module is configured to, during the CyberKnife treatment process, acquire in real time multiple orthogonal dual-channel 2D projection images of the target patient's body corresponding to the area to be detected under multiple projection angle pairs, and determine multiple spatial motion feature vectors of the current area to be detected under different projection angle pairs based on the orthogonal dual-channel 2D projection images and the target deep learning model. The second determining module is configured to determine the target 4D-DVF under the corresponding projection angle pair based on the multiple motion information contained in each spatial motion feature vector; and deform the reference phase of the 4D-CT according to the target 4D-DVF to generate multiple 3D volume images of multiple orthogonal dual-channel 2D projection images under each projection angle pair. The execution module is configured to fuse the multiple 3D volumetric images to generate a four-dimensional volumetric image sequence during the CyberKnife treatment process.

8. A CyberKnife system, characterized in that, The device includes an image processing apparatus configured to perform the method for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for generating real-time four-dimensional volumetric images of CyberKnife based on dual-channel deep learning and individual motion modeling as described in any one of claims 1-7.

10. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

Images