A method, system, electronic device and storage medium for predicting the applicability of a deep inhalation breath hold radiotherapy method

Abstract

The application discloses a kind of applicability prediction methods based on deep inspiration breath hold radiotherapy method, comprising: the original CT three-dimensional volume data is input into 3D deformation perception double branch network, input layer is input into shared encoder, obtains three-dimensional displacement field after being processed through deformation branch decoder and final displacement field prediction head, while being processed through residual branch decoder and obtaining HU difference residual feature map;Space transformer processes three-dimensional displacement field and generates distortion image;Residual correction module processes distortion image, HU difference residual feature map and original data and obtains HU correction residual map;Output layer outputs predicted deep inspiration breath hold CT data;According to the heart dose output DIBH response information calculated from original CT three-dimensional volume data and predicted deep inspiration breath hold CT data.The application is more in line with the physiological nature of chest CT, to generate high-quality and anatomically logical synthetic DIBH-CT.The application also discloses a system, electronic equipment and storage medium for implementing the above method.

Description

Technical Field

[0001] This invention relates to the field of computer data processing technology, and in particular to a suitability prediction method, system, electronic device, and storage medium based on the deep inhalation and breath-holding radiotherapy method. Background Technology

[0002] Breast cancer is the most common malignant tumor among women worldwide, and radiotherapy for left breast cancer presents particular challenges in clinical practice. Radiotherapy, as a core component of comprehensive breast cancer treatment, plays an irreplaceable role in effectively controlling local tumor recurrence and improving survival rates. However, during radiotherapy for left breast cancer, while irradiating the breast and chest wall target area, radiation inevitably affects adjacent vital organs, particularly the heart, ipsilateral lung tissue, and critical structures such as the coronary arteries (e.g., the left anterior descending artery). Studies have shown that even at low doses, the risk of long-term radiation damage to organs such as the heart remains significant, potentially inducing serious complications such as radiation-induced heart disease, coronary artery disease, and radiation pneumonitis, thereby increasing the patient's long-term mortality risk and disease burden.

[0003] To reduce the radiation dose to the heart, the Deep Inspiration Breath Hold (DIBH) technique has been gradually adopted in clinical practice. The basic principle of this technique is to guide the patient to hold their breath after a deep inhalation, allowing the lungs to fully expand. This pushes the heart downwards and backwards away from the chest wall, increasing the physical distance between the heart and the irradiation field, thereby reducing the volume and dose of the heart being irradiated. Current clinical data indicate that for some patients, the DIBH technique can significantly reduce the average dose and high-dose irradiation volume of the heart, demonstrating clear organ protection value.

[0004] To assess the potential benefits of DIBH, current standard clinical procedures require acquiring two sets of simulated CT images of the patient, one in a free-breathing (FB) state and the other in a deep-inspiratory breath-holding (DIBH) state. This not only means that the patient needs to undergo two CT scans, increasing the risk of radiation exposure, but also exacerbates patient fatigue, discomfort, and psychological burden due to the prolonged scanning process. Especially for some elderly, frail, or patients with respiratory diseases, the repetitive positioning, breathing coordination training, and scanning process may induce anxiety or even respiratory discomfort, affecting the smooth progress of treatment.

[0005] Secondly, the success rate and quality of image acquisition during DIBH (Digital Invasive Hemorrhage) are highly dependent on the patient's respiratory coordination, exhibiting significant randomness and uncertainty. Due to individual differences in patients' understanding and execution of breath-holding commands, and the difficulty in maintaining consistent depth and stability of breath-holding each time, DIBH-CT scans often require multiple attempts to obtain acceptable images that meet the design requirements. For patients with poor respiratory coordination, repeated scanning attempts not only prolong positioning time but also easily lead to problems such as decreased image quality and organ position drift, and may even ultimately force the abandonment of the technique due to the inability to obtain stable DIBH images, resulting in a waste of preliminary preparation work. Furthermore, technicians need to invest a significant amount of time in respiratory training guidance and scan monitoring, further exacerbating the strain on clinical human resources. Summary of the Invention

[0006] To address the aforementioned problems in the prior art, this invention provides a suitability prediction method, system, electronic device, and storage medium based on a deep inspiration breath-hold radiotherapy method. The technical problem to be solved by this invention is achieved through the following technical solution: The first aspect of this invention provides a method for predicting the suitability of a deep-inspiratory breath-hold radiotherapy method, comprising the following steps: Preprocessed clinical free-breathing CT 3D volumetric data is input into a trained 3D deformation-sensing dual-branch network for processing to obtain predicted deep inspiration-breath-hold CT data. The 3D deformation-sensing dual-branch network includes: an input layer, a shared encoder, a deformation branch decoder, a final displacement field prediction head, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer. The method for obtaining predicted deep inspiration-breath-hold CT data includes: The input layer inputs the preprocessed clinical free breathing CT three-dimensional volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale three-dimensional feature maps. The deformation branch decoder and the final displacement field prediction head process the CT multi-scale three-dimensional feature map to obtain a three-channel CT three-dimensional displacement field. At the same time, the residual branch decoder performs upsampling, corresponding skip connection splicing and residual fusion processing on the CT multi-scale three-dimensional feature map to obtain the HU difference residual feature map. The spatial transformer normalizes the CT three-dimensional displacement field and performs three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image. The residual correction module stitches the distorted image and the HU difference residual feature map together and then performs residual learning to obtain a HU correction residual map for one channel. The output layer fuses the distorted image and the HU correction residual map to obtain the predicted deep inhalation breath-hold CT data. The cardiac dose is calculated based on the clinical free breathing CT three-dimensional volume data and the predicted deep inspiration and breath-holding CT data, and the DIBH response information is output based on the cardiac dose.

[0007] In one embodiment of the present invention, it further includes: Preprocessing of the raw training data during training of the 3D deformation-aware dual-branch network includes: The collected paired training free breathing CT three-dimensional volume data and training deep inspiration breath-holding CT three-dimensional volume data were used as the original training data; The training free breathing CT three-dimensional volume data and the training deep inspiration breath-holding CT three-dimensional volume data were resampled, rigidly registered, ROI clipped and normalized to obtain spatially registered training data. The training data is divided into a training set, a validation set, and a test set; among which, the spatially registered training deep inspiration breath-hold CT three-dimensional volume data is used as label data.

[0008] In one embodiment of the present invention, the shared encoder includes: a first coding layer, a second coding layer, a third coding layer, a fourth coding layer, and a bottleneck layer; The input layer inputs the preprocessed clinical free-breathing CT 3D volumetric data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale 3D feature maps, including: The first coding layer extracts 16-channel feature maps from the preprocessed clinical free breathing CT three-dimensional volume data and outputs the first coded feature map. The second coding layer extracts a 32-channel feature map based on the first coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the second coding feature map. The third coding layer extracts a 64-channel feature map based on the second coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the third coding feature map. The fourth coding layer extracts a 128-channel feature map based on the third coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the fourth coding feature map. The bottleneck layer extracts 256-channel feature maps based on the fourth encoded feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs a CT multi-scale three-dimensional feature map.

[0009] In one embodiment of the present invention, the deformation branch decoder includes: a fourth deformation branch decoding layer, a third deformation branch decoding layer, a second deformation branch decoding layer, and a first deformation branch decoding layer; The deformation branch decoder and the final displacement field prediction head process the CT multi-scale three-dimensional feature map to obtain a three-channel CT three-dimensional displacement field, including: The fourth deformation branch decoding layer upsamples the CT multi-scale three-dimensional feature map, then concatenates it with the fourth encoded feature map input by the skip connection, and then performs residual fusion and channel compression on the concatenated features to obtain a 128-channel fourth decoded feature map. The third deformation branch decoding layer upsamples the fourth decoding feature map, then concatenates it with the CT multi-scale three-dimensional feature map, and then concatenates it with the third encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 64-channel third decoding feature map. The second deformation branch decoding layer upsamples the third decoding feature map, then concatenates it with the CT multi-scale three-dimensional feature map, and then concatenates it with the second encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 32-channel second decoding feature map. The first deformation branch decoding layer upsamples the second decoding feature map, then concatenates it with the CT multi-scale three-dimensional feature map, and then concatenates it with the first encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 16-channel first decoding feature map. The final displacement field prediction head performs a three-dimensional convolution operation on the first decoded feature map, mapping the number of feature channels to 3, to obtain a three-channel CT three-dimensional displacement field.

[0010] In one embodiment of the present invention, it further includes: a fourth displacement field prediction head, a third displacement field prediction head, and a second displacement field prediction head; When training the 3D deformation-aware dual-branch network, the fourth displacement field prediction head performs a three-dimensional convolution operation on the fourth decoded feature map to obtain a three-channel fourth three-dimensional displacement field. The third displacement field prediction head performs a three-dimensional convolution operation on the third decoded feature map, mapping the number of feature channels to 3, to obtain a 3-channel third three-dimensional displacement field; The second displacement field prediction head performs a three-dimensional convolution operation on the second decoded feature map, mapping the number of feature channels to 3, and obtaining a 3-channel second three-dimensional displacement field. The loss of auxiliary supervision is calculated based on the fourth three-dimensional displacement field, the third three-dimensional displacement field, and the second three-dimensional displacement field.

[0011] In one embodiment of the present invention, the 3D deformation sensing dual-branch network employs an auxiliary supervision mechanism during training to supervise the fourth, third, and second three-dimensional displacement fields during training.

[0012] In one embodiment of the present invention, the main loss function of the 3D deformation-aware dual-branch network during the training process is... for: in, This indicates the loss of image voxel difference. express The weighting coefficients, Represents the 3D structural similarity loss. express The weighting coefficients, Indicates deformation consistency loss. express The weighting coefficients, This represents the smoothing regularization loss. express The weighting coefficients, Indicates the ROI-weighted loss. express The weighting coefficients, This represents the gradient consistency loss. express Weighting coefficients.

[0013] A second aspect of this invention provides a suitability prediction system based on a deep inhalation and breath-hold radiotherapy method, comprising: The prediction module is used to input preprocessed clinical free-breathing CT 3D volumetric data into a trained 3D deformation-sensing dual-branch network for processing, to obtain predicted deep inspiration and breath-hold CT data; wherein, the 3D deformation-sensing dual-branch network includes: an input layer, a shared encoder, a deformation branch decoder, a final displacement field prediction head, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer; the prediction module includes: The input layer is used to input the preprocessed clinical free breathing CT three-dimensional volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale three-dimensional feature maps. The deformation branch decoder and the final displacement field prediction head are used to process the CT multi-scale three-dimensional feature map to obtain a three-channel CT three-dimensional displacement field. At the same time, the residual branch decoder is used to upsample the CT multi-scale three-dimensional feature map, perform corresponding skip connection splicing and residual fusion processing to obtain the HU difference residual feature map. A spatial transformer is used to normalize the CT three-dimensional displacement field and perform three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image. The residual correction module is used to stitch the distorted image and the HU difference residual feature map together and then perform residual learning to obtain a HU correction residual map for one channel. The output layer is used to fuse the distorted image and the HU correction residual map to obtain the predicted deep inspiratory breath-hold CT data. The output module is used to calculate the cardiac dose based on the clinical free breathing CT three-dimensional volume data and the predicted deep inspiration and breath-holding CT data, and output DIBH response information based on the cardiac dose.

[0014] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the suitability prediction method based on the deep inhalation breath-holding radiotherapy method provided in the first aspect of the present invention.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements an applicability prediction method based on a deep inhalation breath-holding radiotherapy method provided in the first aspect of the present invention.

[0016] The beneficial effects of this invention are: This invention utilizes a 3D Deformation-Aware Dual-Branch Network to generate a deep inspiration and breath-hold CT (DIBH-CT) from a free breathing CT (FB-CT). Patients only need one standard free breathing CT scan, eliminating the need for two scans. This avoids repeated attempts and prolonged scan times due to poor breath-holding coordination, significantly reducing patient discomfort, anxiety, and additional radiation exposure. Clinicians and physicists can quickly obtain a synthetic DIBH-CT for radiotherapy planning after receiving the FB-CT, thus shortening the total time from localization to planning and improving the overall efficiency of the radiotherapy process. The "shared encoder + dual decoder" 3D network architecture employed in this invention is not a simple end-to-end black-box mapping, but rather a mechanistic modeling of the complex physiological process of "free breathing → deep inspiration and breath-holding." The conversion from FB to DIBH is not a simple overall translation, but involves complex nonlinear motion involving multiple organs in the thoracic cavity and multi-directional coupling. First, the network learns about the three-dimensional structural deformation caused by respiration, and then it learns about local HU residual correction, thereby generating a synthetic DIBH-CT that better conforms to physiological mechanisms and clinical goals. This branching allows the network to "first learn how organs move, and then learn how density is adjusted," which is more in line with the physiological nature of chest CT, thus generating a high-quality synthetic DIBH-CT that conforms to anatomical logic. Clinical teams can conduct precise "virtual" dosimetric comparisons before patients actually undergo DIBH treatment. By rapidly evaluating the plans designed on the synthetic DIBH-CT, "high responders" with significant cardiac dose reduction and "low responders" with limited benefit can be effectively identified, thus providing objective and quantitative decision-making basis for whether to use DIBH technology for treatment, and promoting the shift of treatment strategies from "general trial" to "precision screening."

[0017] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a suitability prediction method based on a deep inhalation and breath-holding radiotherapy method provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the applicability prediction system architecture for a deep inhalation and breath-holding radiotherapy method provided in an embodiment of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0021] like Figure 1 As shown, the first aspect of this invention provides a method for predicting the suitability of a deep-inspiratory breath-hold radiotherapy method, comprising the following steps: Step 10: Input the preprocessed clinical free breathing CT three-dimensional volume data into the trained 3D deformation perception dual-branch network for processing to obtain the predicted deep inspiration breath-hold CT data.

[0022] The 3D deformation-sensing dual-branch network includes: an input layer, a shared encoder, a deformation branch decoder, a final displacement field prediction head, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer; the method for obtaining predicted deep inspiration-hold CT data includes: Step 101: The input layer inputs the preprocessed clinical free breathing CT three-dimensional volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale three-dimensional feature maps.

[0023] Step 102: The deformation branch decoder and the final displacement field prediction head process the CT multi-scale three-dimensional feature map to obtain the three-channel CT three-dimensional displacement field.

[0024] Step 103: The residual branch decoder upsamples the CT multi-scale 3D feature map, performs corresponding skip connection splicing and residual fusion processing, and obtains the HU difference residual feature map.

[0025] Step 104: The spatial transformer normalizes the CT three-dimensional displacement field and performs three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image.

[0026] Step 105: The residual correction module stitches the distorted image and the HU difference residual feature map together and then performs residual learning to obtain a residual map of the HU correction amount for one channel.

[0027] Step 106: The output layer fuses the distorted image and the HU correction residual map to obtain the predicted deep inhalation breath-hold CT data.

[0028] Step 20: Calculate cardiac dose based on clinical free breathing CT three-dimensional volume data and predicted deep inspiration and breath-holding CT data, and output DIBH response information based on cardiac dose.

[0029] Based on the first aspect of the present invention, the second aspect of the present invention provides a more detailed description of a method for predicting the suitability of a deep inhalation and breath-hold radiotherapy method. The second aspect of the present invention provides a method for predicting the suitability of a deep inhalation and breath-hold radiotherapy method, comprising the following steps: Step 31: Obtain the user's clinical free breathing CT three-dimensional volume data.

[0030] Before this step, the 3D deformation-aware dual-branch network is pre-trained. The 3D deformation-aware dual-branch network consists of: an input layer, a shared encoder, a deformation branch decoder, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer.

[0031] Specifically, the network adopts a 3D structure of "shared encoder + dual decoder", and the network architecture is as follows: Figure 2 As shown: The shared encoder is used to extract 3D anatomical semantic features, providing a shared representation for the two branches. The subsequent deformation and residual decoding branches share this encoder. Deformation Decoder is used to predict three-dimensional displacement fields. Learn organ displacement patterns; The flow head is used to convert decoded features into an interpretable displacement field output; The Spatial Transformer is used to perform differentiable three-dimensional torsion on the input CT based on the displacement field; The residual branch decoder and residual correction module are used to predict the residual correction amount of the HU value and refine the local density.

[0032] The output layer (Output Fusion) is used to add the distortion result to the residual to obtain the final synthesized CT.

[0033] Specifically, the training process of the 3D deformation-aware dual-branch network includes steps A1-A9: Step A1: Obtain paired raw training data, that is, obtain paired training free breathing CT 3D volume data (training FB-CT) and training deep inspiration and breath-hold CT 3D volume data (training DIBH-CT) as raw training data. For each patient, one FB-CT and one DIBH-CT are used.

[0034] Step A2 involves resampling, rigid registration, ROI cropping, and normalization of the original data to obtain training data for spatial registration, thereby achieving spatial registration between FB-CT and DIBH-CT.

[0035] Specifically, firstly, both types of data are resampled to a unified spacing (voxel spacing). Then, the resampled FB-CT data is rigidly registered to the DIBH coordinate system to obtain registered FB-CT data. Next, ROI clipping is performed on the registered FB-CT data and the resampled DIBH-CT data. The clipped registered data pairs (FB-CT and DIBH-CT) are linearly mapped to the [0, 1] interval using a unified HU value cutoff range ([-1000, 1000] HU) and normalization parameters.

[0036] Step A3 involves dividing the registered data into training, validation, and test sets. The spatially registered DIBH-CT 3D volumetric data serves as the label data.

[0037] Step A4: Patch the training set data.

[0038] Step A5: Enter "patch" to start training.

[0039] This step employs a two-stage course-based training approach: Phase 1 (approximately 50-100 epochs): Train only the deformation branch + Spatial Transformer, with a loss of... + The goal is to teach the network how to move the structure to the correct position first.

[0040] The second stage (approximately 100-200 epochs): Jointly train the deformation branch + residual branch (all parameters), with the loss being all loss terms, and refine the HU value based on structural alignment.

[0041] Specifically, the first phase: Step 1: Feature Extraction (Shared Encoder) Input: FB-CT patch [1xDxHxW] (single-channel 3D image patch).

[0042] Process: Downsampling through levels E1→ E2→ E3→ E4→ E5.

[0043] Output: A set of multi-scale feature maps. The deepest E5 feature map has the lowest resolution but the richest semantic information.

[0044] Step 2: Deformation branch First, upsampling and decoding: starting from E5, the spatial resolution is gradually restored through D4 → D3 → D2 → D1.

[0045] Then, the Flow Head predicts the deformation field via F4 → F3 → F2 → Final Flow Head. (dx, dy, dz) (typically 3 output channels [3xDxHxW]).

[0046] meaning: Each voxel in the data records the distance (offset in the x, y, z directions) that needs to be moved from the input image to the target image. Then comes the Spatial Transformer: Operation: Utilizing predictions Spatial interpolation (resampling) is performed on the original input FB to generate the deformed image Warp(FB, ).

[0047] Calculate loss + The parameters are updated until the loss function is minimized or the maximum epoch is reached.

[0048] Phase Two (training based on the results of Phase One): Step 1: Feature Extraction (Shared Encoder) Input: FB-CT patch [1xDxHxW] (single-channel 3D image patch).

[0049] Process: Downsampling through levels E1→ E2→ E3→ E4→ E5.

[0050] Output: A set of multi-scale feature maps. The deepest E5 feature map has the lowest resolution but the richest semantic information.

[0051] Step 2: Deformation branch First, upsampling and decoding: starting from E5, the spatial resolution is gradually restored through D4 → D3 → D2 → D1.

[0052] Then, the Flow Head predicts the deformation field via F4 → F3 → F2 → Final Flow Head. (dx, dy, dz) (typically 3 output channels [3xDxHxW]).

[0053] Then comes the Spatial Transformer: Operation: Utilizing predictions Spatial interpolation (resampling) is performed on the original input FB to generate the deformed image Warp(FB, ).

[0054] Step 3: Residual branching is parallel to step 2.

[0055] Starting from E5, the system upsamples layer by layer through R4 → R3 → R2 → R1 to output a high-resolution residual feature map, which is used to correct the HU prediction of the main branch. The multiple inputs are concatenated in the channel dimension through Residual Refinement, and then processed by residual convolutional blocks to finally output a 1-channel 3D residual volume R (predicted HU residual).

[0056] The core of residual branching logic lies in predicting "how much more HU value needs to be added" instead of directly generating a complete CT scan. The deformed image already contains most of the correct structure; what remains are mainly local density deviations and detail restoration. Residual learning is more stable than regressing the target volume data from zero.

[0057] Step 4: Output Fusion Formula: sCT = Warp(FB, ) + R Warp(FB, R: Macroscopic spatial location matching is completed, and R: Microscopic pixel value correction is completed. The two are added together to obtain the final synthetic image sCT, which is the predicted image. It is aligned with the real DIBH-CT in spatial structure and the pixel values ​​are closer to the real values.

[0058] Step 5: Tag Comparison Tags: Real DIBH-CT patch [1xDxHxW] Objective: To compare the generated sCT with the real DIBH-CT, calculate the principal loss, and guide network training.

[0059] Main loss function during training for: in, This indicates the loss of image voxel difference. express The weighting coefficients, Represents the 3D structural similarity loss. express The weighting coefficients, Indicates deformation consistency loss. express The weighting coefficients, This represents the smoothing regularization loss. express The weighting coefficients, Indicates the ROI-weighted loss. express The weighting coefficients, This represents the gradient consistency loss. express Weighting coefficients.

[0060] Image reconstruction loss Used to constrain the voxel differences between the final sCT and the true DIBH-CT. 3DSSIM or local structural similarity loss is used. Enhance the consistency of organ boundaries and overall structure. Deformation monitoring loss. Used to constrain the geometric consistency between the distorted image and the target image, it encourages the deformation branch to align the general structure first, rather than leaving all the responsibility to the residual branch. Deformation smoothing loss To constrain the spatial continuity of the displacement field, stronger deformation regularization forms such as bending energy can also be used. ROI-weighted loss. Higher weights are assigned to critical areas such as the heart, lungs, and chest wall. Boundary gradient loss. This is used to improve the fidelity of structural edges, especially the boundaries of the heart, lung fields, and chest wall. Formally, the loss can be calculated from the gradient map, which helps reduce over-smoothing problems. This term is crucial because without smoothing constraints, the network might learn deformation fields that are severely distorted locally and do not conform to physiological mechanisms. ROI-weighted loss ensures that the network focuses more on clinically critical regions, rather than simply pursuing the optimal average pixel count for the entire image.

[0061] Preferably, the inference strategy should employ: sliding window inference, patch overlap fusion, and on-test image enhancement (optional). For an 8GB VRAM inference environment, if the patch size is designed reasonably, the sliding window approach can usually run stably.

[0062] The data operation process of each module in the above training process is the same as the data operation process in steps 33-37 of the actual application reasoning stage below, and will not be repeated here.

[0063] It should be noted that the 3D deformation-aware dual-branch network also employs an auxiliary supervision mechanism during training. This mechanism supervises the multi-scale 3D displacement fields output by the fourth, third, and second displacement field prediction heads (Flow Heads F4, F3, and F2). The main loss supervises the final output sCT, while the auxiliary supervision mechanism constrains the intermediate outputs of the deformation branches. The auxiliary supervision is achieved by calculating the multi-scale downsampled... The similarity loss with the corresponding scale DIBH-CT is backpropagated to F4, F3, and F2 to constrain the estimation of the displacement field at each scale. This mechanism ensures that the deep decoder (D4-D2) of the deformation branch first aligns the gross structure at multiple scales, rather than leaving all geometric correction responsibility to the residual branch.

[0064] Here, the coarse / medium / fine-scale deformation fields output by Flow Head F4, Flow Head F3, and Flow Head F1 only participate in the auxiliary supervision loss (Aux Supervision). Multi-scale deformation consistency constraints ensure that the deep decoder learns reasonable deformation features. The final Flow Head outputs the full-resolution deformation field. As the sole inference output, it drives the Spatial Transformer to perform geometric alignment from the FB-CT to the DIBH-CT coordinate system. This design enhances gradient propagation and training stability through multi-scale supervision during the training phase, while using only a single-scale deformation field during the inference phase, balancing accuracy and computational efficiency.

[0065] Specifically, the auxiliary supervision mechanism includes three supervision strategies: Multi-scale flow loss, Smoothness regularization, and Optional ROI weighting.

[0066] The working process of the auxiliary supervision mechanism is as follows: Step B1: The features output from the fourth deformation branch decoding layer Def Decoder D4 are input into the fourth displacement field prediction layer Flow Head F4 for 3D convolution, and the number of feature channels is mapped to 3 to obtain a 3-channel fourth 3D displacement field. The Flow Head F4 outputs a 3-channel flow, coarse scale, which is the coarse-grained displacement field predicted for the deepest layer, capturing the overall thoracic expansion.

[0067] Step B2: The features output from the third deformation branch decoding layer Def Decoder D3 are input into the third displacement field prediction layer Flow Head F3 for 3D convolution operation, and the number of feature channels is mapped to 3 to obtain a 3-channel third 3D displacement field. The Flow Head F3 outputs a 3-channel flow at a mid-scale, representing a mesoscale displacement field that refines organ-level motion.

[0068] Step B3: The features output from the second deformation branch decoding layer Def Decoder D2 are input into the second displacement field prediction layer Flow Head F2 for 3D convolution, and the number of feature channels is mapped to 3 to obtain a 3-channel second 3D displacement field. The Flow Head F2 outputs a 3-channel flow, fine-scale, for predicting a more detailed displacement field.

[0069] Step B4: Calculate the multi-scale flow loss.

[0070] in, These correspond to F2, F3, and F4 respectively. This represents the scale weights, corresponding to the weights of the three scales. Generally, > > (Coarse-scale has higher weight). Indicates the index coordinates or spatial position of a voxel. The weights after ROI mask downsampling Represents a space transformer. Indicates downsampling operation, scale factor , This represents preprocessed clinical free-breathing CT three-dimensional volumetric data. This represents the three-dimensional displacement fields at three scales output by F2, F3, and F4, respectively. This represents the preprocessed, real DIBH-CT 3D volume data. express It is in the i Image domains at various scales (e.g., F2, F3, F4) Iterate through all voxel positions. This represents the numerical stability constant. For example, .

[0071] Step B5, Smoothness regularization. Smoothness regularization loss. : in, Represents the grid coordinates of a voxel (row, column, and layer).

[0072] Discrete form: in, Indicates voxel spacing, usually (CT axial resolution is low).

[0073] Step B6, ROI weighted loss (Optional ROI weighting) Among them, ROI weight Example: Preset regional importance weighting coefficients , , , .

[0074] Preferably, in addition to the three auxiliary supervision strategies mentioned above, Jacobian determinant constraints can also be selectively employed: The Jacobian matrix is ​​as follows: Ensure topology preservation (no folding) of the deformation field.

[0075] Step B7: Sum the individual losses of the auxiliary supervision to obtain the auxiliary supervision loss. : Among them, each item here This represents the corresponding weighting coefficient.

[0076] Here, the total loss of the network consists of the main loss and the auxiliary monitoring loss: Evaluation phase: The model's output sCT is compared with the real DIBH-CT, and commonly used evaluation metrics are shown in Table 1: Table 1 Step 32: Preprocess the three-dimensional volume data of clinical free breathing CT.

[0077] The preprocessing process in this step is the same as that during training. First, the data is resampled to the same spacing (voxel spacing) as the training data. Then, the resampled FB-CT data is rigidly registered to the DIBH coordinate system to obtain the registered FB-CT data. Next, the registered FB-CT data is cropped to the ROI. The cropped registered data pairs (FB-CT and DIBH-CT) are linearly mapped to the [0, 1] interval using a uniform HU value cutoff range ([-1000, 1000] HU) and normalization parameters. Then, a sliding window is used to cut the data into multiple patches, and the overlapping areas are weighted and averaged (e.g., Gaussian weights). Finally, the patch results are stitched back to the complete volume.

[0078] Step 33: Input the preprocessed clinical free breathing CT 3D volume data into the input layer of the trained 3D deformation perception dual-branch network. The input layer inputs the preprocessed clinical free breathing CT 3D volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale 3D feature maps.

[0079] In this step, the input layer receives HU value volumetric data with a three-dimensional spatial structure of (D, H, W). The input data is a single-channel 3D volume of 1 x D x H x W. The input layer is designed to take into account the special characteristics of CT data: unlike ordinary images, CT data uses HU values ​​to measure density. Example sizes are: 96x160x160, 96x192x192, or 128x160x160.

[0080] In this step, the shared encoder includes: a first coding layer, a second coding layer, a third coding layer, a fourth coding layer, and a bottleneck layer.

[0081] Specifically, the preprocessed clinical free-breathing CT three-dimensional volume data is input into a shared encoder for downsampling and convolution processing to obtain CT multi-scale three-dimensional feature maps, including steps 331-335: Step 331: The first encoding layer Encoder E1 extracts 16-channel feature maps from the preprocessed clinical free breathing CT three-dimensional volume data and outputs the first encoded feature map.

[0082] In this step, the structure of the first encoding layer, Encoder E1, includes a Conv3D 3x3x1 kernel and a ResidualBlock, with 16 output channels. This means it includes a 3D convolution kernel of size 3x3x1 and a residual block. The first encoding layer E1 uses anisotropic convolution to preserve high-resolution anatomical details. Due to the large thickness of the CT slice, no convolution is performed on the z-axis; features are extracted only within the cross-section. In the first encoding layer, preprocessed clinical free-breathing CT 3D volumetric data is processed through 3D convolution and residual block operations, outputting a first encoded feature map with 16 channels.

[0083] Step 332: The second coding layer extracts 32-channel feature maps based on the first coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the second coding feature map.

[0084] In this step, the structure of the second encoding layer, Encoder E2, includes a MaxPool 2x2x2 and a Conv3D 3x3x1, with 32 output channels. This means it includes a 2x2x2 max-pooling layer and a 3x3x1 3D convolutional kernel. The second encoding layer uses anisotropic convolution. It receives the first encoded feature map and pre-processed clinical free-breathing CT 3D volumetric data via a broadcast mechanism. The data output from Encoder E1 (the first encoded feature map) is spatially downsampled using 3D max-pooling and then fused (concat) with the pre-processed clinical free-breathing CT 3D volumetric data received via broadcast. After fusion, features are extracted using 3D convolution, resulting in a second encoded feature map with 32 channels. The second encoding layer downsamples once, halving the spatial resolution and doubling the number of channels.

[0085] Step 333: The third coding layer extracts 64-channel feature maps based on the second coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the third coding feature map.

[0086] In this step, the structure of the third encoding layer, Encoder E3, includes a MaxPool 2x2x2 and a Conv3D 3x3x3, with 64 output channels. This means it includes a 2x2x2 max-pooling layer and a 3x3x3 3D ​​convolutional kernel. The third encoding layer receives the second encoding feature map and preprocessed clinical free-breathing CT 3D volumetric data via a broadcast mechanism. The data output by Encoder E2 (the second encoding feature map) is spatially downsampled using 3D max-pooling and then fused with the preprocessed clinical free-breathing CT 3D volumetric data received via broadcast. After fusion, features are extracted using 3D convolution, resulting in a 64-channel third encoding feature map. After downsampling twice, the z-axis information is more compact, and standard 3D convolution is used to establish a 3D context.

[0087] Step 334: The fourth coding layer extracts 128-channel feature maps based on the third coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the fourth coding feature map.

[0088] In this step, the structure of the fourth encoding layer, Encoder E4, includes a MaxPool 2x2x2 and a Conv3D 3x3x3, with 128 output channels. This means it includes a 2x2x2 max-pooling layer and a 3x3x3 3D ​​convolutional kernel. The fourth encoding layer receives the third encoding feature map and preprocessed clinical free-breathing CT 3D volumetric data via a broadcast mechanism. The data output from Encoder E3 (the third encoding feature map) is spatially downsampled using 3D max-pooling and then fused with the preprocessed clinical free-breathing CT 3D volumetric data received via broadcast. After fusion, features are extracted through 3D convolution, resulting in a 128-channel fourth encoding feature map. Here, three downsampling operations are performed to obtain a large receptive field, deeply capturing global 3D deformation patterns.

[0089] In this embodiment, the encoder channels change to 1-16-32-64-128. The initial number of channels is set to 16 instead of the 64 commonly used in 2D networks because 3D feature maps have an added depth dimension, leading to an exponential increase in memory overhead. The lightweight design ensures efficient training even with limited memory. E1 and E2 use 3x3x1 anisotropic convolutions because CT scans have a relatively large slice thickness (typically 2-5 mm) in the z-axis direction and high resolution in the xy-plane (typically 1x1 mm), making direct convolution in the z-axis direction prone to introducing noise. After two downsampling operations, the z-axis information is more compact, and E3 and E4 use standard 3x3x3 convolutions to establish the three-dimensional context.

[0090] Step 335: The bottleneck layer extracts 256-channel feature maps based on the fourth encoded feature map and the preprocessed clinical free breathing CT 3D volume data, outputting a CT multi-scale 3D feature map. The bottleneck layer is located at the deepest part of the encoder and serves as a bridge connecting the encoding and decoding areas.

[0091] In this step, the bottleneck layer, Bottleneck E5, consists of a MaxPool 2x2x2 and a Residual convblock, with 256 output channels. This means it includes a 2x2x2 window max-pooling layer and a residual convolutional block. Bottleneck E5 receives the fourth encoded feature map and pre-processed clinical free-breathing CT 3D volumetric data via a broadcast mechanism. The data output from Encoder E4 (the fourth encoded feature map) is spatially downsampled using 3D max-pooling and then fused with the pre-processed clinical free-breathing CT 3D volumetric data received via broadcast. After fusion, a residual convolution operation is performed to output a 256-channel CT multi-scale 3D feature map. Here, downsampling occurs once, for a total of four downsampling operations, obtaining a minimum feature map with 1 / 16 resolution to maximize the receptive field. Then, a residual convolutional block operation is used to alleviate gradient vanishing and ensure deep trainingability, outputting a 256-channel CT multi-scale 3D feature map. The peak number of output channels is achieved through spatial channel swapping, encoding the richest 3D semantic representation.

[0092] Here, the internal structure of the residual convolution block is two 3x3x3 convolutions plus a skip connection, specifically [Conv3D 3x3x3 →InstanceNorm / BatchNorm → LeakyReLU] → [Conv3D3x3x3 →InstanceNorm / BatchNorm] → [Skip connection (identity mapping)] → [Element-wise Add] → [LeakyReLU].

[0093] Bottleneck E5 is the information compression and distribution center of the entire network—compressing the pixel-level information of the original CT scan into a high-dimensional representation with 256 semantic channels, and then distributing it to the deformation and residual branches, which respectively complete the tasks of "learning motion" and "learning density". It plays a key role in the dual-branch architecture, connecting the upper and lower branches and providing a single source for dual outputs.

[0094] In this embodiment, the shared encoder adopts a lightweight 3D U-Net structure, with both branches sharing the same set of coding features, significantly reducing the number of parameters. The core task of the encoder is to extract multi-scale three-dimensional anatomical semantic features layer by layer from the input CT volume data. The shared encoder fuses the preprocessed clinical free breathing CT three-dimensional volume data (raw data) input through a broadcast mechanism, allowing the deep encoder to retain the original voxel information, which is beneficial for the subsequent fine estimation of the deformation field.

[0095] Step 34, execute steps 341 and 342 in parallel: Step 341: The deformation branch decoder and the final displacement field prediction head process the CT multi-scale three-dimensional feature map to obtain the three-channel CT three-dimensional displacement field, and then step 35 is executed.

[0096] The deformation branch decoder and the final displacement field prediction head serve as deformation branches. The deformation branch decoder adopts a 3D decoder structure and is connected to the shared encoder via a Skip Connection. During the decoding process, the spatial resolution is restored layer by layer, and the high-resolution features of the corresponding level of the encoder are fused together.

[0097] The deformation branch decoder includes: a fourth deformation branch decoding layer, a third deformation branch decoding layer, a second deformation branch decoding layer, and a first deformation branch decoding layer.

[0098] The specific steps of step 341 include: steps 3411-3415: Step 3411: The fourth deformation branch decoding layer upsamples the CT multi-scale 3D feature map, then concatenates it with the fourth encoded feature map input by the skip connection, and then performs residual fusion and channel compression on the concatenated features to obtain a 128-channel fourth decoded feature map.

[0099] In this step, the operation in the fourth deformation branch decoding layer Def Decoder D4 is UpConv + Concatskip E4 + Residual fuse. That is, the output of Bottleneck E5 is input to Def Decoder D4, and the feature skipped after the 3D upsampling convolution (UpConv) operation is connected to the output of Encoder E4 (the fourth encoded feature map) to obtain the concatenated feature map. The concatenated feature map is then subjected to a non-linear transformation operation through a residual block to smoothly fuse "high-level semantics (from E5)" and "low-level details (from E4)". Finally, the channels are compressed to 128 to obtain the fourth decoded feature map.

[0100] In step 3412, the third deformation branch decoding layer upsamples the fourth decoding feature map, then concatenates it with the CT multi-scale 3D feature map. After concatenation, it is concatenated with the third encoded feature map input by the skip connection. Then, the concatenated features are subjected to residual fusion and channel compression to obtain a 64-channel third decoding feature map.

[0101] In this step, the operation in the third deformation branch decoding layer Def Decoder D3 is UpConv + Concatskip E3 + Residual fuse. That is, the output of Def Decoder D4 is input into Def Decoder D3. The features obtained after the 3D upsampling convolution (UpConv) operation are stitched and fused with the CT multi-scale 3D feature map fed into Bottleneck E5. After fusion, the output of Encoder E3 is skipped to obtain the stitched feature map. Then, the stitched feature map is subjected to nonlinear transformation operation through the residual block to smoothly fuse the "high-level semantics" and "low-level details". Finally, the channels are compressed to 64 to obtain the third decoding feature map.

[0102] In step 3413, the second deformation branch decoding layer upsamples the third decoding feature map, then concatenates it with the CT multi-scale 3D feature map, and then concatenates it with the second encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 32-channel second decoding feature map.

[0103] In this step, the operation in the second deformation branch decoding layer Def Decoder D2 is UpConv + Concatskip E2 + Residual fuse. That is, the output of Def Decoder D3 is input into Def Decoder D2. The features obtained after the 3D upsampling convolution (UpConv) operation are stitched and fused with the CT multi-scale 3D feature map fed into Bottleneck E5. After fusion, the output of Encoder E2 is skipped to obtain the stitched feature map. Then, the stitched feature map is subjected to nonlinear transformation operation through the residual block to smoothly fuse the "high-level semantics" and "low-level details". Finally, the channels are compressed to 32 to obtain the second decoded feature map.

[0104] Step 3414: The first deformation branch decoding layer upsamples the second decoding feature map, then concatenates it with the CT multi-scale 3D feature map. After concatenation, it is concatenated with the first encoded feature map input by the skip connection. Then, the concatenated features are subjected to residual fusion and channel compression to obtain the first decoding feature map with 16 channels.

[0105] In this step, the operation in the first deformation branch decoding layer Def Decoder D1 is UpConv + Concatskip E1 + Residual fuse. That is, the output of Def Decoder D2 is input into Def Decoder D1. The features obtained after the 3D upsampling convolution (UpConv) operation are stitched and fused with the CT multi-scale 3D feature map fed into Bottleneck E5. After fusion, the output of Encoder E1 is skipped to obtain the stitched feature map. Then, the stitched feature map is subjected to nonlinear transformation operation through the residual block to smoothly fuse the "high-level semantics" and "low-level details". Finally, the channels are compressed to 16 to obtain the first decoded feature map.

[0106] The deformation branch decoder contains three key operations: UpConv (upsampling convolution, usually implemented by deconvolution or interpolation + convolution) gradually restores spatial resolution; Concat skip concatenates the features of the corresponding level of the encoder in the channel dimension to preserve multi-scale spatial details; and Residual fuse smoothly combines high-level semantics with low-level details through residual blocks.

[0107] Step 3415: The final displacement field prediction head performs a three-dimensional convolution operation on the first decoded feature map, mapping the number of feature channels to 3, to obtain a three-channel CT three-dimensional displacement field, and then executes step 35.

[0108] The Final Flow Head is the output prediction layer of the deformation branch, responsible for converting decoded features into an interpretable 3D displacement field.

[0109] In this step, the Final Flow Head outputs the final full-resolution 3-channel displacement field. (dx,dy,dz), tanh activation is optional.

[0110] In this embodiment, the Final Flow Head is essentially a Conv3D layer with 3 output channels, corresponding to the displacements of the voxel in the x, y, and z directions, dx, dy, and dz, respectively. The last layer can optionally use a tanh activation function (or not use activation), and the displacement range is controlled by a loss function.

[0111] The core value of multi-scale design lies in achieving "coarse-to-fine" displacement prediction: first learning the overall thoracic cavity expansion, and then refining the boundary movements of local organs. Compared with traditional single-scale prediction, this can better model anatomical changes at different granularities.

[0112] Step 342: The residual branch decoder upsamples the CT multi-scale 3D feature map, performs corresponding skip connection splicing and residual fusion processing to obtain the HU difference residual feature map, and then executes step 36.

[0113] After the deformation branch completes the general structural alignment, the image may still have the following problems: inaccurate HU distribution in the lungs, texture deviation near the heart boundary, and loss of details introduced by interpolation. The residual branch is the second core component of the network, responsible for correcting the local HU values ​​of the deformed image.

[0114] The residual branch decoder and the deformation branch decoder have symmetrical structures and both start from Bottleneck and upsample layer by layer, but their task objectives are different: the deformation branch decoder predicts the displacement field, while the residual branch decoder learns the "HU value difference" (Residual), focuses on changes in image density, and predicts the HU residual.

[0115] The residual branch decoder includes: a fourth residual branch decoding layer, a third residual branch decoding layer, a second residual branch decoding layer, and a first residual branch decoding layer.

[0116] The specific steps of step 342 include: steps 3421-3424: Step 3421: The fourth residual branch decoding layer upsamples the CT multi-scale 3D feature map, then concatenates it with the fourth encoded feature map input by the skip connection, and then performs residual fusion and channel compression on the concatenated features to obtain a 128-channel fourth residual feature map.

[0117] In this step, the operation in the fourth residual branch decoding layer Res Decoder R4 is UpConv + Concatskip E4 + Residual fuse. That is, the output of Bottleneck E5 is input to Res Decoder R4, and the feature obtained after the 3D upsampling convolution (UpConv) operation is skipped to the output of Encoder E4 (the fourth encoded feature map) to obtain the concatenated feature map. The concatenated feature map is then subjected to a non-linear transformation operation through the residual block to smoothly fuse the "high-level semantics (from E5)" and "low-level details (from E4)". Finally, the channels are compressed to 128 to obtain the fourth residual feature map.

[0118] Step 3422: The third residual branch decoding layer upsamples the fourth residual feature map, then concatenates it with the CT multi-scale 3D feature map. After concatenation, it is concatenated with the third encoded feature map input by the skip connection. Then, the concatenated features are subjected to residual fusion and channel compression to obtain a 64-channel third residual feature map.

[0119] In this step, the operation in the third residual branch decoding layer Res Decoder R3 is UpConv + Concatskip E3 + Residual fuse. That is, the output of Res Decoder R4 is input to Res Decoder R3. The features obtained after the 3D upsampling convolution (UpConv) operation are concatenated and fused with the CT multi-scale 3D feature map fed into Bottleneck E5. After fusion, the output of Encoder E3 is skipped to obtain the concatenated feature map. Then, the concatenated feature map is subjected to nonlinear transformation operation through the residual block to smoothly fuse the "high-level semantics" and "low-level details". Finally, the channels are compressed to 64 to obtain the third residual feature map.

[0120] Step 3423: The second residual branch decoding layer upsamples the third residual feature map, then concatenates it with the CT multi-scale 3D feature map. After concatenation, it is concatenated with the second encoded feature map input by the skip connection. Then, the concatenated features are subjected to residual fusion and channel compression to obtain a 32-channel second residual feature map.

[0121] In this step, the operation in the second residual branch decoding layer Res Decoder R2 is UpConv + Concatskip E2 + Residual fuse. That is, the output of Res Decoder R3 is input to Res Decoder R2. The features obtained after the 3D upsampling convolution (UpConv) operation are concatenated and fused with the CT multi-scale 3D feature map fed into Bottleneck E5. After fusion, the output of Encoder E2 is skipped to obtain the concatenated feature map. Then, the concatenated feature map is subjected to nonlinear transformation operation through the residual block to smoothly fuse the "high-level semantics" and "low-level details". Finally, the channels are compressed to 32 to obtain the second residual feature map.

[0122] Step 3424: The first residual branch decoding layer upsamples the second residual feature map, then concatenates it with the CT multi-scale 3D feature map. After concatenation, it is concatenated with the first encoded feature map input by the skip connection. The concatenated features are then subjected to residual fusion and channel compression to obtain a 16-channel first residual feature map. Then, step 36 is executed.

[0123] In this step, the operation in the first residual branch decoding layer Res Decoder R1 is UpConv + Concatskip E1 + Residual fuse, which is the input of the output of Res Decoder R2 to Res Decoder R1. The features obtained after the 3D upsampling convolution (UpConv) operation are stitched and fused with the CT multi-scale 3D feature map fed in by Bottleneck E5. After fusion, the output of Encoder E1 is skipped to obtain the stitched feature map. Then, the stitched feature map is subjected to nonlinear transformation operation through the residual block to smoothly fuse the "high-level semantics" and "low-level details". Finally, the channels are compressed to 16 to obtain the HU difference residual feature map, and then step 36 is executed.

[0124] The Bottleneck E5 serves as a shared semantic bottleneck, with its output simultaneously fed into all levels of both the deformation branch decoder (D1-D4) and the residual branch decoder (R1-R4). This design ensures that the two branches perform multi-scale decoding based on a unified high-level feature representation: the deformation branch uses the global context of the E5 to estimate organ-level displacements, while the residual branch performs local HU corrections based on the same semantic judgments.

[0125] Step 35: The spatial transformer normalizes the CT three-dimensional displacement field and performs three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image.

[0126] In this step, the spatial transformer is the key bridge connecting the deformation branch and the residual branch, performing a differentiable 3D image warping operation. Its core function is to transform the input FB-CT image according to the predicted displacement field. Perform a spatial transformation to generate an intermediate image that is roughly aligned with the target DIBH structure.

[0127] In this step, the operations in the Spatial Transformer are: Normalize flow togrid → Trilinear sampling → Warp(FB, )= I warp Specifically, the CT three-dimensional displacement field is normalized to a sampling grid coordinate system, ranging from [-1, 1]. Based on the generated sampling grid, three-dimensional trilinear interpolation sampling is performed on the original input image (preprocessed clinical free breathing CT three-dimensional volume data). The new pixel values ​​obtained from the sampling are then refilled into a new 3D image tensor to form a distorted image. I warp .

[0128] Trilinear interpolation is a three-dimensional version of bilinear interpolation, performing linear interpolation simultaneously in the x, y, and z directions, resulting in a smooth and continuous result. Its core advantages include: Fully differentiable—supports end-to-end backpropagation, and gradients can flow from the output all the way back to the deformation branch.

[0129] Smooth and continuous – the interpolation results do not have obvious step boundaries.

[0130] Physically explainable – once the displacement field is visualized, the direction and amplitude of organ movement can be analyzed intuitively.

[0131] Alternatively, the direct displacement field can be upgraded to a stationary velocity field, and then integrated into a diffeomorphic deformation, which is more robust and medically sound than a conventional displacement field.

[0132] Step 36, the residual correction module corrects the distorted image. I warp After splicing the HU difference residual feature map with the residual feature map, residual learning is performed to obtain a channel HU correction residual map R.

[0133] The Residual Refinement module is the final output stage of the residual branch, responsible for converting the decoded features into actual HU correction values. Its inputs include: the highest-level features of the residual branch (the HU difference residual feature map output by Res Decoder R1) and the distorted... I warp The R1 output HU difference residual feature map provides semantic guidance for local HU correction, which is already distorted. I warp Provides aligned anatomical context, concats the two, processes them with residual blocks, and outputs a 1-channel HU residual R.

[0134] The core of the design logic lies in predicting "how much more HU value needs to be added" instead of directly generating a complete CT scan. The deformed image already contains most of the correct structure; what remains is mainly local density deviation and detail restoration. Residual learning is more stable than regressing the target volume data from zero, and it is easier to obtain clear boundaries and true HU distribution.

[0135] Step 37: The output layer fuses the distorted image and the HU correction residual image to obtain the predicted deep inspiration breath-hold CT data.

[0136] In this step, the final output of the network is obtained by simple addition: sCT = Warp(FB, ) + R Warp(FB, The geometry is provided by R, and the density correction is provided by R. The two are added together to obtain the final synthetic CT (sCT, i.e., the predicted deep inspiratory breath-hold CT data). This decoupled design ensures that the deformation branch focuses on structural alignment, while the residual branch focuses on strength repair.

[0137] Step 38: Calculate the corresponding cardiac dose based on the clinical free breathing CT three-dimensional volume data and the predicted deep inspiration and breath-holding CT data, and output DIBH response information based on the cardiac dose.

[0138] In this step, the clinical free breathing CT three-dimensional volume data (the rawest data without preprocessing) and the predicted deep inspiration and breath-hold CT data are input into the planning system. Based on the treatment plan, the cardiac dose is assessed through dosimetry, and the response information is output.

[0139] For example, based on the same patient's clinical free-breathing CT 3D volumetric data and predicted deep-inspiratory breath-hold CT data, FB (First-Inspiratory) and DIBH (Deep-Inspiratory Hemorrhage) plans are created, and dose parameters are compared. If DIBH reduces MHD by >1-2 Gy or significantly decreases LAD (Low Adsorption Dose), the output is "DIBH Effective Response," and deep-inspiratory breath-hold radiotherapy is recommended for these patients. If dosimetric assessment data show a relative reduction in cardiac dose to MHD of <10% (or an absolute reduction of <0.5-1 Gy), the output is "DIBH Effective Response is Low," and deep-inspiratory breath-hold radiotherapy is not recommended.

[0140] In this embodiment, the explicit modeling of respiratory motion mechanisms using a 3D deformation-sensing dual-branch network is achieved not by mapping the network from the basic lung function (FB) to the second-to-last lung function (DIBH) in a black-box manner, but by allowing the network to "first learn how the organs move, and then learn how to adjust the density." This design improves the interpretability of the model and is more consistent with the physiological nature of chest CT scans. Compared to traditional 2D U-Nets, it offers better three-dimensional volumetric consistency, avoiding issues such as inter-slice discontinuities and boundary jumps; it also provides stronger interpretability, allowing for intuitive analysis of lung expansion and the appropriateness of cardiac displacement; and it is more conducive to subsequent clinical tasks, especially cardiac dosimetry-related assessments.

[0141] Furthermore, current 2D U-Net processes each slice independently, making it difficult to model the continuous respiratory motion of the thoracic cavity along the z-axis. 3D bi-branch networks, on the other hand, can directly learn organ displacement and density changes in 3D space, which is more consistent with the generation logic from FB to DIBH. Ordinary 2D methods are prone to problems such as inter-layer discontinuities, boundary jumps, and unstable organ volumes. 3D networks inherently possess volumetric data context, which helps improve volumetric consistency. Traditional generative networks only provide the resulting image, while this approach simultaneously outputs the deformation field, allowing for intuitive analysis: where the lungs expand, whether the heart displacement is reasonable, and which areas' changes mainly come from structural deformation and which from HU correction. The ultimate goal is to determine whether DIBH can effectively reduce cardiac dose; therefore, accurate modeling of the relative position of the heart and lungs and thoracic cavity deformation is often more critical than pure pixel errors. This approach has a greater chance of improving subsequent dosimetric indicators.

[0142] A third aspect of this invention provides a suitability prediction system based on a deep inhalation and breath-hold radiotherapy method, comprising: The prediction module is used to input preprocessed clinical free-breathing CT 3D volumetric data into a trained 3D deformation-sensing dual-branch network for processing, obtaining predicted deep inspiration and breath-hold CT data. The 3D deformation-sensing dual-branch network includes: an input layer, a shared encoder, a deformation branch decoder, a final displacement field prediction head, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer. The prediction module includes: The input layer is used to input the preprocessed clinical free breathing CT 3D volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale 3D feature maps. The deformation branch decoder and the final displacement field prediction head are used to process the CT multi-scale three-dimensional feature map to obtain the three-channel CT three-dimensional displacement field. At the same time, the residual branch decoder is used to upsample the CT multi-scale three-dimensional feature map, perform corresponding skip connection splicing and residual fusion processing to obtain the HU difference residual feature map. The spatial transformer is used to normalize the CT three-dimensional displacement field and perform three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image. The residual correction module is used to stitch together the distorted image and the HU difference residual feature map and then perform residual learning to obtain a HU correction residual map for one channel. The output layer is used to fuse the distorted image and the HU correction residual map to obtain the predicted deep inspiration breath-hold CT data. The output module is used to calculate the cardiac dose based on the clinical free breathing CT three-dimensional volume data and the predicted deep inspiration and breath-holding CT data, and output DIBH response information based on the cardiac dose.

[0143] In one embodiment of the present invention, preprocessing of the original training data during the training of the 3D deformation-aware dual-branch network includes: The collected paired training free breathing CT three-dimensional volume data and training deep inspiration breath-holding CT three-dimensional volume data were used as the original training data; The training data for spatial registration were obtained by resampling, rigid registration, ROI clipping, and normalization of the training free breathing CT three-dimensional volume data and the training deep inspiration breath-hold CT three-dimensional volume data. The training data was divided into training set, validation set and test set; among them, the spatially registered training deep inspiration breath-hold CT three-dimensional volume data was used as label data.

[0144] In one embodiment of the present invention, the shared encoder includes: a first coding layer, a second coding layer, a third coding layer, a fourth coding layer, and a bottleneck layer; The input layer takes preprocessed clinical free-breathing CT 3D volumetric data and inputs it into the shared encoder for downsampling and convolution processing to obtain CT multi-scale 3D feature maps, including: The first coding layer extracts 16-channel feature maps from preprocessed clinical free breathing CT 3D volume data and outputs the first coding feature map. The second coding layer extracts 32-channel feature maps based on the first coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the second coding feature map. The third coding layer extracts 64-channel feature maps based on the second coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the third coding feature map. The fourth coding layer extracts 128-channel feature maps based on the third coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the fourth coding feature map. The bottleneck layer extracts 256-channel feature maps based on the fourth encoded feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs a CT multi-scale three-dimensional feature map.

[0145] In one embodiment of the present invention, the deformation branch decoder includes: a fourth deformation branch decoding layer, a third deformation branch decoding layer, a second deformation branch decoding layer, and a first deformation branch decoding layer; The deformation branch decoder and the final displacement field prediction head process the CT multi-scale 3D feature map to obtain a three-channel CT 3D displacement field, including: The fourth deformation branch decoding layer upsamples the CT multi-scale 3D feature map, then concatenates it with the fourth encoded feature map input by the skip connection, and then performs residual fusion and channel compression on the concatenated features to obtain a 128-channel fourth decoded feature map. The third deformation branch decoding layer upsamples the fourth decoding feature map, then concatenates it with the CT multi-scale 3D feature map, and then concatenates it with the third encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 64-channel third decoding feature map. The second deformation branch decoding layer upsamples the third decoding feature map, then concatenates it with the CT multi-scale 3D feature map, and then concatenates it with the second encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 32-channel second decoding feature map. The first deformation branch decoding layer upsamples the second decoding feature map, then concatenates it with the CT multi-scale 3D feature map, and then concatenates it with the first encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain the first decoding feature map with 16 channels. The final displacement field prediction head performs a three-dimensional convolution operation on the first decoded feature map, mapping the number of feature channels to 3, and obtaining a three-channel CT three-dimensional displacement field.

[0146] In one embodiment of the present invention, there are a fourth displacement field prediction head, a third displacement field prediction head, and a second displacement field prediction head; When training the 3D deformation-aware dual-branch network, the fourth displacement field prediction head performs a three-dimensional convolution operation on the fourth decoded feature map to obtain a three-channel fourth three-dimensional displacement field. The third displacement field prediction head performs a three-dimensional convolution operation on the third decoded feature map, mapping the number of feature channels to 3, to obtain a 3-channel third three-dimensional displacement field; The second displacement field prediction head performs a three-dimensional convolution operation on the second decoded feature map, mapping the number of feature channels to 3, and obtaining a 3-channel second three-dimensional displacement field. The loss of auxiliary supervision is calculated based on the fourth three-dimensional displacement field, the third three-dimensional displacement field, and the second three-dimensional displacement field.

[0147] In one embodiment of the present invention, the 3D deformation-sensing dual-branch network employs an auxiliary supervision mechanism during training to supervise the training of the fourth, third, and second three-dimensional displacement fields.

[0148] In one embodiment of the present invention, the main loss function of the 3D deformation-sensing dual-branch network during the training process is... for: in, This indicates the loss of image voxel difference. express The weighting coefficients, Represents the 3D structural similarity loss. express The weighting coefficients, Indicates deformation consistency loss. express The weighting coefficients, This represents the smoothing regularization loss. express The weighting coefficients, Indicates the ROI-weighted loss. express The weighting coefficients, This represents the gradient consistency loss. express Weighting coefficients.

[0149] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described applicability prediction method based on a deep inhalation breath-holding radiotherapy method provided by the present invention.

[0150] A fifth aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described applicability prediction method for a deep inhalation and breath-hold radiotherapy method provided in the present invention.

[0151] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage system located remotely from the aforementioned processor.

[0152] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware systems.

[0153] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0154] For system / electronic device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be found in the description of the method embodiments.

[0155] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for predicting the suitability of radiotherapy based on deep inspiration and breath-holding, characterized in that, Includes the following steps: Preprocessed clinical free-breathing CT 3D volumetric data is input into a trained 3D deformation-sensing dual-branch network for processing to obtain predicted deep inspiration-breath-hold CT data. The 3D deformation-sensing dual-branch network includes: an input layer, a shared encoder, a deformation branch decoder, a final displacement field prediction head, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer. The method for obtaining predicted deep inspiration-breath-hold CT data includes: The input layer inputs the preprocessed clinical free breathing CT three-dimensional volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale three-dimensional feature maps. The deformation branch decoder and the final displacement field prediction head process the CT multi-scale three-dimensional feature map to obtain a three-channel CT three-dimensional displacement field. At the same time, the residual branch decoder performs upsampling, corresponding skip connection splicing and residual fusion processing on the CT multi-scale three-dimensional feature map to obtain the HU difference residual feature map. The spatial transformer normalizes the CT three-dimensional displacement field and performs three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image. The residual correction module stitches the distorted image and the HU difference residual feature map together and then performs residual learning to obtain a HU correction residual map for one channel. The output layer fuses the distorted image and the HU correction residual map to obtain the predicted deep inhalation breath-hold CT data. The cardiac dose is calculated based on the clinical free breathing CT three-dimensional volume data and the predicted deep inspiration and breath-holding CT data, and the DIBH response information is output based on the cardiac dose.

2. The method as described in claim 1, characterized in that, Also includes: Preprocessing of the raw training data during training of the 3D deformation-aware dual-branch network includes: The collected paired training free breathing CT three-dimensional volume data and training deep inspiration breath-holding CT three-dimensional volume data were used as the original training data; The training free breathing CT three-dimensional volume data and the training deep inspiration breath-holding CT three-dimensional volume data were resampled, rigidly registered, ROI clipped and normalized to obtain spatially registered training data. The training data is divided into a training set, a validation set, and a test set; among which, the spatially registered training deep inspiration breath-hold CT three-dimensional volume data is used as label data.

3. The method as described in claim 1, characterized in that, The shared encoder includes: a first coding layer, a second coding layer, a third coding layer, a fourth coding layer, and a bottleneck layer; The input layer inputs the preprocessed clinical free-breathing CT 3D volumetric data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale 3D feature maps, including: The first coding layer extracts 16-channel feature maps from the preprocessed clinical free breathing CT three-dimensional volume data and outputs the first coded feature map. The second coding layer extracts a 32-channel feature map based on the first coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the second coding feature map. The third coding layer extracts a 64-channel feature map based on the second coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the third coding feature map. The fourth coding layer extracts a 128-channel feature map based on the third coding feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs the fourth coding feature map. The bottleneck layer extracts 256-channel feature maps based on the fourth encoded feature map and the preprocessed clinical free breathing CT three-dimensional volume data, and outputs a CT multi-scale three-dimensional feature map.

4. The method as described in claim 1, characterized in that, The deformation branch decoder includes: a fourth deformation branch decoding layer, a third deformation branch decoding layer, a second deformation branch decoding layer, and a first deformation branch decoding layer; The deformation branch decoder and the final displacement field prediction head process the CT multi-scale three-dimensional feature map to obtain a three-channel CT three-dimensional displacement field, including: The fourth deformation branch decoding layer upsamples the CT multi-scale three-dimensional feature map, then concatenates it with the fourth encoded feature map input by the skip connection, and then performs residual fusion and channel compression on the concatenated features to obtain a 128-channel fourth decoded feature map. The third deformation branch decoding layer upsamples the fourth decoding feature map, then concatenates it with the CT multi-scale three-dimensional feature map, and then concatenates it with the third encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 64-channel third decoding feature map. The second deformation branch decoding layer upsamples the third decoding feature map, then concatenates it with the CT multi-scale three-dimensional feature map, and then concatenates it with the second encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 32-channel second decoding feature map. The first deformation branch decoding layer upsamples the second decoding feature map, then concatenates it with the CT multi-scale three-dimensional feature map, and then concatenates it with the first encoded feature map input by the skip connection. Finally, the concatenated features are subjected to residual fusion and channel compression to obtain a 16-channel first decoding feature map. The final displacement field prediction head performs a three-dimensional convolution operation on the first decoded feature map, mapping the number of feature channels to 3, to obtain a three-channel CT three-dimensional displacement field.

5. The method as described in claim 1, characterized in that, Also includes: The fourth displacement field prediction head, the third displacement field prediction head, and the second displacement field prediction head; When training the 3D deformation-aware dual-branch network, the fourth displacement field prediction head performs a three-dimensional convolution operation on the fourth decoded feature map to obtain a three-channel fourth three-dimensional displacement field. The third displacement field prediction head performs a three-dimensional convolution operation on the third decoded feature map, mapping the number of feature channels to 3, to obtain a 3-channel third three-dimensional displacement field; The second displacement field prediction head performs a three-dimensional convolution operation on the second decoded feature map, mapping the number of feature channels to 3, and obtaining a 3-channel second three-dimensional displacement field. The loss of auxiliary supervision is calculated based on the fourth three-dimensional displacement field, the third three-dimensional displacement field, and the second three-dimensional displacement field.

6. The method as described in claim 5, characterized in that, The 3D deformation sensing dual-branch network employs an auxiliary supervision mechanism during training to supervise the fourth, third, and second three-dimensional displacement fields.

7. The method as described in claim 6, characterized in that, The main loss function of the 3D deformation-sensing dual-branch network during training. for: in, This represents the image voxel difference loss. express The weighting coefficients, Represents the 3D structural similarity loss. express The weighting coefficients, Indicates deformation consistency loss. express The weighting coefficients, This represents the smoothing regularization loss. express The weighting coefficients, Indicates the ROI-weighted loss. express The weighting coefficients, This represents the gradient consistency loss. express Weighting coefficients.

8. A suitability prediction system for a deep-inspiratory breath-hold radiotherapy method, characterized in that, include: The prediction module is used to input preprocessed clinical free-breathing CT 3D volumetric data into a trained 3D deformation-sensing dual-branch network for processing, to obtain predicted deep inspiration and breath-hold CT data; wherein, the 3D deformation-sensing dual-branch network includes: an input layer, a shared encoder, a deformation branch decoder, a final displacement field prediction head, a spatial transformer, a residual branch decoder, a residual correction module, and an output layer; the prediction module includes: The input layer is used to input the preprocessed clinical free breathing CT three-dimensional volume data into the shared encoder for downsampling and convolution processing to obtain CT multi-scale three-dimensional feature maps. The deformation branch decoder and the final displacement field prediction head are used to process the CT multi-scale three-dimensional feature map to obtain a three-channel CT three-dimensional displacement field. At the same time, the residual branch decoder is used to upsample the CT multi-scale three-dimensional feature map, perform corresponding skip connection splicing and residual fusion processing to obtain the HU difference residual feature map. A spatial transformer is used to normalize the CT three-dimensional displacement field and perform three-dimensional trilinear interpolation sampling on the preprocessed clinical free breathing CT three-dimensional volume data based on the normalized data to generate a distorted image. The residual correction module is used to stitch the distorted image and the HU difference residual feature map together and then perform residual learning to obtain a HU correction residual map for one channel. The output layer is used to fuse the distorted image and the HU correction residual map to obtain the predicted deep inspiratory breath-hold CT data. The output module is used to calculate the cardiac dose based on the clinical free breathing CT three-dimensional volume data and the predicted deep inspiration and breath-holding CT data, and output DIBH response information based on the cardiac dose.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the suitability prediction method based on the deep inhalation breath-holding radiotherapy method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the suitability prediction method based on the deep inhalation breath-holding radiotherapy method as described in any one of claims 1 to 7.

Images