Image registration method and device for tumor tracking and network model establishment method

By using the registration network model of respiratory end images and orthogonal angle 2D images in radiotherapy, the problem of excessive 2D/3D registration time is solved, and fast and high-precision tumor tracking is achieved, meeting the real-time needs of radiotherapy.

CN119068026APending Publication Date: 2024-12-03SHENZHEN INST OF ADVANCED TECH CHINESE ACAD OF SCI

Patent Information

Application Number
CN202310638931.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing 2D/3D image registration methods have too long registration time during radiotherapy and cannot meet real-time requirements.

Method used

By acquiring the respiratory end image and two 2D images based on orthogonal angles, input it to the registration network model, extract features using residual blocks, combine the 3D feature extraction network and spatial transformation layer, and calculate and predicted 3D CT images and real 3D CT images The maximum similarity between them is iteratively optimized to train registration network models to achieve fast and high-precision tumor tracking.

Benefits of technology

It effectively reduces the radiation dose, realizes high-precision 3D tumor tracking during the radiotherapy process, and meets the real-time needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068026A_ABST
    Figure CN119068026A_ABST
Patent Text Reader

Abstract

The invention provides an image registration method and device for tumor tracking and a network model establishment method. The image registration method for tumor tracking comprises the following steps: acquiring a breathing end image as a first floating image; obtaining two orthogonal angle-based 2D images of a target phase; and inputting the first floating image and the two 2D images based on the orthogonal angle into a pre-trained registration network model to obtain a 3D registration image for tumor tracking during radiotherapy. By means of the scheme, the irradiation dose can be effectively reduced, and the requirement for real-time performance can be met while high precision is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of medical device technology, and in particular, relates to an image registration method and device for tumor tracking, and a network model establishment method. Background Art

[0002] The purpose of radiotherapy is to kill tumor cells as much as possible while protecting normal tissues. However, the shape and position of organs and tumors will change due to respiratory movement, cardiac movement and the progress of radiotherapy. If you want to ensure that the irradiation field closely follows the target area without harming the surrounding normal tissues and organs, you need to collect images during radiotherapy and align them with the images used to formulate the radiotherapy plan. For images used to formulate radiotherapy plans, 3D images have rich anatomical structure information and are a better choice for formulating radiotherapy plans. However, for images used in radiotherapy, the reconstruction of 3D images takes a certain amount of time, and the subsequent alignment with the images before radiotherapy takes a certain amount of time, which makes it difficult to meet the needs of real-time radiotherapy.

[0003] In order to ensure the real-time requirements, considering the fast imaging speed of 2D images, 2D images will also be used in radiotherapy images. 2D / 3D registration can be divided into traditional image registration and deep learning-based image registration. For traditional image registration, iterative optimization still needs to be performed one by one during the testing phase. The registration speed is slow, usually taking several minutes or even dozens of minutes, which is difficult to meet the real-time requirements in radiotherapy. With the development of artificial intelligence and deep learning, the registration method based on deep learning replaces the cumbersome iterative optimization process in the testing process with predicted values, which can improve the computational efficiency to a certain extent. However, the existing deep learning-based registration method has the problem of large dose (that is, 2D images are usually multiple and multi-angle), which leads to the problem of long registration time.

[0004] Currently, no effective solution has been proposed to the problem that the existing 2D / 3D registration takes too long to meet the real-time requirements during radiotherapy. Summary of the invention

[0005] The purpose of the present application is to provide an image registration method and device, and a network model establishment method for tumor tracking, which can achieve the technical effect of reducing the radiation dose and achieving high-precision registration while meeting real-time requirements.

[0006] The present application provides an image registration method and device for tumor tracking, and a network model establishment method which are implemented as follows:

[0007] An image registration method for tumor tracking, the method comprising:

[0008] acquiring an end-of-respiration image as a first floating image;

[0009] Obtain two 2D images based on orthogonal angles of the target phase;

[0010] The first floating image and the two 2D images based on orthogonal angles are input into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

[0011] In one embodiment, the registration network model is established as follows:

[0012] acquiring a respiratory image of a predetermined phase in the 4D CT image as a second floating image;

[0013] Performing data expansion based on the second floating image to obtain a 3D CT image of each expiratory phase;

[0014] By projecting light, the obtained 3D CT images of each expiratory phase are projected to obtain two 2D DRRs at orthogonal angles corresponding to each respiratory phase;

[0015] The registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the second floating image.

[0016] In one embodiment, the registration network model is obtained by performing model training on two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the second floating image, including:

[0017] The residual block is used to extract the features of 2D DRR, and the channel layer is used as the third dimension feature to output a three-dimensional feature map;

[0018] Inputting the three-dimensional feature map and the second floating image into a 3D feature extraction network to obtain a three-dimensional deformation field;

[0019] Applying the three-dimensional deformation field to the second floating image through a spatial transformation layer to obtain a predicted 3D CT image corresponding to the 2D DRR;

[0020] The maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train the registration network model.

[0021] In one embodiment, in the process of calculating the maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion to iteratively optimize the network model and train the registration network model, the following formula is used as the loss function:

[0022]

[0023] L=λ1LDice (F seg ,P seg )+λ2L MI (F CT ,P CT )+λ3L Reg (φ p )

[0024] Among them, F represents the true value, P represents the predicted value, n represents the number of categories in the image, i represents the i-th category of the image, p represents all elements in the entire deformation field, λ1, λ2, λ3 represent L Dice , L MI , L Reg The weight, L Dice represents the loss between two segmentation labels, L MI represents the loss between two images, L Reg represents the regularization loss of the deformation field, F seg represents the segmentation label of the real 3D CT image, P seg represents the predicted segmentation label of the 3D CT image, F CT represents the real 3D CT image, P CT represents the predicted 3D CT image, φ p represents the predicted deformation field.

[0025] In one embodiment, data expansion is performed based on the second floating image to obtain 3D CT images of each expiratory phase, including:

[0026] obtaining a deformation field between a plurality of phases according to the second floating image;

[0027] Multiple mixed deformation fields are obtained by superimposing deformation fields within a phase on deformation fields between multiple phases;

[0028] A plurality of mixed deformation fields are applied to the end-of-exhalation image to obtain a 3D CT image of each exhalation phase.

[0029] In one embodiment, obtaining a deformation field between a plurality of phases according to the second floating image comprises:

[0030] Matching other M phases through the second floating image to obtain a deformation field between the M phases, wherein M is 7 or 9;

[0031] Two of the M inter-phase deformation fields are randomly selected for random weight superposition to obtain multiple inter-phase deformation fields from the end of exhalation to the remaining phases.

[0032] In one embodiment, the phase deformation field is obtained as follows:

[0033] Set the random control point and random movement distance of the target phase;

[0034] According to the set random control points and random moving distances, thin plate spline interpolation simulation is used to obtain the deformation field in multiple phases of the target phase.

[0035] In one embodiment, the two 2D images based on orthogonal angles include: 2D X-ray projection and / or 2D R-R.

[0036] A registration network model establishment method, comprising:

[0037] acquiring a respiratory image of a predetermined phase in the 4D CT image as a floating image;

[0038] Performing data expansion based on the floating image to obtain a 3D CT image of each expiratory phase;

[0039] By projecting light, the obtained 3D CT images of each expiratory phase are projected to obtain two 2D DRRs at orthogonal angles corresponding to each respiratory phase;

[0040] The registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image.

[0041] In one embodiment, the registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image, including:

[0042] The residual block is used to extract the features of 2D DRR, and the channel layer is used as the third dimension feature to output a three-dimensional feature map;

[0043] Inputting the three-dimensional feature map and the floating image into a 3D feature extraction network to obtain a three-dimensional deformation field;

[0044] Applying the three-dimensional deformation field to the floating image through a spatial transformation layer to obtain a predicted 3D CT image corresponding to the 2D DRR;

[0045] The maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train the registration network model.

[0046] In one embodiment, data expansion is performed based on the floating image to obtain 3D CT images of each expiratory phase, including:

[0047] Obtaining deformation fields between multiple phases according to the floating image;

[0048] Multiple mixed deformation fields are obtained by superimposing deformation fields within a phase on deformation fields between multiple phases;

[0049] A plurality of mixed deformation fields are applied to the end-of-exhalation image to obtain a 3D CT image of each exhalation phase.

[0050] An image registration device for tumor tracking, comprising:

[0051] A first acquisition module, used for acquiring a respiratory end image as a first floating image;

[0052] The second acquisition module is used to acquire two 2D images of the target phase based on orthogonal angles;

[0053] The registration module is used to input the first floating image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

[0054] A terminal device comprises a processor and a memory for storing instructions executable by the processor, wherein the steps of the above method are implemented when the processor executes the instructions.

[0055] A computer-readable storage medium stores a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0056] The image registration method for tumor tracking provided in this application obtains a respiratory end image and two 2D images based on orthogonal angles, and inputs the respiratory end image and the two 2D images based on orthogonal angles into a registration network model to obtain a 3D registered image for tumor tracking during radiotherapy. In the above example, image registration can be achieved by selecting only two projections based on orthogonal angles, which can effectively reduce the radiation dose and meet the real-time requirements while ensuring high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 is a method flow chart of an embodiment of an image registration method for tumor tracking provided by the present application;

[0059] Figure 2 It is a method flow chart of an embodiment of a method for establishing a registration network model provided by the present application;

[0060] Figure 3 It is a logical schematic diagram of an embodiment of a 2D / 3D non-rigid image registration method for lung tumor tracking provided by the present application;

[0061] Figure 4 It is a hardware structure block diagram of an electronic device for an image registration method for tumor tracking provided by the present application;

[0062] Figure 5 It is a schematic diagram of the module structure of an embodiment of an image registration device for tumor tracking provided by the present application. DETAILED DESCRIPTION

[0063] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0064] Figure 1 It is a method flow chart of an embodiment of the image registration method for tumor tracking provided by the present application. Although the present application provides method operation steps or device structures as shown in the following embodiments or drawings, more or fewer operation steps or module units may be included in the method or device based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure described in the embodiments of the present application and shown in the drawings. When the method or module structure is applied to an actual device or terminal product, it can be connected according to the method or module structure shown in the embodiment or drawings for sequential execution or parallel execution (for example, a parallel processor or a multi-threaded processing environment, or even a distributed processing environment).

[0065] Specifically, Figure 1 As shown, the above-mentioned image registration method for tumor tracking may include the following steps:

[0066] Step 101: Acquire a respiratory end image as a first floating image;

[0067] Step 102: Acquire two 2D images of the target phase based on orthogonal angles;

[0068] Step 103: Input the first floating image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

[0069] The two 2D images based on orthogonal angles may include: 2D X-ray projection / or 2D DRR. That is, during implementation, a respiratory end image and two 2D images at orthogonal angles are used as inputs to the registration network model to obtain the registration result.

[0070] In the above example, by acquiring the respiratory end image and two 2D images based on orthogonal angles, and inputting the respiratory end image and two 2D images based on orthogonal angles into the registration network model, a 3D registration image for tumor tracking during radiotherapy is obtained. In the above example, image registration can be achieved by selecting only two projections based on orthogonal angles, which can effectively reduce the radiation dose, and the high accuracy can meet the real-time requirements.

[0071] For 2D / 3D registration, we first need to consider the consistency of spatial dimensions. In this case, the dimensionality increase method is used to transform the 2D / 3D registration problem into a 3D / 3D registration problem. First, the residual block is used to extract the features of the 2D DRR, and the channel layer is used as the third dimension feature to output a three-dimensional feature map. Then, the three-dimensional feature map and the floating image (i.e., the MCT image at the end of exhalation with data expansion) are input into the 3D feature extraction network. Among them, the 3D feature extraction network is a registration network based on the attention mechanism 3D / 3D. Under the joint action of the encoding and decoding mechanism and the attention mechanism, the network outputs a three-dimensional deformation field. Then, the three-dimensional deformation field is applied to the floating image through the spatial transformation layer to obtain the corresponding 3D CT image PCT corresponding to the predicted DRR (PCT is the image output by the model, and it is also the predicted 3DCT corresponding to the 2D DRR, and FCT is the prediction). The true value (FCT is the real 3D CT obtained by the mixed data expansion in S1, which is the true value) is known. By calculating the maximum similarity between the predicted value and the true value, the model training can be completed through continuous iterative optimization.

[0072] Based on this, a method for establishing a registration network model is provided in this example, which can be Figure 2 As shown, the following steps are included:

[0073] Step 201: obtaining a respiratory image of a predetermined phase in a 4D CT (Computed Tomography) image as a second floating image; that is, obtaining an image of a certain phase in the 4D CT image as a floating image;

[0074] Specifically, the 4D CT image of the lungs can be selected as the experimental object, the end of exhalation can be used as the second floating image, and the remaining stages can be used as reference images. The 4D CT of the lung image contains 3D CT of multiple respiratory phases. After acquiring the projection data, 4D CT performs phase splitting on the projection data according to the diaphragm position in the 2D projection data, generally divided into 8 to 10 phases, and then the projection data on each phase is reconstructed separately to form a 3D CT image. The 3D CT images at different phases are superimposed to form a 4D CT image.

[0075] Step 202: performing data expansion based on the second floating image to obtain a 3D CT image of each expiratory phase;

[0076] During implementation, data expansion is performed based on the second floating image to obtain 3D CT images of each expiratory phase. The deformation fields between multiple phases are obtained based on the floating image; multiple mixed deformation fields are obtained by superimposing the deformation fields within the phases with the deformation fields between multiple phases; and the multiple mixed deformation fields are applied to the second floating image to obtain 3D CT images of each expiratory phase.

[0077] Among them, the deformation fields between multiple phases can be obtained by matching other M phases through the second floating image to obtain deformation fields between M phases, where M is 7 or 9; any two of the deformation fields between the M phases are randomly selected and superimposed with random weights to obtain multiple deformation fields between phases from the end of exhalation to the remaining phases.

[0078] For example, the respiratory end image in the 4D CT image is taken as a phase, that is, as a floating image, and then a floating image is used to match the other 9 phases to obtain the deformation field between the phases; two of the obtained 9 phase deformation fields are randomly selected for random weight superposition to obtain a large number of phase deformation fields.

[0079] The deformation field within the phase can be a random control point and a random moving distance set for the target phase; according to the set random control point and random moving distance, a thin plate spline interpolation simulation is used to obtain the deformation field within multiple phases of the target phase. That is, several control points are selected, and a thin plate spline interpolation simulation algorithm is used to obtain the deformation field within a large number of phases.

[0080] Step 203: Projecting the obtained 3D CT images of each expiratory phase by light projection to obtain two 2D DRR (Digitally Reconstructed Radiograph) images corresponding to each respiratory phase at orthogonal angles;

[0081] Step 204: A registration network model is obtained by performing model training using the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the second floating image.

[0082] When training the model, the features of the 2D DRR can be extracted using the residual block, and the channel layer is used as the third-dimensional feature to output a three-dimensional feature map; the three-dimensional feature map and the floating image are input into a 3D feature extraction network to obtain a three-dimensional deformation field; the three-dimensional deformation field is applied to the floating image through a spatial transformation layer to obtain a predicted 3D CT image corresponding to the 2D DRR; the maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train the registration network model.

[0083] Among them, in the process of model training, the mutual information between the prediction and the gold standard, the Dice similarity index between the segmentation labels and the smooth constraint of the deformation field can be used as the loss function. Specifically, the loss function can be expressed as:

[0084]

[0085] L=λ1L Dice (F seg ,P seg )+λ2L MI (F CT ,P CT )+λ3L Reg (φ p )

[0086] Among them, F represents the true value, P represents the predicted value, n represents the number of categories in the image, i represents the i-th category of the image, p represents all elements in the entire deformation field, λ1, λ2, λ3 represent L Dice , L MI , L Reg The weight, L Dice represents the loss between two segmentation labels, L MI represents the loss between two images, L Reg represents the regularization loss of the deformation field, F seg represents the segmentation label of the real 3D CT image, P seg represents the predicted segmentation label of the 3D CT image, F CT represents the real 3D CT image, P CT represents the predicted 3D CT image, φ p represents the predicted deformation field.

[0087] That is, during the training process, data enhancement can be performed first, and the exhalation image of a certain phase in the 4D CT is selected as the floating image, and the remaining phases are used as reference images to perform traditional image registration in sequence to obtain 9 inter-phase deformation fields, and then two are selected from them to be superimposed with random weights to obtain a large number of deformation fields, and TPS is used to simulate random deformation within the phase, and the inter-phase deformation and the intra-phase deformation are superimposed with random weights, so as to obtain a large number of deformation fields representing each respiratory phase, and then act on the floating image to obtain a large number of images (3D) representing each respiratory phase. Then, through the tracking method, the light projection is obtained for the above-mentioned enhanced 3D image to obtain the corresponding 2D DRR at an orthogonal angle. Then, the obtained DRR and the floating image are input into the network for training. The network output is the 3D image of the predicted DRR corresponding to the moment, and the true value is known (obtained by the above-mentioned data expansion). The maximum similarity between the two is calculated to complete the model training.

[0088] When the above model is used for clinical implementation, the floating image is the planning CT (before radiotherapy) when making a radiotherapy plan, and the 2D DRR can be an X-ray or CBCT projection (during radiotherapy). During the implementation process, the planning CT and the 2D images during radiotherapy can be directly substituted into the trained model to obtain the 3D images during radiotherapy.

[0089] The above method is described below in conjunction with a specific embodiment. However, it should be noted that this specific embodiment is only for better illustrating the present application and does not constitute an improper limitation on the present application.

[0090] Considering that the dimension can be reduced to 2D / 2D registration based on 2D / 3D registration, the model training is completed by evaluating the similarity between DRR (Digitally Reconstructed Radiograph) and X-ray, but the dimension reduction method will cause a certain degree of loss of spatial information. In addition, even with the support of graphics processing units (GPUs), forward projection, back projection and DRR generation are also very computationally intensive. In the existing image registration, iterative optimization needs to be performed one by one during the testing phase, and the registration speed is slow. The existing deep learning-based registration methods have the problems of high dose (2D images are usually multiple and multi-angle) and long registration time.

[0091] In view of the problem that retrospective 3D / 3D registration is difficult to meet the needs of real-time radiotherapy, and the problem that 2D / 3D registration has a large radiation dose and a long registration time, this example provides a 2D / 3D non-rigid registration algorithm based on orthogonal angles, which can quickly and accurately complete the 3D lung tumor tracking when only two 2D X-ray projections in radiotherapy are needed. Because the 2D images only use two 2D X-ray projections at orthogonal angles in radiotherapy, the radiation dose can be effectively reduced, and the organ and tumor tracking accuracy can still be high while speeding up the radiotherapy process.

[0092] Specifically, a method for 2D / 3D image registration for lung tumor tracking is provided, which may include the following steps:

[0093] S1: Hybrid Data Augmentation:

[0094] Considering that paired 2D / 3D medical images at the same moment are relatively scarce, a large number of 3D CT images representing various respiratory stages of the lungs can be obtained through data augmentation.

[0095] When implementing, you can use the traditional image registration Demons to register in sequence to obtain the phase-to-phase deformation φ1,...i,j, and then randomly extract two random weights and superimpose them to obtain a large number of phase-to-phase deformations from the end of exhalation to the remaining phases; then use thin plate spline interpolation simulation to obtain a large number of deformations simulating a certain phase by setting random control points and random moving distances. Then, superimpose the phase-to-phase deformation and the phase-to-phase deformation with random weights to obtain a large number of mixed deformation fields, and then apply them to the end of exhalation to achieve a 3D CTF representing each exhalation phase of the lungs. CT .

[0096] That is, in this example, the 4D CT image of the lungs is selected as the experimental object, and the end of exhalation is used as the floating image M CT , and the rest of the phases are used as reference images. The 4D CT of the lung image contains 3D CT of multiple respiratory phases. After acquiring the projection data, 4D CT performs phase splitting on the projection data according to the diaphragm position in the 2D projection data, generally divided into 8 to 10 phases, and then reconstructs the projection data on each phase to form a 3D CT image. The 3D CT images at different phases are superimposed to form a 4D CT image.

[0097] S2: Digital reconstruction image generation:

[0098] Specifically, the image after data expansion is a 3D CT image representing the radiotherapy process, and what is needed for model training is the 2D DRR (Digitally Reconstructed Radiograp) at the corresponding moment. Therefore, in this example, a light projection method is used to project the 3D CT image to obtain a 2D DRR at an orthogonal angle at the corresponding moment to simulate the real 2D X-ray projection for completing model training.

[0099] That is, through the light projection method, the 3D CT representing each respiratory stage of the lungs after data expansion is projected to obtain two 2D DRRs at orthogonal angles at corresponding moments to simulate the real X-ray projection in radiotherapy.

[0100] S3: 2D / 3D registration network:

[0101] For 2D / 3D registration, we first need to consider the consistency of spatial dimensions. In this case, the dimensionality increase method is used to transform the 2D / 3D registration problem into a 3D / 3D registration problem. First, the residual block is used to extract the features of the 2D DRR, and the channel layer is used as the third-dimensional feature to output a three-dimensional feature map. The three-dimensional feature map and the floating image (i.e., the end-expiratory MCT image with data expansion) are then input into the 3D feature extraction network. Among them, the 3D feature extraction network is based on the Attention Unet 3D / 3D registration network. Under the joint action of the encoding and decoding mechanism and the attention mechanism, the network outputs a three-dimensional deformation field φ p Then, the three-dimensional deformation field is applied to the floating image through the spatial transformation layer to obtain the corresponding 3D CT image PCT corresponding to the predicted DRR (PCT is the image output by the model and also the predicted 3DCT corresponding to the 2D DRR, FCT is the prediction). The true value F CT (FCT is the true 3D CT obtained by expanding the mixed data in S1, which is the true value) is known. By calculating the maximum similarity between the predicted value and the true value and continuously iterating and optimizing, the model training can be completed.

[0102] In the inference stage, only one end-expiratory image MCT based on orthogonal angles during radiotherapy and another 2D X-ray projection or DRR need to be input into the network to obtain a 3D image at the time of radiotherapy.

[0103] Among them, in the process of model training, the mutual information between the prediction and the gold standard, the Dice similarity index between the segmentation labels and the smoothness constraint of the deformation field are used as the loss function:

[0104]

[0105] L=λ1LDice (F seg ,P seg )+λ2L MI (F CT ,P CT )+λ3L Reg (φ p )

[0106] Among them, F represents the true value, P represents the predicted value, n represents the number of categories in the image, i represents the i-th category of the image, p represents all elements in the entire deformation field, λ1, λ2, λ3 represent L Dice , L MI , L Reg The weight, L Dice represents the loss between two segmentation labels, L MI represents the loss between two images, L Reg represents the regularization loss of the deformation field, F seg represents the segmentation label of the real 3D CT image, P seg Indicates the predicted segmentation label of the 3D CT image. In this example, λ1, λ2, and λ3 can be 0.5, 0.5, and 0.1, respectively. Seg represents the segmentation label, F represents the true value, which is the gold standard, P represents the predicted value, and φ p represents the predicted deformation field.

[0107] In the above example, the dimensionality is increased, and the channel layer is used as the feature of the third dimension. The 2D / 3D registration is transformed into a 3D / 3D registration problem. Under the action of residual blocks and attention mechanisms, non-rigid 2D / 3D registration based on orthogonal angles is achieved. Through hybrid data expansion, digital reconstruction image generation, 2D / 3D image registration, etc., fast 3D lung tumor tracking at a lower radiation dose is achieved. In other words, only two orthogonal angles of X-ray projection are needed to complete the tracking of 3D tumors with high accuracy, which can greatly reduce the radiation dose received by patients and can be used to simplify clinical hardware equipment.

[0108] In this application, it is 2D / 3D registration, but the registration needs to ensure that the two images are in one spatial dimension, so the 2D / 3D can be reduced to 2D / 2D or upgraded to 3D / 3D. The existing method is mainly to reduce the dimension and convert it into 2D / 2D, but this will lead to the loss of spatial information. Therefore, in this example, the dimension is upgraded to 3D / 3D, which can alleviate the loss of spatial information. For 2D images, in this example, only two projections based on orthogonal angles are selected, which further reduces the irradiation dose. This method can be used to study the changes of elastic organs with respiratory movement, and to realize tumor tracking based on orthogonal angle X-ray projections during radiotherapy.

[0109] During implementation, the ultimate goal of registering the real patient's tumor and the tumor represented by the phantom is to align the two images before and after the registration, that is, to obtain a 3D image during radiotherapy by using the image used by the doctor to formulate the radiotherapy plan before radiotherapy.

[0110] Taking a specific example, Figure 3 As shown, the 2D / 3D non-rigid image registration method for lung tumor tracking includes the following process:

[0111] a) Training process:

[0112] Firstly, by performing hybrid data enhancement on the moving image MCT and the corresponding segmented image Mseg, a large number of CT FCT and segmentation Fseg representing each respiratory phase are obtained.

[0113] Then, the FCT image is projected to obtain two-dimensional DRR90 and DRR00, which are input into the registration network together with the floating image MCT, and the registration network outputs the predicted deformation field φ p , and φ p Acting on the floating image MCT and the floating segmentation map Mseg, the corresponding predicted images PCT and Pseg are obtained.

[0114] b) Hybrid data augmentation.

[0115] c) Reasoning stage.

[0116] In reasoning, only two X-rays or CBCT projections at orthogonal angles during radiotherapy and the PCT image of the specified plan before radiotherapy need to be input to obtain the 3D organs and target areas during radiotherapy.

[0117] Based on the above method, experiments were conducted on the dynamic lung phantom from the end of exhalation to the end of inhalation for patient data with tumors. The experimental results showed that both real patient and phantom experiments achieved good tumor tracking results. That is, the method provided in this example can quickly complete the tracking of 3D organs and tumors with only two orthogonal angles of X-rays.

[0118] The method embodiments provided in the above embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on an electronic device as an example, Figure 4 is a hardware structure block diagram of an electronic device for an image registration method for tumor tracking provided by the present application. Figure 4As shown, the electronic device 10 may include one or more (only one is shown in the figure) processors 02 (the processor 02 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 04 for storing data, and a transmission module 06 for communication functions. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0119] The memory 04 can be used to store software programs and modules of application software, such as program instructions / modules corresponding to the image registration method for tumor tracking in the embodiment of the present application. The processor 02 executes various functional applications and data processing by running the software programs and modules stored in the memory 04, that is, the image registration method for tumor tracking of the above-mentioned application is implemented. The memory 04 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 04 may further include a memory remotely arranged relative to the processor 02, and these remote memories can be connected to the electronic device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The transmission module 06 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the electronic device 10. In one example, the transmission module 06 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 06 can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0121] At the software level, the above-mentioned image registration device for tumor tracking can be Figure 5 As shown, it may include:

[0122] A first acquisition module 501 is used to acquire a respiratory end image;

[0123] The second acquisition module 502 is used to acquire two 2D images of the target phase based on orthogonal angles;

[0124] The registration module 503 is used to input the respiratory end image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

[0125] In one embodiment, the registration network model can be established as follows:

[0126] acquiring a respiratory image of a predetermined phase in the 4D CT image as a floating image;

[0127] Performing data expansion based on the floating image to obtain a 3D CT image of each expiratory phase;

[0128] By projecting light, the obtained 3D CT images of each expiratory phase are projected to obtain two 2D DRRs at orthogonal angles corresponding to each respiratory phase;

[0129] The registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image.

[0130] In one embodiment, the registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image, which may include:

[0131] The residual block is used to extract the features of 2D DRR, and the channel layer is used as the third dimension feature to output a three-dimensional feature map;

[0132] Inputting the three-dimensional feature map and the floating image into a 3D feature extraction network to obtain a three-dimensional deformation field;

[0133] Applying the three-dimensional deformation field to the floating image through a spatial transformation layer to obtain a predicted 3D CT image corresponding to the 2D DRR;

[0134] The maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train the registration network model.

[0135] In one embodiment, in the process of calculating the maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion to iteratively optimize the network model and train the registration network model, the following formula can be used as the loss function:

[0136]

[0137] L=λ1L Dice (F seg ,P seg )+λ2LMI (F CT ,P CT )+λ3L Reg (φ p )

[0138] Among them, F represents the true value, P represents the predicted value, n represents the number of categories in the image, i represents the i-th category of the image, p represents all elements in the entire deformation field, λ1, λ2, λ3 represent L Dice , L MI , L Reg The weight, L Dice represents the loss between two segmentation labels, L MI represents the loss between two images, L Reg represents the regularization loss of the deformation field, F seg represents the segmentation label of the real 3D CT image, P seg represents the predicted segmentation label of the 3D CT image, F CT represents the real 3D CT image, P CT represents the predicted 3D CT image, φ p represents the predicted deformation field.

[0139] In one embodiment, data expansion is performed based on the floating image to obtain 3DCT images of each expiratory phase, including:

[0140] Obtaining deformation fields between multiple phases according to the floating image;

[0141] Multiple mixed deformation fields are obtained by superimposing deformation fields within a phase on deformation fields between multiple phases;

[0142] Applying multiple mixed deformation fields to the end-of-exhalation image to obtain a 3D CT image of each exhalation phase;

[0143] In one embodiment, obtaining a deformation field between multiple phases according to the floating image may include:

[0144] Matching other M phases through the floating image to obtain the deformation field between the M phases, where M is 7 or 9;

[0145] Two of the M inter-phase deformation fields are randomly selected for random weight superposition to obtain multiple inter-phase deformations from the end of exhalation to the remaining phases.

[0146] In one embodiment, the intra-phase deformation can be obtained as follows: random control points and random moving distances of the target phase are set; based on the set random control points and random moving distances, thin plate spline interpolation simulation is used to obtain deformation fields within multiple phases of the target phase.

[0147] In one embodiment, the two 2D images based on orthogonal angles may include: 2D X-ray projections and / or 2D DRR.

[0148] In this example, a registration network model establishment device is also provided, which may include:

[0149] An acquisition module, used for acquiring a respiratory image of a predetermined phase in a 4D CT image as a floating image;

[0150] An expansion module, used for performing data expansion based on the floating image to obtain a 3D CT image of each expiratory phase;

[0151] A projection module is used to project the obtained 3D CT images of each exhalation phase by light projection to obtain two 2D DRRs at orthogonal angles corresponding to each respiratory phase;

[0152] The training module is used to obtain a registration network model by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image.

[0153] In one embodiment, the training module can be specifically used to extract the features of the 2D DRR using the residual block, and use the channel layer as the third dimension feature to output a three-dimensional feature map; the three-dimensional feature map and the floating image are jointly input into the 3D feature extraction network to obtain a three-dimensional deformation field; through the spatial transformation layer, the three-dimensional deformation field is applied to the floating image to obtain a predicted 3D CT image corresponding to the 2D DRR; the maximum similarity between the predicted 3D CT image and the true 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train to obtain a registration network model.

[0154] In one embodiment, data expansion is performed based on the floating image to obtain 3D CT images of each expiratory phase, which may include: obtaining deformation fields between multiple phases based on the floating image; obtaining multiple mixed deformation fields by superimposing the deformation fields within the phases with the deformation fields between multiple phases; and applying the multiple mixed deformation fields to the end-of-respiration image to obtain 3D CT images of each expiratory phase.

[0155] The embodiments of the present application also provide a specific implementation of an electronic device capable of implementing all the steps in the image registration method for tumor tracking in the above embodiments, wherein the electronic device specifically includes the following contents: a processor, a memory, a communication interface, and a bus; wherein the processor, the memory, and the communication interface communicate with each other through the bus; the processor is used to call a computer program in the memory, and when the processor executes the computer program, all the steps in the image registration method for tumor tracking in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0156] Step 1: Acquire the end-of-respiration image as the first floating image;

[0157] Step 2: Obtain two 2D images of the target phase based on orthogonal angles;

[0158] Step 3: Input the first floating image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

[0159] From the above description, it can be seen that the embodiment of the present application obtains a respiratory end image and two 2D images based on orthogonal angles, and inputs the respiratory end image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy. In the above example, image registration can be achieved by selecting only two projections based on orthogonal angles, which can effectively reduce the radiation dose, and the high accuracy can meet the real-time requirements.

[0160] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps in the image registration method for tumor tracking in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps in the image registration method for tumor tracking in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0161] Step 1: Acquire the end-of-respiration image as the first floating image;

[0162] Step 2: Obtain two 2D images of the target phase based on orthogonal angles;

[0163] Step 3: Input the first floating image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

[0164] From the above description, it can be seen that the embodiment of the present application obtains a respiratory end image and two 2D images based on orthogonal angles, and inputs the respiratory end image and the two 2D images based on orthogonal angles into the registration network model to obtain a 3D registered image for tumor tracking during radiotherapy. In the above example, image registration can be achieved by selecting only two projections based on orthogonal angles, which can effectively reduce the radiation dose, and the high accuracy can meet the real-time requirements.

[0165] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0166] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0167] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0168] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0169] Although the present specification embodiment provides the method operation steps as described in the embodiment or flow chart, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiment is only one way in the order of execution of many steps, and does not represent a unique execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel (such as a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment) according to the method shown in the embodiment or the accompanying drawings. The term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements not only includes those elements, but also includes other elements not clearly listed, or also includes elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements.

[0170] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing the embodiments of this specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0171] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0173] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0175] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0176] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0177] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0178] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0179] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0180] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0181] The above is only an example of the embodiment of the present specification and is not intended to limit the embodiment of the present specification. For those skilled in the art, the embodiment of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiment of the present specification shall be included in the scope of the claims of the embodiment of the present specification.

Claims

1. An image registration method for tumor tracking, characterized in that: The method comprises: acquiring an end-of-respiration image as a first floating image; Obtain two 2D images based on orthogonal angles of the target phase; The first floating image and the two 2D images based on orthogonal angles are input into a pre-trained registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

2. The method according to claim 1, characterized in that The registration network model is trained in the following manner: acquiring a respiratory image of a predetermined phase in the 4D CT image as a second floating image; Performing data expansion based on the second floating image to obtain a 3D CT image of each expiratory phase; By projecting light, the obtained 3D CT images of each expiratory phase are projected to obtain two 2D DRRs at orthogonal angles corresponding to each respiratory phase; The registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the second floating image.

3. The method according to claim 2, characterized in that The registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the second floating image, including: The residual block is used to extract the features of 2D DRR, and the channel layer is used as the third dimension feature to output a three-dimensional feature map; Inputting the three-dimensional feature map and the second floating image into a 3D feature extraction network to obtain a three-dimensional deformation field; Applying the three-dimensional deformation field to the second floating image through a spatial transformation layer to obtain a predicted 3D CT image corresponding to the 2D DRR; The maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train the registration network model.

4. The method according to claim 3, characterized in that In the process of calculating the maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion, iteratively optimizing the network model and training the registration network model, the following formula is used as the loss function: L=λ1L Dice (F seg ,P seg )+λ2L MI (F CT ,P CT )+λ3L Reg (f p ) Among them, F represents the true value, P represents the predicted value, n represents the number of categories in the image, i represents the i-th category of the image, p represents all elements in the entire deformation field, λ1, λ2, λ3 represent L Dice , L MI , L Reg The weight, L Dice represents the loss between two segmentation labels, L MI represents the loss between two images, L Reg represents the regularization loss of the deformation field, F seg represents the segmentation label of the real 3D CT image, P seg represents the predicted segmentation label of the 3D CT image, F CT represents the real 3D CT image, P CT represents the predicted 3D CT image, φ p represents the predicted deformation field.

5. The method according to claim 2, characterized in that: Data expansion is performed based on the second floating image to obtain 3D CT images of each expiratory phase, including: Obtaining a deformation field between a plurality of phases according to the second floating image; Multiple mixed deformation fields are obtained by superimposing deformation fields within a phase on deformation fields between multiple phases; A plurality of mixed deformation fields are applied to the end-of-exhalation image to obtain a 3D CT image of each exhalation phase.

6. The method according to claim 5, characterized in that Obtaining a plurality of deformation fields between phases according to the second floating image, including: Matching other M phases through the second floating image to obtain a deformation field between the M phases, wherein M is 7 or 9; Two of the M inter-phase deformation fields are randomly selected for random weight superposition to obtain multiple inter-phase deformation fields from the end of exhalation to the remaining phases.

7. The method according to claim 5, characterized in that The phase internal deformation field is obtained as follows: Set the random control point and random movement distance of the target phase; According to the set random control points and random moving distances, thin plate spline interpolation simulation is used to obtain the deformation fields in multiple phases of the target phase.

8. The method according to claim 1, characterized in that The two 2D images based on orthogonal angles include: 2D X-ray projection / or 2D DRR.

9. A method for establishing a registration network model, characterized in that: include: acquiring a respiratory image of a predetermined phase in the 4D CT image as a floating image; Performing data expansion based on the floating image to obtain a 3D CT image of each expiratory phase; By projecting light, the obtained 3D CT images of each expiratory phase are projected to obtain two 2D DRRs at orthogonal angles corresponding to each respiratory phase; The registration network model is obtained by performing model training on the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image.

10. The method according to claim 9, characterized in that The registration network model is obtained by training the two 2D DRRs at orthogonal angles corresponding to each respiratory phase and the floating image, including: The residual block is used to extract the features of 2D DRR, and the channel layer is used as the third dimension feature to output a three-dimensional feature map; Inputting the three-dimensional feature map and the floating image into a 3D feature extraction network to obtain a three-dimensional deformation field; Applying the three-dimensional deformation field to the floating image through a spatial transformation layer to obtain a predicted 3DCT image corresponding to the 2D DRR; The maximum similarity between the predicted 3D CT image and the real 3D CT image obtained by data expansion is calculated to iteratively optimize the network model and train the registration network model.

11. The method according to claim 9, characterized in that Data expansion is performed based on the floating image to obtain 3D CT images of each expiratory phase, including: Obtaining deformation fields between multiple phases according to the floating image; Multiple mixed deformation fields are obtained by superimposing deformation fields within a phase on deformation fields between multiple phases; A plurality of mixed deformation fields are applied to the end-of-exhalation image to obtain a 3D CT image of each exhalation phase.

12. An image registration device for tumor tracking, characterized in that: include: A first acquisition module, used for acquiring a respiratory end image as a first floating image; The second acquisition module is used to acquire two 2D images of the target phase based on orthogonal angles; The registration module is used to input the first floating image and the two 2D images based on orthogonal angles into a pre-trained registration network model to obtain a 3D registered image for tumor tracking during radiotherapy.

13. A terminal device, comprising a processor and a memory for storing instructions executable by the processor, characterized in that: When the processor executes the instructions, the steps of the method according to any one of claims 1 to 8 are implemented.

14. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Lung 4D-CT image exhaling process middle phase image reconstruction method based on registration

    CN104361568A

Cited By

  • Tumor position monitoring method and system

    CN119868823A

  • Dynamic tumor tracking method and apparatus, computer device, and storage medium

    WO2026123201A1