A Method for Correcting Respiratory Motion Artifacts in PET / CT Images Based on Rotationally Isovariant SO(3) Group Convolutional Neural Networks
By using a PET/CT image correction method based on rotationally equivariant SO(3) group convolutional neural network, the problem of respiratory motion artifact correction was solved, the imaging quality and the accuracy of tumor identification were improved, and the non-rigid motion feature extraction and deformation field smooth construction of complex human organs were realized.
Patent Information
- Application Number
- CN202311712916.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-13
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-12-13
AI Technical Summary
Existing PET/CT imaging techniques have difficulty accurately correcting artifacts caused by respiratory motion in chest and abdominal scans, which affects the assessment of lung lesions. Existing methods also suffer from problems such as patient discomfort, data capture errors, high requirements for respiratory signal accuracy, and image instability.
We employ a rotationally equivariant SO(3) group convolutional neural network, combined with a cascaded pyramid convolutional neural registration network and a spatial transformation network. By extracting the non-rigid motion features of human organs through rotationally equivariant SO(3) group convolutional kernels, we gradually construct a smooth deformation field and perform image distortion transformation. We then optimize image registration using loss functions and regularization loss functions.
It effectively corrects respiratory motion artifacts, improves the imaging quality of PET/CT images, enhances the ability to determine the benign or malignant nature of tumors, and improves the reflection of local changes in the deformation field and the non-uniformity of data distribution.
Smart Images

Figure CN117635750B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for correcting respiratory motion artifacts in PET / CT images based on a rotationally equivariant SO(3) group convolutional neural network, belonging to the field of nuclear medicine imaging technology. Background Technology
[0002] PET / CT (Positron Emission Tomography / Computed Tomography) imaging technology has played a crucial role in tumor diagnosis in recent years due to its high accuracy. It can assess not only individual lung lesions but also detect metastatic lesions throughout the body. However, due to the inconsistency between PET and CT scan times, coupled with physiological movements such as breathing, scans of the chest and abdominal regions are prone to artifacts, affecting the accurate assessment of lung lesions. Current research on respiratory motion artifact correction methods based on PET / CT focuses on external equipment, data-driven approaches, respiratory gating, and image registration. External equipment-based methods have drawbacks such as patient discomfort and long preparation times. Data-driven methods are prone to capturing erroneous data, such as overall patient movement. Respiratory gating methods require high accuracy of respiratory signals, suffer from unstable image acquisition, and struggle to ensure image consistency.
[0003] With the recent rise of artificial intelligence, many scholars have focused on research based on convolutional neural networks. However, due to the complex and variable irregular curved surface structure of human organs and tissues, ordinary CNN networks cannot complete non-rigid transformations. Furthermore, the deformation field is crucial for achieving image alignment and accurate transformation. Respiratory motion artifacts can cause pixels in certain areas of the deformation field to collapse or overlap, failing to accurately reflect local changes in the image and resulting in uneven distribution of deformation field data. Therefore, we consider using the rotationally isotropic SO(3) group for modeling. The SO(3) group focuses on describing rotation in three-dimensional space, making it particularly suitable for simulating biological structures that rotate and change position with respiration. By applying the structure of the SO(3) group, the dynamic changes of the lungs and tumors during respiration can be effectively captured. Summary of the Invention
[0004] This invention provides a method for correcting respiratory motion artifacts in PET / CT images based on a rotationally equivariant SO(3) group convolutional neural network. This method is used to correct respiratory motion artifacts in PET / CT images, improve imaging quality, and can be used to assist doctors in judging the benign or malignant nature of tumors.
[0005] The technical solution of this invention is: a method for correcting respiratory motion artifacts in PET / CT images based on rotationally equivariant SO(3) group convolutional neural networks, the method comprising:
[0006] Step 1: First, define a new rotationally equivariant SO(3) group convolution kernel. This convolution kernel can be used to mine the non-rigid motion features of human organs. Focus on mining the non-rigid motion features of human organs to achieve in-depth and detailed analysis of organs and tumors.
[0007] Step 2: Combining the proposed cascaded pyramid convolutional neural registration network, a registration network with embedded rotational equivariant convolution kernels is used to gradually construct a smooth deformation field from coarse to fine.
[0008] Step 3: Use an STN network to distort the deformation field to achieve artifact correction.
[0009] Step 4: Calculate the similarity between the registered image and the fixed image and the smoothness of the deformation field by measuring and regularizing the loss function.
[0010] Furthermore, the specific steps of Step 1 are as follows:
[0011] Step 1.1: First, analyze the definitions of Lie groups and group convolution;
[0012] Step 1.2: Based on the meaning of Lie groups and group convolution, SO(3) groups are selected to construct the rotationally equivariant SO(3) group convolution kernel of the present invention with a cube as the unit element. Its rotation axis consists of 4 cubic axes and 6 quadratic axes. After rotation, 15 orientation-preserving rotation subgroups can be obtained.
[0013] Step 1.3, fix image I f and moving image I m The features are input into the Pyramid Encoder network to extract feature maps at five different stages. These feature maps are then convolved with a 3D SO(3) convolution kernel during the downsampling and upsampling stages to obtain the convolutional image F(x,y,z). The specific calculation is as follows:
[0014]
[0015]
[0016] Where * represents convolution operation, and SO3Conv represents SO(3) group kernel.
[0017] Furthermore, the specific steps of Step 2 are as follows:
[0018] Step 2.1, fix the smallest size image I f and moving image I m Concatenate them into Concat(f,m), Concat and I f The deformation field at the current stage is obtained by processing a single piece.
[0019] Step 2.2, First Stage Deformation Field The Moving image is processed in the next stage before the deformation field is extracted in the second stage.
[0020] Step 2.3: After extracting the deformation field in the fifth stage, the deformation fields of the five stages are superimposed to obtain the final deformation field.
[0021] Furthermore, the specific steps of Step 2.1 are as follows:
[0022] Step 2.1.1: In the Pyramid Encoder module, rotating equivariant SO(3) group convolution kernels are used to learn different dimensions and sizes of the Fixed and Moving images. In each convolution process, rotating equivariant SO(3) group convolution kernels are used to obtain 15 weighted convolution kernels with different dimensions, and Instance3dNormalize and LeakyReLU activation function operations are performed to learn complex non-rigid motion features. At this time, the output after processing by the Pyramid Encoder module is:
[0023]
[0024] in This is the first stage characteristic of Moving. This is a characteristic of the second stage of Moving. This is the first stage characteristic of Fixed. This is the second-stage characteristic of Fixed, and so on;
[0025] Step 2.2: In the cascaded registration stage, using five structures with identical structures but different processing sizes, a multi-head self-attention mechanism is first employed. As a query, Using key and value, this method preserves global motion information while capturing local motion at different locations; the formula flow of this module is as follows:
[0026]
[0027]
[0028]
[0029] in, h, w, l represent the feature map size, c represents the number of channels, and Q represents the number of channels. 1 yes The weight values K of the image after linear layer processing 1 yes After linear layer processing, the weight value, V 1 yes After the weights are processed by the linear layer, the first-stage values Q, K, and V are obtained. Q and K are then calculated as a dot product:
[0030] NA1 = softmax(Q) 1 ·K 1 )
[0031] The next step is to obtain multiple subfields at this level by calculating a neighborhood attention map-weighted regular displacement field:
[0032]
[0033] at this time This represents the weight values of each channel in the deformation field of the first stage. A typical deformation field has three channels; however, to ensure the deformation field contains sufficiently rich information, it is usually... The number of channels will be greater than 3, and then the Flow-net module will change the number of deformation field channels to 3:
[0034]
[0035] At the same time, in order not to lose tumor details, To perform enhanced feature detail extraction separately, the InceptionV2 module is set to have 3 output channels.
[0036]
[0037] Finally The first-stage deformation field is obtained by adding the two parallel-processed deformation fields:
[0038]
[0039] Furthermore, Step 2.2 specifically includes the following:
[0040] The result Upsampling of the deformation field doubles its size, and then... Twisting, the twisted Again again The next stage of processing ensures that the branches can fully utilize contextual information while also more meticulously depicting the details and local features of artifacts and tumors. This also improves the network's generalization ability and enhances the model's versatility.
[0041] Furthermore, Step 2.3 includes superimposing five different sized stage deformation fields to enable the registration network to obtain the registration deformation field of the PET / CT image.
[0042] Furthermore, in Step 3, the deformation field and the original-sized Moving image are distorted using a Spatial Transformation Network (STN) to calculate the corrected image. To achieve the purpose of artifact correction.
[0043] Furthermore, Step 4 includes: using a loss function to evaluate the degree of matching between the registered image and the original fixed image; optimizing the image registration process by minimizing the loss function so that the registered image is as close as possible to the fixed image.
[0044]
[0045]
[0046] Among them, I f It is a fixed image, I m It is an image with motion artifacts. It is a deformation field, i.e., I f Each voxel to I m The offset of the corresponding voxel. For I m Predicted image after STN network distortion transformation Used to calculate I f and Similarity between them It acts on The regularization is used to make the deformation field smoother, and λ is the regularization parameter.
[0047] The beneficial effects of this invention are: the method for correcting respiratory motion artifacts in PET / CT images based on rotationally equivariant SO(3) group convolutional neural networks, the proposed rotationally equivariant SO(3) group can effectively extract the complex and varied irregular curved surface non-rigid motion of human organs and tissues, and improve the situation where the deformation field cannot accurately reflect the local changes of the image and the deformation field data is unevenly distributed. Attached Figure Description
[0048] Figure 1 This is a flowchart from the present invention;
[0049] Figure 2 This is a basic structural diagram of the cascaded pyramid registration network used in this invention;
[0050] Figure 3 This is a diagram of the Pyramid Encode structure used in this invention;
[0051] Figure 4 These are simulations of the correction results of the method of this invention under different planes of the human lung;
[0052] Figure 5 This is a diagram showing the correction results of the method of this invention under different aspects of clinical PET data. Detailed Implementation
[0053] Example: Figures 1-5 As shown, a method for correcting respiratory motion artifacts in PET / CT images based on a rotationally isovariant SO(3) group convolutional neural network is presented. The method includes:
[0054] Step 1: First, define a new rotationally equivariant SO(3) group convolution kernel. This kernel can be used to mine the non-rigid motion features of human organs, focusing on mining these features to achieve in-depth and detailed analysis of organs and tumors. The specific steps of Step 1 are as follows:
[0055] Step 1.1: First, analyze the definitions of Lie groups and group convolution;
[0056] Definition 1: A Lie group is a mathematical object that possesses both group and smooth manifold structure, and whose group operations (such as multiplication) are smooth. Simply put, a Lie group is a group that can undergo continuous transformations, such as rotations and translations. Generally, Lie groups can be represented using matrices. For example, the special orthogonal group SO(n) is the set of all n×n orthogonal matrices with a determinant of 1, and can be represented as:
[0057] SO(n)={R∈R^(n×n)∣R T R = I,,det(R) = 1 (1) In formula (1), R T I is the transpose of R, I is the identity matrix, and det(R) is the determinant of matrix R.
[0058] Definition 2: Group Convolution: Group convolution is used to handle the invariance of input data under certain transformations (such as rotation and translation). Unlike traditional convolution, group convolution takes these transformations of the input data into account and integrates them into the convolution operation.
[0059] Let f and g be functions defined on a Lie group G, then their group convolution (f*g) is defined as:
[0060] (f*g)(x)=∫ G f(y)h(y -1 x)dy (2)
[0061] For the discrete group, the integral becomes a summation:
[0062] (f*g)(x)=∑y∈G f(y)g(y -1 x) (3)
[0063] In deep learning, this means that the network can learn the characteristics of the data under group transformations, thereby improving the model's invariance or equivariance to transformations such as rotation and scaling.
[0064] Step 1.2: Based on the meaning of Lie groups and group convolution, the SO(3) group is selected to construct the rotationally equivariant SO(3) group convolution kernel of this invention with a cube as the unit element. Its rotation axis consists of 4 cubic axes and 6 quadratic axes. After rotation, 15 orientation-preserving rotation subgroups can be obtained; their specific definitions are as follows:
[0065] Let the center point of the cube be represented by a three-dimensional unit vector v: ‖v‖=1, and the group action under each rotation be represented by a 3×3 orthogonal matrix R, det(R)=1. Then the SO(3) group convolution can be written as the inner product between the signal and the rotation filter:
[0066]
[0067] Since the group SO(3) is a Lie group of a three-dimensional manifold, and Since it is a two-dimensional manifold, the convolution at this point is a function on SO(3), for which we need to... pass The group action defined by the composite operation As The domain of its own action, i.e. The representation of ρ(g) on the above is:
[0068]
[0069] The second convolution of the newly constructed SO(3) group kernel in the network is:
[0070]
[0071] Step 1.3, fix image I f and moving image I m The images are input into different feature extraction modules of Pyramid Encoder to extract feature maps of 5 different sizes. The fixed image I is then used to extract feature maps of 5 different sizes. f and moving image I m The features are input into the Pyramid Encoder network to extract feature maps at five different stages. These feature maps are then convolved with a 3D SO(3) convolution kernel during the downsampling and upsampling stages to obtain the convolutional image F(x,y,z). The specific calculation is as follows:
[0072]
[0073]
[0074] Where * represents convolution operation, and SO3Conv represents SO(3) group kernel.
[0075] Step 2: Combining the proposed cascaded pyramid convolutional neural registration network, a registration network embedded with rotationally equivariant convolutional kernels is used to progressively construct a smooth deformation field from coarse to fine. The specific steps of Step 2 are as follows:
[0076] Step 2.1, fix the smallest size image I f and moving image I m Concatenate them into Concat(f,m), Concat and I f The deformation field at the current stage is obtained by processing a single piece.
[0077] Step 2.2, First Stage Deformation Field The Moving image is processed in the next stage before the deformation field is extracted in the second stage.
[0078] Step 2.3: After extracting the deformation field in the fifth stage, the deformation fields of the five stages are superimposed to obtain the final deformation field.
[0079] Furthermore, the specific steps of Step 2.1 are as follows:
[0080] Step 2.1.1: In the Pyramid Encoder module, rotating equivariant SO(3) group convolution kernels are used to learn different dimensions and sizes of the Fixed and Moving images. In each convolution process, rotating equivariant SO(3) group convolution kernels are used to obtain 15 weighted convolution kernels with different dimensions, and Instance3dNormalize and LeakyReLU activation function operations are performed to learn complex non-rigid motion features. At this time, the output after processing by the Pyramid Encoder module is:
[0081]
[0082] in This is the first stage characteristic of Moving. This is a characteristic of the second stage of Moving. This is the first stage characteristic of Fixed. This is the second-stage characteristic of Fixed, and so on;
[0083] Step 2.2: In the cascaded registration stage, using five structures with identical structures but different processing sizes, a multi-head self-attention mechanism is first employed. As a query, Using key and value, this method preserves global motion information while capturing local motion at different locations; the formula flow of this module is as follows:
[0084]
[0085]
[0086]
[0087] in, h, w, l represent the feature map size, c represents the number of channels, and Q represents the number of channels. 1 yes The weight values K of the image after linear layer processing 1 yes After linear layer processing, the weight value, V 1 yes After the weights are processed by the linear layer, the first-stage values Q, K, and V are obtained. Q and K are then calculated as a dot product:
[0088] NA1 = softmax(Q) 1 ·K 1 (10)
[0089] The next step is to obtain multiple subfields at this level by calculating a neighborhood attention map-weighted regular displacement field:
[0090]
[0091] at this time This represents the weight values of each channel in the deformation field of the first stage. A typical deformation field has three channels; however, to ensure the deformation field contains sufficiently rich information, it is usually... The number of channels will be greater than 3, and then the Flow-net module will change the number of deformation field channels to 3:
[0092]
[0093] At the same time, in order not to lose tumor details, we To perform enhanced feature detail extraction separately, the InceptionV2 module is set to have 3 output channels.
[0094]
[0095] Finally The first-stage deformation field is obtained by adding the two parallel-processed deformation fields:
[0096]
[0097] Furthermore, Step 2.2 specifically includes the following:
[0098] First stage deformation field The Moving image is processed in the next stage, followed by the second stage of deformation field extraction, and the obtained... Upsampling of the deformation field doubles its size, and then... Twisting, the twisted Again again The next stage of processing ensures that the branches can fully utilize contextual information while also more meticulously depicting the details and local features of artifacts and tumors. This also improves the network's generalization ability and enhances the model's versatility.
[0099]
[0100]
[0101] Furthermore, Step 2.3 includes superimposing five different sized stage deformation fields to enable the registration network to obtain the registration deformation field of the PET / CT image.
[0102] After the deformation field of the fifth stage is extracted, the deformation fields of the five stages are superimposed to obtain the final deformation field.
[0103]
[0104] Step 3: Use an STN network to distort the deformation field to achieve artifact correction.
[0105] Furthermore, in Step 3, the deformation field and the original-sized Moving image are distorted using a Spatial Transformation Network (STN) to calculate the corrected image. To achieve the purpose of artifact correction.
[0106] Step 4: Calculate the similarity and deformation field smoothness between the registered image and the fixed image using a loss function and a regularized loss function. Step 4 includes: using a loss function to evaluate the matching degree between the registered image and the original fixed image; optimizing the image registration process by minimizing the loss function, so that the registered image is as close as possible to the fixed image.
[0107]
[0108]
[0109] Among them, I f It is a fixed image, I m It is an image with motion artifacts. It is a deformation field, i.e., I f Each voxel to I m The offset of the corresponding voxel. For I m Predicted image after STN network distortion transformation Used to calculate I f and Similarity between them It acts on The regularization is used to make the deformation field smoother, and λ is the regularization parameter.
[0110] In one feasible implementation, it further includes:
[0111] Step 5: Evaluate the model using simulation data and clinical data;
[0112] Step 6: Further optimize the parameters to obtain a parameter model applicable to most artifact images.
[0113] Specifically, Step 5 involves evaluating the model using simulation and clinical data. Geant4 is a widely used software toolkit in physics for simulating complex particle interactions and radiation effects of matter. Geant4 is widely used in various simulation projects, including Gate software. Gate (Geometry and Tracking) is a simulation software developed based on Geant4, primarily used in medical physics, especially in research on positron emission tomography (PET), single-photon emission computed tomography (SPECT), radiotherapy, and other imaging techniques. We used Gate to simulate the PET / CT scanning environment and non-uniform rational B-splines (NURBs) cardiac torso (NCAT) simulation software to simulate the dynamic body membranes of the lungs during respiratory motion, obtaining the raw PET data. First, data from the same time point were selected and set as reference images after reconstruction. Then, data from other time points were reconstructed, and these reconstructed images were used as images to be registered. Next, these simulation data underwent standardization preprocessing, including cropping to ensure consistent image size, adjusting to zero mean to eliminate bias, and normalization to unify the data range. Finally, these data were allocated in a 70% and 30% ratio as the training and test sets, respectively, for use in subsequent image analysis and model training processes.
[0114] Experiment 1:
[0115] Images of the cross-section of the simulated cardiac membrane after respiratory artifact registration and correction using the method of this invention ( Figure 4 The following index analysis was conducted. After registration using other registration models, including our own, the image registered using VoxelMorph (proposed by Balakrishnan et al.) showed an Ncc value of 97.47%, an HD value of 2.0249, a DSC value of 58.36%, and an IOU value of 41.2%. The image registered using ModeT (proposed by Haiqiao Wang et al.) showed an Ncc value of 97.18%, an HD value of 1.56717, a DSC value of 68.61%, and an IOU value of 52.21%. The image registered using SymTransformer (proposed by Maingrui et al.) showed an Ncc value of 98.01%, an HD value of 1.56717, a DSC value of 63.36%, and an IOU value of 46.38%. The image registered using our method showed a CC value of 98.71%, an HD value of 1.2073, a DSC value of 79.98%, and an IOU value of 66.63%. Therefore, the method of this invention can effectively correct artifacts in PET / CT images.
[0116] Experiment 2:
[0117] Images of three planes after registration and correction of clinical data using the method of this invention ( Figure 5 Quality assessment was conducted. As can be seen from the image, due to the presence of artifacts, the tumors in different parts of the moving image are mixed up and have unclear boundaries, making it impossible to separate them well from the surrounding tissues. After registration using other registration models, including my own, the image registered using VoxelMorph (proposed by Balakrishnan et al.) has an Ncc value of 90.21%, an HD value of 4.78, a DSC value of 65.99%, and an IOU value of 49.24%. The image registered using ModeT (proposed by Haiqiao Wang et al.) has an Ncc value of 91.93%, an HD value of 4.54123, a DSC value of 79.22%, and an IOU value of 65.59%. The image registered using SymTransformer (proposed by MaMingrui et al.) has an Ncc value of 91.23%, an HD value of 4.63685, a DSC value of 72.33%, and an IOU value of 56.66%. The image registered using my method has a CC value of 92.28%, an HD value of 4.4155, a DSC value of 80.75%, and an IOU value of 67.72%. Therefore, the method of this invention can effectively correct artifacts in PET / CT images.
[0118] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.
Claims
1. A method for correcting respiratory motion artifacts in PET / CT images based on rotationally equivariant SO(3) group convolutional neural networks, characterized in that: The method includes: Step 1: First, define the rotationally equivariant SO(3) group convolution kernel. This convolution kernel can be used to mine the non-rigid motion features of human organs. Focus on mining the non-rigid motion features of human organs to achieve in-depth and detailed analysis of organs and tumors. Step 2: Combine the cascaded pyramid convolutional neural registration network and use the registration network with embedded rotational isovariant convolution kernels to gradually build a smooth deformation field from coarse to fine. Step 3: Use an STN network to distort the deformation field to achieve artifact correction. Step 4: Calculate the similarity between the registered image and the fixed image and the smoothness of the deformation field by measuring and regularizing the loss function. The specific steps of Step 1 are as follows: Step 1.1: First, analyze the definitions of Lie groups and group convolution; Step 1.2: Based on the meaning of Lie groups and group convolution, select SO(3) group to construct the rotationally equivariant SO(3) group convolution kernel with the cube as the unit element. Its rotation axis consists of 4 cubic axes and 6 quadratic axes. After rotation, 15 orientation-preserving rotation subgroups can be obtained. Step 1.3, fix image I f and moving image I m The features are input into the Pyramid Encoder network to extract feature maps at five different stages. These feature maps are then convolved with a 3D SO(3) convolution kernel during the downsampling and upsampling stages to obtain the convolutional image F(x,y,z). The specific calculation is as follows: ; ; Where * represents convolution operation, and SO3Conv represents SO(3) group kernel; The specific steps of Step 2 are as follows: Step 2.1: Set the smallest fixed image I f and moving image I m spell , with I f The deformation field at the current stage is obtained by processing a single piece. ; Step 2.2, First Stage Deformation Field The Moving image is processed in the next stage before the deformation field is extracted in the second stage. Step 2.3: After extracting the deformation field in the fifth stage, the deformation fields of the five stages are superimposed to obtain the final deformation field. The specific steps of Step 2.1 are as follows: Step 2.1.1: In the Pyramid Encoder module, rotating equivariant SO(3) group convolution kernels are used to learn different dimensions and sizes of the Fixed and Moving images. In each convolution process, rotating equivariant SO(3) group convolution kernels are used to obtain 15 weighted convolution kernels with different dimensions, and Instance3dNormalize and LeakyReLU activation function operations are performed to learn complex non-rigid motion features. At this time, the output after processing by the Pyramid Encoder module is: ; in This is the first stage characteristic of Moving. This is a characteristic of the second stage of Moving. This is the first stage characteristic of Fixed. This is the second-stage characteristic of Fixed, and so on; Step 2.2: In the cascaded registration stage, using five structures with identical structures but different processing sizes, a multi-head self-attention mechanism is first employed. As a query, Using key and value, this method preserves global motion information while capturing local motion at different locations; the formula flow of this module is as follows: ; ; ; in, , Indicates the size of the feature map. Indicates the number of channels. yes The weight values of the image after linear layer processing yes Weight values after linear layer processing yes After the weights are processed by the linear layer, we obtain Q, K, and V for the first stage. Q and K are then first calculated as a dot product: ; The next step is to obtain multiple subfields at this level by calculating a neighborhood attention map-weighted regular displacement field: ; at this time This represents the weight values of each channel in the deformation field of the first stage. A typical deformation field has three channels; however, to ensure the deformation field contains sufficiently rich information, it is usually... The number of channels will be greater than 3, and then the Flow-net module will change the number of deformation field channels to 3: ; At the same time, in order not to lose tumor details, To perform enhanced feature detail extraction separately, the InceptionV2 module is set to have 3 output channels. ; Finally , The first-stage deformation field is obtained by adding the two parallel-processed deformation fields: 。 2. The method for correcting respiratory motion artifacts in PET / CT images based on a rotationally equivariant SO(3) group convolutional neural network according to claim 1, characterized in that: Step 2.2 specifically includes the following: The result Upsampling the deformation field doubles its size, and then... Twisting, the twisted Again again Then, the next stage of processing is carried out. This processing ensures that the branches can make full use of the context information while also more meticulously depicting the details and local features of artifacts and tumors. At the same time, it also improves the generalization ability of the network and enhances the versatility of the model.
3. The method for correcting respiratory motion artifacts in PET / CT images based on a rotationally equivariant SO(3) group convolutional neural network according to claim 1, characterized in that: Step 2.3 involves superimposing five deformation fields of different sizes to enable the registration network to obtain the registration deformation field of the PET / CT image. .
4. The method for correcting respiratory motion artifacts in PET / CT images based on a rotationally equivariant SO(3) group convolutional neural network according to claim 1, characterized in that: In Step 3, the Spatial Transformation Network (STN) is used to distort the deformation field and the original-size Moving image, and the corrected image is calculated. To achieve the purpose of artifact correction.
5. The method for correcting respiratory motion artifacts in PET / CT images based on a rotationally equivariant SO(3) group convolutional neural network according to claim 1, characterized in that: Step 4 includes: using a loss function to evaluate the matching degree between the registered image and the original fixed image; optimizing the image registration process by minimizing the loss function so that the registered image is as close as possible to the fixed image. ; ; in, It is a fixed image. It is an image with motion artifacts. It is a deformation field, that is Each voxel in The offset of the corresponding voxel. ( )for Predicted image after STN network distortion transformation Used to calculate and ( Similarity between ) It acts on Regularization is used to make the deformation field smoother. It is the regularization parameter.
Citation Information
Patent Citations
PET breathing motion image artifact registration correction method based on three-dimensional convolutional neural network
CN115049752A
Image processing method and device, electronic equipment and computer storage medium
CN116977364A
