Gradient adjustment medical image registration method and system based on multi-resolution hierarchical structure
Patent Information
- Application Number
- CN202410646716.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-23
- Publication Date
- 2026-09-22
- Estimated Expiration
- 2044-05-23
AI Technical Summary
[0004]本发明在多分辨率图像配准中引入分层次梯度调节方案，替代手动设置超参数调整损失平衡，解决因为搜索最佳超参数的费时问题，具体提出了一种基于多分辨率层次化结构的梯度调节医学图像配准方法及系统，实现更适合从粗到细图像配准的损失平衡
[0052]与现有技术相比，本发明基于多分辨率层次化结构的梯度调节医学图像配准方法及系统，利用多分辨率层次化结构对医学图像从全局到局部完成多尺度配准，本发明提出的梯度调节策略利用分辨率层次越高，正则化效果应该越弱的思想有效解决了现阶段多分辨率配准中损失权衡单一的问题；同时梯度调节方法利用损失梯度构建坐标系后得到的正交分量，将其经过合成等处理进行梯度更新，剔除了费时费力的手工设置超参数方法，有效解决相似性损失和正则化损失之间的冲突关系。
Smart Images
Figure CN118505762B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image registration technology, specifically relating to a gradient-adjusted medical image registration method and system based on a multi-resolution hierarchical structure. This invention uses a multi-resolution hierarchical structure for registration and introduces a gradient adjustment method to facilitate the trade-off between loss and gain. Background Technology
[0002] Medical image registration is a crucial technique in medical image processing, used to spatially or feature-wise align medical images acquired from different time points, modalities, or imaging devices. This technique has wide applications in clinical diagnosis, surgical planning, and treatment monitoring. Traditional registration methods typically require computationally intensive iterative optimization, making them inefficient in practice. Deep learning-based image registration methods, however, demonstrate more competitive performance and runtime advantages. To address complex and changing environments, multi-resolution image registration schemes have been formally proposed, progressively adjusting images from coarse to fine and from global to local dimensions to achieve more accurate matching results.
[0003] Furthermore, in image registration across multi-resolution structures, a key focus of evaluation and optimization is minimizing a composite objective function, which includes a similarity loss to penalize appearance differences between image pairs and a regularization loss term to ensure the smoothness of the deformation field. Typically, hyperparameters are introduced into the objective function to balance registration accuracy and smoothness of the deformation field. However, optimal hyperparameter values can vary significantly across different image modalities or structures, and even small variations can greatly impact accuracy. Determining these values manually or with other validation data requires substantial computation and manpower, and may lead to suboptimal, misleading results that hinder progress, especially when researchers may resort to values from the literature that are insufficient for their specific datasets or registration tasks. Therefore, reducing or even circumventing hyperparameter search is a crucial step in developing, evaluating, and deploying registration methods. Two major challenges need to be addressed in multi-resolution registration at different levels: firstly, experimentally determining the optimal hyperparameters at each level is too time-consuming and redundant when using traditional hyperparameter loss balancing methods; secondly, the specificity of loss balancing at each level needs to be addressed, requiring the design of more suitable adjustment schemes that incorporate the characteristics of each level. Based on this, the present invention proposes a gradient-adjusted medical image registration method and system based on a multi-resolution hierarchical structure. Summary of the Invention
[0004] This invention introduces a hierarchical gradient adjustment scheme in multi-resolution image registration to replace manual setting of hyperparameters to adjust loss balance, thus solving the time-consuming problem of searching for the optimal hyperparameters. Specifically, it proposes a gradient adjustment medical image registration method and system based on a multi-resolution hierarchical structure to achieve a loss balance more suitable for coarse-to-fine image registration.
[0005] The present invention adopts the following technical solution:
[0006] A gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure includes the following steps:
[0007] S1. Obtain the medical image dataset and divide it into training and test sets;
[0008] S2. Construct a multi-resolution hierarchical network architecture, with resolution divided into three layers, and registration performed from low to high;
[0009] S3. Develop a multi-level gradient adjustment strategy;
[0010] S4. Determine the similarity loss function and regularization loss function to be used for optimization;
[0011] S5. Design a gradient adjustment implementation scheme for a multi-resolution hierarchical network;
[0012] S6. Input the training set from the dataset into the designed multi-resolution hierarchical network architecture, complete the training by adjusting the gradients of each layer, and save the model.
[0013] Furthermore, in step S1, a medical image dataset obtained through MRI is acquired, and operations such as slice cropping are performed to form a two-dimensional image dataset, which is then divided into a test set D at a ratio of 1:4. test and training set D train ;
[0014] Furthermore, in step S2, the registration architecture estimates the deformation field in a multi-layered manner. Each layer of the registration network consists of two parts: a deformation network and a spatial transformation network. The deformation network extracts features and generates the deformation field, while the spatial transformation network transforms the moving image into a deformed image based on the deformation field. This multi-layered registration structure is characterized by the network adaptively deepening from low to high layers as the image resolution increases to adjust the focus from global to local displacement. This allows the shallow layers of the network to learn global deformation from low-resolution images, while the deeper layers can learn fine deformation from high-resolution images.
[0015] Furthermore, in step S3, the image registration problem can be expressed by a formula: A fixed image F and a moving image M are paired and input into f. θ In (typically using neural networks), the deformation field is generated. The optimal parameter θ is typically updated using standard mini-batch gradient descent, as shown below:
[0016]
[0017] Where α is the learning rate; λ is the hyperparameter balancing the two losses; L sim and L reg These are the similarity loss and the regularization loss, respectively.
[0018] Current multi-resolution structural image registration uses the same hyperparameters for each layer, ignoring the differences between different layers. To adapt to the characteristics of different resolution levels, this invention proposes adjusting the hyperparameters at different levels using different regularization effects. Furthermore, to address the time-consuming nature of hyperparameter tuning for each layer, a gradient adjustment method is used to resolve the conflicting relationships affecting the balance during two-constraint gradient descent, rewriting the formula in a non-hyperparameter form:
[0019]
[0020] Where i = 1, 2, 3, representing the registration of each layer from low to high resolution; φ i The gradient optimization scheme for each specific topic at each layer is represented as follows:
[0021] For low-resolution image registration (i=1), greater emphasis is placed on global deformation to ensure that the initial deformation field information is effectively transferred to the next level of higher-resolution images to complete intermediate registration. Using a strong regularized gradient as a scheme to train image networks with fewer details that affect similarity can better meet the requirements of coarse training.
[0022] In this invention, as the resolution increases (i = 2, 3), the effect of regularization should converge appropriately, as more and more detailed information needs to be considered, requiring more accurate generation of high-resolution images (greater than the set threshold). Compared to over-regulation, which can easily lead to blurred details, gradually decreasing regularization loss gradients better ensure the accuracy of generated images and complete layer-by-layer training.
[0023] For medium-resolution image registration (i=2), moderate regularization and similarity loss are likely key factors. To achieve better registration results, a balance is achieved by adjusting the strength of regularization and the weights of the similarity loss. At this stage, gradient-based alignment may be a stable choice, contributing to improved robustness of the entire process.
[0024] For high-resolution image registration (i=3), global detail information is required, demanding higher accuracy in the generation of high-resolution images. Compared to over-regulation, which can easily lead to blurred details, using a weaker regularized loss gradient can better ensure the accuracy of the generated image, completing the final layer of training.
[0025] Furthermore, in step S4, the loss function guiding the training and optimization of multi-resolution structural image registration generally consists of two parts: a similarity loss and a regularization loss function, which respectively ensure the accuracy and smoothness of image registration, wherein the similarity loss L sim You can choose the mean squared error (MSE loss function):
[0026]
[0027] Alternatively, the Normalized Cross-Correlation (NCC) loss function:
[0028]
[0029] Where p represents a pixel in the fixed image and the moving image, and Ω represents the entire image region. This indicates that M passes through a deformation field. The distorted image being processed, then and This represents the average voxel value within a local window centered on voxel p.
[0030] The regularization loss L reg The L2 regularization loss function is used to ensure that the displacement value at a certain position is similar to the displacement values at its neighboring positions, as expressed as:
[0031]
[0032] in, It is the gradient of pixel p in the displacement field U in the horizontal x and vertical y directions. The two-dimensional gradient is approximated by forward lookup:
[0033]
[0034] Two loss functions guide mini-batch gradient descent, generating the similarity loss gradient G. sim and regularization loss gradient G reg Then, the two gradients are performed according to the specific resolution requirements of the subject, and the parameters are updated.
[0035] Furthermore, in step S5, in order to use gradient adjustment to solve the loss balance problem and introduce different regularization effects, the gradient of the model parameters with respect to the loss function is first calculated through the backpropagation algorithm, where the gradient represents the rate of change of the loss function with respect to each parameter, and has the direction and magnitude of change.
[0036] The angular relationship between the two loss gradients can be divided into two types: similarity loss gradient G. sim and regularization loss gradient G regWhen the angle between them is acute, it indicates that optimization will proceed in a common direction. In this case, the dominant gradient G can be used as a reference. sim When the angle is obtuse, the gradient effect needs to be mitigated or eliminated. Therefore, three gradient adjustment schemes are designed for different scenarios:
[0037] Firstly, during low-resolution training, in order to maximize the effect of regularized gradients and eliminate the impact of gradient conflicts, G is used... sim Decomposition yields perpendicular to G reg The projection vector G of the normal plane sim⊥ Orthogonal components do not conflict. G is synthesized directly. sim⊥ and G reg The result depends mainly on G reg The comprehensive gradient G total The formula is as follows:
[0038]
[0039] During higher-level training, a method is proposed that combines similarity loss gradient as the dominant information and regularized gradient as the auxiliary direction optimization at higher resolution levels, thereby proposing to use G... sim Projected onto G reg The projected gradient is obtained on the normal plane, and this projected gradient is directly used as the output composite gradient G. total G total Numerically, all are determined by G sim To control it, the direction is controlled by G. reg The influence is orthogonal to it and will not be related to G in any direction. reg Gradient conflicts arise. The calculation formula is as follows:
[0040]
[0041] The highest resolution layer gradient adjustment should satisfy the condition under minimum regularization. First, for G... sim Construct coordinate axes, and place G reg Decompose along the coordinate axes to obtain the vertical projection component G reg⊥ Combination G reg⊥ and G sim A new integrated gradient is obtained, which mainly depends on G. sim The magnitude and direction of the gradient G reduce the constraints of regularization, better meeting the requirements. total The calculation formula is as follows:
[0042]
[0043] During training from low to high resolution, the update direction of the comprehensive gradient obtained by gradient processing gradually moves closer to the similarity loss gradient, while the influence of regularization gradually decreases, indicating that the hierarchical gradient update adjustment method and strategy selection are closely related.
[0044] Furthermore, in step S6, the training set from the dataset is input into the designed multi-resolution hierarchical network architecture. The deformation field and model obtained from low-resolution registration are passed to higher-resolution layers, and the training is performed sequentially to obtain the final model. The loss function is provided by the loss mentioned in step S3, and the gradient update of each layer is completed according to the gradient adjustment scheme in S5.
[0045] This invention also discloses a gradient-adjusted image registration system based on a multi-resolution hierarchical structure, which, based on the above method, includes the following modules:
[0046] Dataset creation module: Collect medical images, create datasets, and divide them into training and testing sets;
[0047] Multi-resolution hierarchical network architecture module: Construct a three-layer hierarchical network architecture with different resolutions (upper, middle, and lower). The model obtained by registration of the upper network will be combined with the upsampled resolution image for training to obtain a new model.
[0048] Gradient adjustment strategy module: Formulate loss balance optimization schemes for registration from low to high resolution levels. Starting with gradient adjustment, formulate weak regularization gradient adjustment strategies for high resolution and strong regularization gradient adjustment strategies for low resolution.
[0049] The loss function adjustment module determines the similarity loss function and regularization loss function used for optimization. The loss function that controls the training optimization consists of two parts: one is the similarity loss that ensures the accuracy of image registration with fixed images and the registration images, and the other is the regularization loss that ensures smooth deformation. The two are coordinated with each other.
[0050] Gradient adjustment implementation module: Design the gradient adjustment implementation scheme for multi-resolution hierarchical networks; based on the strategy module, design the specific implementation of the hierarchical gradient adjustment scheme, use gradient operations and other methods to process the gradient of the loss function, reconstruct the orthogonal correlation between the two gradients, and obtain a new optimized gradient;
[0051] Training model saving module: The training set is input into the multi-resolution hierarchical network architecture for layer-by-layer training. At the same time, the loss gradient is adjusted by the corresponding gradient adjustment method at different layers to update the gradient and save the model.
[0052] Compared with existing technologies, the present invention provides a gradient-adjusted medical image registration method and system based on a multi-resolution hierarchical structure. It utilizes the multi-resolution hierarchical structure to complete multi-scale registration of medical images from global to local. The gradient adjustment strategy proposed in this invention effectively solves the problem of a single loss trade-off in current multi-resolution registration by utilizing the idea that the higher the resolution level, the weaker the regularization effect. At the same time, the gradient adjustment method uses the orthogonal components obtained after constructing the coordinate system using the loss gradient, and performs gradient updates by synthesizing and other processing, eliminating the time-consuming and laborious method of manually setting hyperparameters, and effectively resolving the conflict between similarity loss and regularization loss. Attached Figure Description
[0053] Figure 1 The flowchart below shows a preferred embodiment of the present invention: a gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure.
[0054] Figure 2 This is a schematic diagram of the multi-level gradient adjustment strategy in step S4 of a preferred embodiment of the present invention.
[0055] Figure 3 This is a block diagram of a gradient-adjusted medical image registration system based on a multi-resolution hierarchical structure, according to a preferred embodiment of the present invention. Detailed Implementation
[0056] The following specific implementation examples illustrate the embodiments of the present invention, providing a clearer and more understandable understanding of the effects and advantages of the invention for those skilled in the art. Furthermore, the implementation methods of the present invention are not limited to the proposed methods and other different applications exist. The details provided in this specification can also be modified in various ways according to different viewpoints and applications without departing from the spirit of the present invention.
[0057] like Figure 1-2 As shown, this embodiment addresses the shortcomings of existing technologies in loss trade-offs during multi-resolution image registration by providing a gradient-adjusted image registration method based on a multi-resolution hierarchical structure. The specific steps are as follows:
[0058] S1. Acquire medical images, create a dataset, and divide it into a training set and a test set;
[0059] S2. Construct a multi-layered network architecture consisting of three layers of registration networks with different resolution levels, and perform progressive registration from low to high level to achieve registration from coarse to fine and from global to local.
[0060] S3. Develop a multi-level gradient adjustment strategy, where the gradient of each level is obtained by mini-batch gradient descent from the loss.
[0061] S4. Determine the loss function used to optimize the model during training, consisting of similarity loss and regularization loss, to ensure accuracy and precision;
[0062] S5. Design a specific implementation scheme for gradient adjustment of a multi-resolution hierarchical network, where different gradient adjustments are used for three-layer registration to satisfy registration characteristics from coarse to fine;
[0063] S6. Input the training set from the dataset into the designed multi-layer network architecture, complete the training by adjusting the gradients of each layer, and save the model.
[0064] The specific steps of this embodiment are described below:
[0065] In step S1, a brain neuroimaging dataset is selected, the original images are sliced and extracted to obtain two-dimensional images, and image scaling and other operations are performed. The dataset is then divided into a test set D at a 1:4 ratio. test and training set D train During the training process, two images are randomly selected from each group of images to form an image pair.
[0066] In step S2, the registration architecture estimates the deformation field in a multi-layered manner. The registration network adaptively deepens from low-resolution layers to the final high-resolution layers as the image resolution increases to adjust the focus on global-local displacement. This allows the shallow layers of the network to learn global deformation from low-resolution images, while the deeper layers learn fine deformation from high-resolution images. More specifically, the shallowest low-resolution layer takes the lowest-resolution image as input and outputs the coarsest deformation field. Subsequently, the deformation field obtained at this level is combined with the next level of moving image to adapt to higher resolution image layers and output the corresponding horizontal deformation field. By analogy, the finest deformation field is finally obtained at the highest resolution layer registration.
[0067] Each layer of the registration network primarily consists of a deformation network and a spatial transformation network. The deformation network employs a U-Net structure, comprising an encoder and a decoder with skip connections. Each layer from the encoder is cascaded to the corresponding layer in the decoder to enhance feature aggregation and prevent gradient vanishing. The U-Net takes moving and stationary image pairs as input and outputs a deformation field. This deformation field is used to distort motion images via STN.
[0068] In step S3, image registration estimates the moving image I. m and fixed image I f Nonlinear corresponding fields between This process is mathematically expressed as For deep learning-based registration methods, f θ (Using a neural network) It takes a pair of stationary and moving images as input and outputs a deformation field with optimal parameters θ. Typically, θ can be updated using standard mini-batch gradient descent, as shown below:
[0069]
[0070] Where α is the learning rate; λ is the hyperparameter that balances the two losses.
[0071] However, hyperparameter tuning is time-consuming and highly dependent on experience, making it difficult to reach the optimal solution. Therefore, further analysis of the optimization process of the above gradient update formula reveals that registration accuracy and spatial regularization (also known as smoothness) are potentially controversial in model optimization, as these two constraints may have different directions and strengths during gradient descent. Based on this, relevant gradient processing methods are used to resolve the controversial relationship, transforming it into an orthogonal relationship. Updating only one loss gradient is sufficient to correlate it with the other loss term. Simultaneously, combined with a multi-resolution image registration regularization constraint adjustment strategy, the formula is rewritten in a non-hyperparameter form under different schemes:
[0072]
[0073] Where i = 1, 2, 3 represents each layer of registration from low to high resolution; The gradient is solved using the mini-batch gradient descent method; φ i The gradient optimization scheme for each specific topic at each layer is represented as follows:
[0074] For low-resolution image registration (i=1), the task is to complete the initial global deformation, and the initial deformation field is passed to the next level of higher-resolution images to complete the next level of intermediate registration. Using a strong regularized gradient as a method to train image networks with fewer details that affect similarity may better meet the requirements of coarse training.
[0075] For medium-resolution image registration (i=2), moderate regularization and similarity loss are likely key factors. To achieve better registration results, a balance is achieved by adjusting the strength of regularization and the weights of similarity loss. At this stage, gradient-based alignment is a stable choice that helps improve the robustness of the entire process.
[0076] For high-resolution image registration (i=3), global detail information is required, demanding higher precision in the generation of high-resolution images. Compared to over-regulation, which can easily lead to blurred details, using a weaker regularized loss gradient can better ensure the accuracy of the generated images, completing the final layer of training.
[0077] In step S4, the loss function for the image registration loss balance optimization of each layer under the multi-resolution structure consists of two parts: 1) similarity loss, which is used to measure the similarity between the fixed image and the interpolated image, encourages deformation, and is also the key part of the optimization during the training process; 2) regularization loss, which is used to estimate the constrained deformation field, prevent overfitting, and is equivalent to assisting the balance optimization effect.
[0078] (1) Similarity loss L sim Choose the mean squared error (MSE loss function):
[0079]
[0080] Where p represents the pixels in the fixed and moving images, and Ω represents the entire image region. Calculate the moving image I m According to the deformation field Perform deformation treatment;
[0081] (2) Regularization loss L reg A diffusion regulator was selected. The condition that causes the displacement value at a certain position to be similar to the displacement values at its neighboring positions is expressed as:
[0082]
[0083] in It is the gradient of the displacement field U on voxel p, which is a two-dimensional gradient in the horizontal and vertical directions. The two-dimensional gradient is approximated by forward lookup:
[0084]
[0085] Where p {x,y} It calculates the pixel value at the current pixel position {x,y}.
[0086] Two loss functions guide mini-batch gradient descent, generating the similarity loss gradient G. sim and regularization loss gradient G reg Then, the two gradients are performed according to the specific resolution requirements of the subject, and the parameters are updated.
[0087] In step S5, the gradient adjustment of the multi-resolution hierarchical network is mainly divided into three parts, with different layers using corresponding adjustment schemes. It is known that there are two possible positional relationships between the two loss gradients: the similarity loss gradient G... sim and regularization loss gradient G reg When the angle between them is acute, they are compatible and will not conflict, indicating that they will optimize in a common direction. In this case, the dominant gradient G can be used as a reference. simPerform network updates separately; if the angle between the two is obtuse, it indicates a conflict between the two loss gradients. Conflicting gradients are a major cause of non-smooth deformation, and this gradient conflict needs to be eliminated during loss balancing. Therefore, based on the idea in step S4, the specific design is as follows:
[0088] During training at low resolution layers, strong regularization is needed to satisfy the characteristics of coarse registration. Therefore, maximizing the effect of the regularized gradient and eliminating the influence of gradient conflicts is achieved by using G... sim Decomposition yields perpendicular to G reg The projection vector G of the normal plane sim⊥ Orthogonal components do not conflict. Then, G is directly synthesized. sim⊥ and G reg The result depends mainly on G reg The comprehensive gradient G total The formula is as follows:
[0089]
[0090] During the training of higher resolution layers, orthogonal projection is used to dominate gradient processing to optimize the geometric representation of depth image registration, thus removing G... reg The size of G is controlled to reduce the impact of regularization. sim and G reg When a conflict occurs, combining the gradient-dominant information from the similarity loss with the auxiliary direction optimization effect of regularization at higher resolutions, we propose to... sim Projected onto G reg The projected gradient is obtained on the normal plane, and this projected gradient is the output comprehensive gradient G. total Firstly, numerically, they are all determined by G. sim Control, secondly and G reg They must be orthogonal, meaning they will not be perpendicular to G in any direction. reg Gradient conflict occurs when two gradients share the same optimization direction. The calculation formula is as follows:
[0091]
[0092] Among them, Cos(G sim G reg This relates to calculating the angles of two gradients, derived from the cosine value calculation formula: At this point, the influence of regularization is reduced. Compared to the previous training layer, the regularization loss only plays a directional role, discarding the magnitude effect, which is consistent with the strategy.
[0093] In the highest resolution layer, i.e., the original resolution layer, the weak regularization constraint is utilized to further reduce the effect of the regularization gradient based on the previous layer, while strengthening the effect of the similarity loss gradient. The first two methods, based on stronger regularization, aim to make the dominant gradient G... sim Adaptive G reg To satisfy the orthogonality and non-conflict requirement, at the third level, G is used. reg With G sim Processing the orthogonal components to weaken G reg The resulting impact. Similar to the lowest resolution layer regularization method, firstly, G... sim Construct coordinate axes, and place G reg Decompose along the coordinate axes to obtain the vertical projection component G reg⊥ Combination G reg⊥ and G sim The new composite gradient is obtained. Composite gradient G total The calculation formula is as follows:
[0094]
[0095] During training from low to high resolution, the update direction of the comprehensive gradient obtained by gradient processing gradually moves closer to the similarity loss gradient, while the influence of regularization gradually decreases, indicating that the hierarchical gradient update adjustment method and strategy selection are closely related.
[0096] In step S6, the medical image training dataset is input into the designed network architecture, starting with low-resolution hierarchical image registration. Image pairs are reduced to one-quarter of their original size and input into the U-Net network to extract features and generate deformation fields. Simultaneously calculate the regularization loss function L. reg1 The deformation field x1 is processed through an STN spatial transformation network to obtain a deformation image, and then the similarity loss L between the deformation image and the fixed image is calculated. sim1 The loss function chosen is MSE; L reg1 and L sim1 After gradient descent, low-level gradient adjustment is performed, followed by gradient update, and iterative training is carried out. The model obtained in each epoch is tested using a test set, the mean squared error is calculated, and the model with the best performance under a specified number of iterations, Model1, is saved.
[0097] Then, the image pairs reduced by half resolution and Model 1 are fed into the network for mid-layer training. The moving image is deformed according to Model 1, and then features are extracted, and image distortion is performed to obtain the deformation field x2 and the regularization loss L. reg2 and L sim2Perform mid-level gradient adjustment and save the best-performing model, Model2; then train the highest-resolution layer network in the same way and save the final best-performing model, Model3.
[0098] like Figure 3 As shown, this embodiment discloses a gradient-adjusted medical image registration system based on a multi-resolution hierarchical structure. Based on the above method embodiment, it includes the following modules:
[0099] Dataset creation module: Collect medical images, create datasets, and divide them into training and testing sets;
[0100] Multi-resolution hierarchical network architecture module: Constructs a three-layer network architecture with different resolution levels, and performs registration from low to high;
[0101] Gradient adjustment strategy module: Formulate loss balance optimization schemes for registration from low to high resolution, and formulate weak regularization gradient adjustment strategies for high resolution and strong regularization for low resolution.
[0102] Loss function adjustment module: Determines the similarity loss function and regularization loss function to be used for optimization;
[0103] Gradient adjustment implementation module: Designing a gradient adjustment implementation scheme for multi-resolution hierarchical networks;
[0104] Training model saving module: The training set is input into the multi-resolution hierarchical network architecture for layer-by-layer training. At the same time, the loss gradient is adjusted by the corresponding gradient adjustment method at different layers to update the gradient and save the model.
[0105] Other aspects of this embodiment can be found in the above method embodiments.
[0106] In summary, this invention proposes a gradient-adjusted medical image registration method and system based on a multi-resolution hierarchical structure. This invention guides gradient updates using a layered gradient adjustment strategy within a multi-resolution hierarchical structure. The gradient processing method for each layer is combined with the resolution of that layer, and different regularizations are used at different resolution levels to achieve better results. Furthermore, by employing the concept of orthogonal gradients, both loss gradients are processed to achieve orthogonal effects. Combined with orthogonal projection and orthogonal component operations, gradients can be jointly updated based on similarity and regularization constraints during registration optimization, eliminating the time-consuming experimental problems associated with manually setting hyperparameters. Based on the different registration priorities at different resolutions, targeted gradient adjustment strategies are formulated, significantly improving the overall registration performance of the architecture.
[0107] The above description is merely a detailed explanation of preferred embodiments and principles of the present invention. For those skilled in the art, there may be changes in specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.
Claims
1. A gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure, characterized in that, Includes the following steps: S1. Obtain the medical image dataset and divide it into training and test sets; S2. Construct a multi-resolution hierarchical network architecture, with resolution divided into three layers, and registration performed from low to high. S3. Develop a multi-level gradient adjustment strategy; S4. Determine the similarity loss function and regularization loss function to be used for optimization; S5. Design gradient adjustment implementation schemes for multi-resolution hierarchical networks; specifically, in this step, design gradient adjustment implementation schemes for three different scenarios: During low-resolution hierarchical training, by using G sim Decomposition yields perpendicular to G reg The projection vector G of the normal plane sim⊥ Orthogonal components do not conflict; G is synthesized directly. sim⊥ and G reg , depends on G reg The comprehensive gradient G total The formula is as follows: During training of higher resolution layers, it is proposed to use G sim Projected onto G reg The projected gradient is obtained on the normal plane, and this projected gradient is directly used as the output composite gradient G. total G total Numerically, all are determined by G sim To control it, the direction is controlled by G. reg The influence is orthogonal to it and will not be related to G. reg Gradient conflicts arise; the formula is as follows: Highest resolution layer gradient adjustment, for G sim Construct coordinate axes, and place G reg Decompose along the coordinate axes to obtain the vertical projection component G reg⊥ Combination G reg⊥ and G sim Obtain the new integrated gradient The calculation formula is as follows: S6. Input the training set from the dataset into the designed multi-resolution hierarchical network architecture, complete the training by adjusting the gradients of each layer, and save the model.
2. The gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure according to claim 1, characterized in that, In step S1, a medical image dataset is obtained, sliced to obtain two-dimensional images, scaled to create a dataset, and then divided into a test set and a training set at a ratio of 1:
4.
3. The gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure according to claim 1 or 2, characterized in that, In step S2, the multi-resolution hierarchical network architecture is constructed as follows: the registration architecture estimates the deformation field in a multi-layered manner. Each layer of the registration network includes a deformation network and a spatial transformation network. The deformation network extracts features and generates the deformation field, and the spatial transformation network converts the moving image into a deformed image based on the deformation field.
4. The gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure according to claim 3, characterized in that, In step S3, image registration estimates the nonlinear correspondence field φ between the moving image M and the stationary image F, expressed as φ = f θ (F, M), f θ Neural networks are employed; for deep learning-based registration methods, f θ Using a fixed image and a moving image pair as input, the deformation field is output through the optimal parameter θ; θ is updated using standard mini-batch gradient descent, as shown in the following equation: in, and These are the similarity loss and the regularization loss, respectively; α is the learning rate; λ is a hyperparameter that balances the two losses. Rewrite the above equation in non-hyperparametric form under different schemes: Where i=1,2,3 represents each layer of registration from low to high resolution; The gradient is solved using the mini-batch gradient descent method; The optimization scheme for each theme gradient is represented as follows: For low-resolution image registration (i=1), the task is to complete the initial global deformation, and the initial deformation field will be passed to the next higher-resolution image to complete the next intermediate registration. For medium-resolution image registration, i.e., i=2, a balance is achieved by adjusting the intensity of regularization and the weight of similarity loss; For high-resolution image registration (i=3), global detail information is required, necessitating accurate generation of the high-resolution image.
5. The gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure according to claim 4, characterized in that, In step S4, the loss function that guides the training and optimization of multi-resolution structural image registration consists of two parts. Components: Similarity loss and regularization loss function, where the similarity loss function is either the mean squared error (MSE) loss function or the normalized cross-correlation (NCC) loss function; the MSE loss function is as follows: The normalized cross-correlation NCC loss function is as follows: Where p represents the pixel in the fixed image and the moving image, Represents the entire image area. This indicates that M passes through a deformation field. Processed deformed images, and This represents the average voxel value within a local window centered on voxel p; The regularization loss function chosen is the L2 regularization loss function, expressed as: in, It is the gradient of pixel p in the displacement field U in the horizontal x and vertical y directions. The two-dimensional gradient is approximated by forward lookup: The two loss functions guide mini-batch gradient descent respectively, generating similarity loss gradients G. sim and regularization loss gradient G reg Then, the two gradients are operated on according to the resolution and subject requirements, and the parameters are updated.
6. The gradient-adjusted medical image registration method based on a multi-resolution hierarchical structure according to claim 1, characterized in that, In step S6, the training set in the dataset is input into the designed multi-resolution hierarchical network architecture. The deformation field and model obtained from the low-resolution registration are passed to the higher-resolution layers, and the training is carried out sequentially to obtain the final model.
7. A gradient-adjusted medical image registration system based on a multi-resolution hierarchical structure, wherein the method described in any one of claims 1-6 is characterized in that... Includes the following modules: Dataset creation module: Collect medical images, create datasets, and divide them into training and testing sets; Multi-resolution hierarchical network architecture module: Constructs a three-layer network architecture with different resolution levels, and performs registration from low to high; Gradient adjustment strategy module: Formulate loss balance optimization schemes for registration from low to high resolution, and formulate weak regularization gradient adjustment strategies for high resolution and strong regularization for low resolution. Loss function adjustment module: Determines the similarity loss function and regularization loss function to be used for optimization; Gradient adjustment implementation module: Designing a gradient adjustment implementation scheme for multi-resolution hierarchical networks; Training model saving module: The training set is input into the multi-resolution hierarchical network architecture for layer-by-layer training. At the same time, the loss gradient is adjusted by the corresponding gradient adjustment method at different layers to update the gradient and save the model.