An Adaptive Motion Decomposition Method for Medical Image Registration

Abstract

This invention discloses an adaptive motion decomposition method for medical image registration, belonging to the field of medical image analysis technology. The method includes: constructing a primary medical registration model based on the Transformer motion decomposition module and the adaptive recursive module; training the primary medical registration model using a loss function to obtain a final medical registration model after training; inputting a fixed image and a moving image into the final medical registration model; and finally outputting a full-resolution fine deformation field to achieve registration between the moving and fixed images. This invention overcomes the limitations of traditional medical registration by improving the Transformer motion decomposition-based medical registration model, enabling it to automatically achieve different degrees of refinement decomposition for complex medical images in different regions, thereby improving the accuracy and efficiency of medical registration.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, and more specifically, to an adaptive motion decomposition method for medical image registration. Background Technology

[0002] Deformable image registration is crucial in medical image analysis, but it remains challenging to efficiently handle large anatomical deformations. Deep learning methods have shown promise in this field, but often lack adaptability and interpretability for clinical applications. In registration tasks, Transformers are typically used to compute mutual attention between features in stationary and moving images. For a given point in the stationary image, the Transformer model searches for all possible corresponding points in the moving image and computes a weighted correspondence. This is more effective at handling large-scale deformations than simple local matching.

[0003] Currently, Transformer-based deep learning networks are quite mature in medical image registration. However, the use of Transformer in these networks typically only leverages its self-attention mechanism to enhance feature learning (similar to segmentation tasks), without adequately designing for the registration task itself. This includes things like introducing more complex feature extraction networks or optimizing the computational efficiency of the Transformer. Nevertheless, these methods still have some shortcomings:

[0004] (1) Traditional registration methods formulate registration as an optimization problem that iteratively minimizes the difference metric. While these methods provide theoretically reasonable solutions, they suffer from high computational complexity and sensitivity to initialization. Early methods often struggled to handle large deformations and lacked interpretability in their deformation patterns.

[0005] (2) Introducing the Transformer architecture into medical image registration has shown significant improvements in capturing long-range dependencies and complex spatial relationships. However, existing Transformer-based methods typically employ a fixed computational graph, treating all image regions equally regardless of their deformation complexity. This one-size-fits-all approach leads to low computational efficiency and performs poorly in regions requiring varying degrees of detail processing.

[0006] (3) The general Transformer motion decomposition module processes all regions with uniform complexity and lacks the adaptive refinement required to deal with complex situations.

[0007] Therefore, there is an urgent need for an adaptive motion decomposition method for medical image registration. When faced with medical images of varying complexity, the adaptive recursive Transformer will decompose and refine them to different degrees, thereby achieving accurate and efficient registration. Summary of the Invention

[0008] This invention provides an adaptive motion decomposition method for medical image registration. Its purpose is to improve the medical registration model based on Tranformer motion decomposition, enabling it to automatically refine complex medical images of different regions to varying degrees, thereby improving the accuracy and efficiency of medical registration.

[0009] To achieve the above objectives, the present invention provides an adaptive motion decomposition method for medical image registration, specifically including the following steps:

[0010] S1. Construct a Transformer motion decomposition module to process the fixed image feature map F and the moving image feature map M to generate multiple motion subfields. Then, fuse the motion subfields through a competitive weighting module to obtain the initial deformation field.

[0011] S2. Construct an adaptive recursive module based on recursive Transformer, introduce a hierarchical LoRA module and set up a router network, and dynamically decide the number of recursive calculations based on the hidden state of each token in the image.

[0012] S3. Construct a primary medical registration model including the Transformer motion decomposition module and the adaptive recursive module, and train the primary medical registration model using a loss function. After training, the final medical registration model is obtained.

[0013] S4. Input the fixed image and the moving image into the final medical registration model, and finally output a full-resolution fine deformation field to achieve registration between the moving image and the fixed image.

[0014] Preferably, in S1, the processing procedure of the Transformer motion decomposition module specifically includes:

[0015] S11. Perform linear projection proj and LayerNorm normalization on each point on the fixed image feature map F and the moving image feature map M respectively to obtain the query vector Q and the key value vector K.

[0016] S12. For each query vector on the current scale feature map, perform a dot product operation with the key vector only within the local neighborhood window centered on its corresponding position, and calculate the similarity score.

[0017] S13. The similarity score is then normalized by the Softmax function after being biased by position to obtain a multi-head neighborhood attention map.

[0018] S14. Using the multi-head neighborhood attention map, the value vectors in the neighborhood are weighted and summed to calculate the regular displacement field, and a motion subfield with S possible motions for a point is generated.

[0019] Among them, obtaining the multi-head neighborhood attention graph The expression is:

[0020] ;

[0021] in, p These are the spatial coordinates on the feature map, representing the position of any pixel or voxel on the fixed image feature map F. s For the index of attention head, To fix the image position p The corresponding local neighborhood window in the moving image For the first s The position offset parameter of the attention head. T This represents transposition.

[0022] Preferably, in S1, the fusion process of the competition weighting module specifically includes:

[0023] S15. Perform upsampling processing on the motion subfield;

[0024] S16. Obtain the weights of each motion subfield through three layers of convolution;

[0025] S17. The corresponding motion subfields are weighted and summed according to their weights to obtain the initial deformation field after fusion.

[0026] The formula for calculating the weights and the weighted sum is as follows:

[0027] ;

[0028] ;

[0029] in, For the first s The fusion weight of each sports subfield, For the convolutional block ConvBlock used to compute the weights, cat For feature splicing operations, For the first S Each sports field This represents the initial deformation field after fusion.

[0030] Preferably, in S2, the specific content of the hierarchical LoRA module includes:

[0031] Based on the shared base layer weight matrix W′ of the recursive Transformer, a LoRA adapter is added for each iteration. This adapter learns a low-rank matrix to adjust the output vector. The calculation formula is:

[0032] ;

[0033] Where x is the input vector, This is an adjustment introduced by LoRA.

[0034] Preferably, in S2, the specific content of the router network includes:

[0035] S21. In each recursive step r, the router uses its learnable parameter weights and combines them with the hidden state of the token to calculate the importance score of each token through an activation function.

[0036] S22. Sort all tokens by importance score and select the top k tokens with the highest scores to proceed to the next recursive step;

[0037] S23. As the recursion depth increases, the range of active tokens is gradually narrowed, and subsequent recursive calculations are only performed on the filtered tokens.

[0038] Preferably, in S3, the formula for calculating the loss function is:

[0039] ;

[0040] in, For the total loss function, This is a similarity loss function used to measure the similarity loss of a fixed image. With the first d Moving the image after recursive deformation Similarity; It is a regularization loss used to ensure the smoothness and physical rationality of the deformation field; It is an auxiliary loss function used to stabilize the adaptive routing mechanism; , , Here, D is the weight hyperparameter, and D is the recursion depth.

[0041] Wherein, the auxiliary loss function The calculation formula is:

[0042] ;

[0043] Where N is the total number of tokens after feature map transformation. It is the router's output score for the token. It is the target label.

[0044] Preferably, S4 specifically includes:

[0045] Fixed and moving images are input into a two-stream convolutional network for encoding to obtain feature pyramids of different scales.

[0046] The ARModeT adaptive recursive motion decomposition module is recursively called from the coarsest to the finest scale to estimate the deformation increment at each scale and combine it with the coarse-scale deformation field to finally output a fine deformation field at full resolution.

[0047] Preferably, the specific working content of the ARModeT adaptive recursive motion decomposition module includes:

[0048] S41. Initialize the deformation field to zero field or the sampling result of the previous layer, and set the recursion depth D as a function D(x) that varies with the spatial position x;

[0049] S42. At each recursion depth d, deform the moving image features;

[0050] S43. Calculate the routing score for each spatial location x by using a small neural network combined with an activation function;

[0051] S44. Compare the routing score with the β percentile of the set of all scores G in the current step. Compare the results, generate a mask, and only retain routes with scores greater than [a certain value]. The token;

[0052] S45. Perform cross-attention calculation on the retained tokens and combine it with the PredictionHead function to obtain the deformation increment;

[0053] S46. Update the deformation field based on the deformation increment to complete the calculation of the current recursion depth until the set recursion depth is reached.

[0054] Therefore, the adaptive motion decomposition method for medical image registration described above has the following advantages compared with the prior art:

[0055] (1) The present invention decomposes complex deformation into multiple interpretable motion subfields through the Transformer motion decomposition module, and then captures the multi-mode motion of different tissues in medical images more accurately through competitive weighted fusion, which significantly improves the registration accuracy.

[0056] (2) The present invention introduces a layered LoRA module to achieve a lightweight design, which significantly reduces the number of parameters while maintaining performance; the number of recursive decisions in the router network dynamic decision-making is reduced, and computing resources are concentrated in the deformable complex area, which significantly improves inference efficiency and reduces computing power requirements.

[0057] (3) This invention considers the smoothness of the deformation field by regularization loss constraint to avoid distorted deformation; the auxiliary loss stable routing mechanism makes the deformation prediction more in line with the anatomical structure and physiological movement law, and improves the clinical usability of the registration results.

[0058] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0059] Figure 1 This is a diagram illustrating the overall architecture of the Transformer motion decomposition module in an embodiment of an adaptive motion decomposition method for medical image registration according to the present invention.

[0060] Figure 2 This is an architectural diagram of the competitive weighting module in an embodiment of an adaptive motion decomposition method for medical image registration according to the present invention;

[0061] Figure 3 This is a lightweight recursive Tranformer framework diagram in an embodiment of an adaptive motion decomposition method for medical image registration according to the present invention.

[0062] Figure 4 This is a diagram illustrating the implementation of an adaptive recursive Transformer in an embodiment of an adaptive motion decomposition method for medical image registration according to the present invention.

[0063] Figure 5 This is a network framework diagram of the final medical registration model in an embodiment of an adaptive motion decomposition method for medical image registration according to the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0065] Example 1

[0066] This embodiment demonstrates a specific application of the final medical registration model of the present invention.

[0067] first, Figure 1 The overall architecture of the Transformer motion decomposition module is shown, and the specific steps are as follows:

[0068] For each point on the fixed image feature map F and the moving image feature map M, linear projection proj and LayerNorm (LN) normalization are performed respectively to obtain the query vector Q and the key value vector K.

[0069] The query (Q) is generated by F. A Q vector is generated from the feature vector at each spatial location. Q can be understood as the question posed by each spatial location at the current scale regarding "how it should move." Both the key (K) and value (V) are generated from the moving image features M. K acts as a searchable "index label," while V stores the "details" corresponding to each K (such as feature appearance and potential displacement trends).

[0070] The formula for calculating the neighborhood attention of the s-th head at position p is expressed as follows:

[0071]

[0072] in, For linear projection, and These are the query vector and key vector of the S-th attention head, respectively.

[0073] For each query vector on the fixed-scale feature map, the module does not allow it to be computed with all keys K on the reference feature M. Instead, it predefines a local neighborhood window centered on its corresponding position and normalizes only all key vectors within its neighborhood window to generate a set of attention weights, i.e., similarity scores. After positional biasing, the similarity scores are normalized using the Softmax function to obtain a multi-head neighborhood attention map. The multi-head neighborhood attention map is used to perform a weighted summation of the value vectors within the neighborhood to calculate the regular displacement field, generating S possible motion subfields for a point.

[0074] Among them, obtaining the multi-head neighborhood attention graph The expression is:

[0075] ;

[0076] in, p These are the spatial coordinates on the feature map, representing the position of any pixel or voxel on the fixed image feature map F. s For the index of attention head, To fix the image position p The corresponding local neighborhood window in the moving image For the first s The position offset parameter of the attention head. T Represents transposition; Let be the query vector of the attention head at position p; To fix the image position p In the local neighborhood window corresponding to the moving image, at the attention head s, the key vector after transpose.

[0077] Then the competition-weighted module starts working, such as Figure 2 As shown: First, the motion subfields are upsampled; then, the weights of each motion subfield are obtained through three convolutional layers; the corresponding motion subfields are weighted and summed according to the weights, and a normalization function is used to compete for the motion modes of each voxel to obtain the fused initial deformation field. ;

[0078] The formula for calculating the weights and the weighted sum is as follows:

[0079] ;

[0080] ;

[0081] in, For the first s The fusion weight of each sports subfield, For the convolutional block ConvBlock used to compute the weights, cat For feature splicing operations, For the first S Each sports field This represents the initial deformation field after fusion.

[0082] The following is a description of the adaptive recursive module of the recursive Tranformer:

[0083] Figure 3 This demonstrates a lightweight recursive Transformer framework. Independent multi-layer Transformers have a large number of parameters, while recursive Transformers, by utilizing shared parameter settings, can significantly improve computational efficiency. For example, converting a standard Transformer (e.g., 18 layers) to a recursive architecture—that is, a block containing K layers (e.g., 9 layers) repeats B times (e.g., 2 times)—halves the number of parameters. In the "Recursive Transformer," we reduce model size by having multiple layers share the same set of parameters. However, this is like having an actor play multiple roles in different acts of a play; while it saves manpower, the actor may not perfectly adapt to the unique needs of each role, leading to performance degradation. The hierarchical LoRA module addresses this problem: it adds a lightweight, learnable "adapter" to each iteration on a shared "base layer," allowing the same set of parameters to fine-tune its behavior to adapt to tasks of varying depths. In this way, the model maintains a small size while gaining flexibility.

[0084] The specific contents of the hierarchical LoRA module include:

[0085] Based on the shared base layer weight matrix W′ of the recursive Transformer, a LoRA adapter is added for each iteration. This adapter learns a low-rank matrix to adjust the output vector. The calculation formula is:

[0086] ;

[0087] Where x is the input vector, This is an adjustment term introduced by LoRA, which decomposes it into the product of two smaller matrices: A is a dimension reduction matrix and B is a dimension increase matrix. The goal is to make the "shared weights + LoRA" as close as possible to the weights of the original model.

[0088] In traditional recursive Transformers, all vector tokens undergo the same number of recursive calculations, resulting in a waste of computational resources. This embodiment addresses this issue by introducing a lightweight router. The router is a small neural network that makes decisions in real-time based on the hidden state of each token. Figure 4 In each recursive step, the router selects the top k tokens to be processed, gradually narrowing the range of active tokens as the recursion depth increases. At each Transformer layer, the expert router network is responsible for evaluating each input token, allowing the k tokens with the highest evaluation scores to proceed to the next Transformer layer.

[0089] The specific content of the router network includes:

[0090] At each recursive step r, the router uses its learnable parameter weights, combined with the hidden state of the token, to calculate the importance score of each token through an activation function;

[0091] ;

[0092] in, It represents the hidden state of the t-th token before entering the r-th recursive block (which can be viewed as the information accumulated by the token up to this point). These are learnable parameter weights unique to the r-th router. It is an activation function, such as Sigmoid or Tanh, used to compress scores into a fixed range (e.g., 0 to 1).

[0093] After calculating the scores of all tokens, the router doesn't allow all tokens with scores exceeding a certain threshold to pass. Instead, it employs a method to ensure a fixed computational budget: it selects the top k tokens with the highest scores. Only tokens selected in step r are eligible to proceed to step r+1 for re-evaluation. Thus, as the recursion depth increases, the number of active tokens decreases, and computation is concentrated on the most "difficult" tokens.

[0094] Based on the Transformer motion decomposition module and the aforementioned adaptive recursive module, a primary medical registration model is constructed, such as... Figure 5 The network framework of the model is shown.

[0095] Figure 5 The left side shows the image encoding process of this network. Fixed and moving images are processed by a two-stream convolutional network consisting of convolutional layers and average pooling layers, outputting feature maps of different scales (M5, M4, M3, M2, M1; F5, F4, F3, F2, F1). Using the feature pyramids obtained in the encoding stage, the deformation field is recursively estimated and optimized from the coarsest scale (5) to the finest scale (1). At each scale, the deformation increment at the current scale is estimated using the ARModeT adaptive recursive motion decomposition module and combined with the deformation field from a coarser scale. Before each level of computation, the features of the moving image are first distorted (STN) with the currently estimated deformation field, enabling subsequent ARModeT modules to compute more accurate residual deformations on the pre-aligned features, simplifying the learning task. The final output is a fine-grained, full-resolution deformation field Φ used to align the moving image with the stationary image.

[0096] The specific tasks of the ARModeT module include:

[0097] Initialize the deformation field to zero field or the sampling result of the previous layer, and set the recursion depth D as a function D(x) that varies with the spatial position x;

[0098] At each recursion depth d, the moving image features are deformed:

[0099] ;

[0100] Warp is the warp operation; The deformation field is at a recursion depth of d-1; The features of the moving image after deformation at the recursion depth d;

[0101] Then, using a small neural network combined with an activation function, the routing score for each spatial location x is calculated:

[0102] ;

[0103] in, For normalization; For routing;

[0104] Compare the routing score with the β percentile of the set of all scores G for the current step. Compare the results, generate a mask, and only retain routes with scores greater than [a certain value]. The token;

[0105] Cross-attention is calculated on the retained tokens, and the deformation increment is obtained by combining it with the PredictionHead function:

[0106] ;

[0107] ;

[0108] in, For attention at recursion depth d; For cross-attention functions; For a binary decision function, ; For the changing deformation field component; The PreditionHead function is used to compete for weighted sub-deformation fields and synthesize the total deformation field;

[0109] The deformation field is updated based on the aforementioned deformation increment: It completes the calculation of the current recursion depth until the set recursion depth is reached.

[0110] After the model is built, the primary medical registration model is trained using a loss function, the formula for which the loss function is calculated is as follows:

[0111] ;

[0112] in, For the total loss function, This is a similarity loss function used to measure the similarity loss of a fixed image. With the first d Moving the image after recursive deformation Similarity; It is a regularization loss used to ensure the smoothness and physical rationality of the deformation field; It is an auxiliary loss function used to stabilize the adaptive routing mechanism; , , Here, D is the weight hyperparameter, and D is the recursion depth.

[0113] Wherein, the auxiliary loss function The calculation formula is:

[0114] ;

[0115] Where N is the total number of tokens after feature map transformation. It is the router's output score for the token. It is the target label.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An adaptive motion decomposition method for medical image registration, characterized in that, Specifically, the following steps are included: S1. Construct a Transformer motion decomposition module to process the fixed image feature map F and the moving image feature map M to generate multiple motion subfields. Then, fuse the motion subfields through a competitive weighting module to obtain the initial deformation field. S2. Construct an adaptive recursive module based on recursive Transformer, introduce a hierarchical LoRA module and set up a router network. Dynamically decide the number of recursive calculations based on the hidden state of each token in the image. The router will select the top k tokens to be processed, and gradually narrow the range of active tokens as the recursion depth increases. At each Transformer level, the expert router network is responsible for evaluating each input token and allowing the k tokens with the highest evaluation scores to enter the next Transformer level. S3. Based on the Transformer motion decomposition module and the adaptive recursive module, a primary medical registration model is constructed, and the primary medical registration model is trained using a loss function. After training, the final medical registration model is obtained. The formula for calculating the loss function is as follows: ； in, For the total loss function, This is a similarity loss function used to measure the similarity loss of a fixed image. With the first d Moving the image after recursive deformation Similarity; It is a regularization loss used to ensure the smoothness and physical rationality of the deformation field; It is an auxiliary loss function used to stabilize the adaptive routing mechanism; , , Here, D is the weight hyperparameter, and D is the recursion depth. Wherein, the auxiliary loss function The calculation formula is: ； Where N is the total number of tokens after feature map transformation. It is the router's output score for the token. It is the target label; S4. Input the fixed image and the moving image into the final medical registration model, and finally output the full-resolution deformation field to achieve the registration of the moving image and the fixed image.

2. The adaptive motion decomposition method for medical image registration according to claim 1, characterized in that, In S1, the processing procedure of the Transformer motion decomposition module specifically includes: S11. Perform linear projection proj and LayerNorm normalization on each point on the fixed image feature map F and the moving image feature map M respectively to obtain the query vector Q and the key value vector K. S12. For each query vector on the current scale feature map, perform a dot product operation with the key vector only within the local neighborhood window centered on its corresponding position, and calculate the similarity score. S13. The similarity score is then normalized by the Softmax function after being biased by position to obtain a multi-head neighborhood attention map. S14. Using the multi-head neighborhood attention map, the value vectors in the neighborhood are weighted and summed to calculate the regular displacement field, and a motion subfield with S possible motions for a point is generated. Among them, obtaining the multi-head neighborhood attention graph The expression is: ； in, p These are the spatial coordinates on the feature map, representing the position of any pixel or voxel on the fixed image feature map F. s For the index of attention head, To fix the image position p The corresponding local neighborhood window in the moving image For the first s The position offset parameter of the attention head. T This represents transposition.

3. The adaptive motion decomposition method for medical image registration according to claim 1, characterized in that, In S1, the fusion process of the competition weighting module specifically includes: S15. Perform upsampling processing on the motion subfield; S16. Obtain the weights of each motion subfield through three layers of convolution; S17. The corresponding motion subfields are weighted and summed according to their weights to obtain the initial deformation field after fusion. The formula for calculating the weights and the weighted sum is as follows: ； ； in, For the first s The fusion weight of each sports subfield, For the convolutional block ConvBlock used to compute the weights, cat For feature splicing operations, For the first S Each sports field This represents the initial deformation field after fusion.

4. The adaptive motion decomposition method for medical image registration according to claim 1, characterized in that, In S2, the specific contents of the hierarchical LoRA module include: Based on the shared base layer weight matrix W′ of the recursive Transformer, a LoRA adapter is added for each iteration. This adapter learns a low-rank matrix to adjust the output vector. The calculation formula is: ； Where x is the input vector, This is an adjustment introduced by LoRA.

5. The adaptive motion decomposition method for medical image registration according to claim 1, characterized in that, In S2, the specific content of the router network includes: S21. In each recursive step r, the router uses its learnable parameter weights and combines them with the hidden state of the token to calculate the importance score of each token through an activation function. S22. Sort all tokens by importance score and select the top k tokens with the highest scores to proceed to the next recursive step; S23. As the recursion depth increases, the range of active tokens is gradually narrowed, and subsequent recursive calculations are only performed on the filtered tokens.

6. The adaptive motion decomposition method for medical image registration according to claim 1, characterized in that, S4 includes the following specific content: Fixed and moving images are input into a two-stream convolutional network for encoding to obtain feature pyramids of different scales. The ARModeT adaptive recursive motion decomposition module is recursively called from the coarsest to the finest scale to estimate the deformation increment at each scale and combine it with the coarse-scale deformation field to finally output the full-resolution deformation field.

7. The adaptive motion decomposition method for medical image registration according to claim 6, characterized in that, The specific tasks of the ARModeT adaptive recursive motion decomposition module include: S41. Initialize the deformation field to zero field or the sampling result of the previous layer, and set the recursion depth D as a function D(x) that varies with the spatial position x; S42. At each recursion depth d, deform the moving image features; S43. Calculate the routing score for each spatial location x by using a small neural network combined with an activation function; S44. Compare the routing score with the β percentile of the set of all scores G in the current step. Compare the results, generate a mask, and only retain routes with scores greater than [a certain value]. The token; S45. Perform cross-attention calculation on the retained tokens and combine it with the PredictionHead function to obtain the deformation increment; S46. Update the deformation field based on the deformation increment to complete the calculation of the current recursion depth until the set recursion depth is reached.

Images