Medical image registration method based on frozen pre-training segmentation encoder transfer adaptation

Abstract

This invention discloses a medical image registration method based on frozen pre-trained segmentation encoder transfer adaptation. Its key feature is the use of a shared frozen 3D segmentation encoder as the feature backbone network. Multi-scale anatomical perceptual feature pyramids are extracted from the image to be registered and the target image. During the coarse-to-fine residual pyramid decoding process, feature space similarity constraints and a difference-product interaction fusion module are combined to predict deformation updates step-by-step and generate the final deformation field, thereby achieving accurate registration of the image to be registered to the target image. Compared with existing technologies, this invention utilizes anatomical prior information to improve the accuracy, deformation rationality, and cross-dataset generalization ability of medical image registration. It can adapt to different 3D pre-trained segmentation encoders and coarse-to-fine registration decoding architectures, improving the model's performance in medical image registration tasks. It also solves problems such as strong dependence on intensity similarity, insufficient robustness under limited data conditions, and weak generalization ability under domain offset conditions.

Description

Technical Field

[0001] This invention relates to the fields of medical image registration and deep learning technology, specifically a method based on frozen pre-processing.

[0002] Medical image registration method for training segmentation encoders for transfer adaptation Background Technology

[0003] The goal of medical image registration is to establish a spatial correspondence between the image to be registered and the target image to obtain a deformation field that can characterize local tissue displacement. While existing deep learning-based deformable registration methods can quickly output registration results through forward inference, they primarily rely on image intensity similarity metrics, such as mean squared error, mutual information, or local normalized cross-correlation. These methods are prone to matching ambiguities in low-texture regions, areas with blurred organ boundaries, and when there are appearance differences. Furthermore, if the registration encoder is trained from scratch, its ability to represent anatomical boundaries, spatial layout, and context is often limited by the size of the registration training set, resulting in insufficient robustness of the model to new datasets or anatomical distributions.

[0004] Existing techniques attempt to incorporate pre-trained representations or segmentation priors to enhance registration performance. However, directly utilizing features from the base model often requires additional adaptation, discrete search, or iterative optimization, resulting in significant engineering overhead. Methods relying on segmentation output-level guidance are typically limited by label coverage and segmentation quality, and errors may be amplified during the registration process.

[0005] In summary, existing registration methods are heavily reliant on intensity similarity, lack robustness under limited data conditions, and exhibit weak generalization ability under domain offset conditions. Therefore, how to enable registration networks to fully utilize transferable 3D anatomical representations and improve cross-dataset generalization ability without significantly increasing inference complexity remains a pressing technical problem to be solved. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by designing a medical image registration method based on frozen pre-trained segmentation encoder transfer adaptation. This method employs a shared frozen 3D segmentation encoder as the feature backbone network. Multi-scale anatomical perception feature pyramids are extracted from the image to be registered and the target image. During the coarse-to-fine residual pyramid decoding process, feature space similarity constraints and a difference-product interaction fusion module are combined to predict deformation updates level by level and generate the final deformation field, thereby achieving accurate registration of the image to be registered to the target image. This method transfers a large-scale pre-trained 3D segmentation encoder to the medical image registration network as a shared frozen feature backbone. It utilizes the anatomical boundaries, spatial layout, and multi-scale contextual features learned by the pre-trained segmentation model to construct a shared anatomical perception feature base. Based on this, the difference-product interaction fusion module and multi-scale feature space similarity constraints are combined to improve the accuracy of deformation field estimation, deformation rationality, and cross-dataset generalization ability. This invention improves the accuracy and stability of medical image registration without requiring de novo training of the registration encoder or direct participation of segmentation labels in registration training. It eliminates the need for additional fine-tuning or iterative optimization during the testing phase, effectively enhancing the accuracy, deformation reasonableness, and cross-dataset generalization ability of medical image registration. The invention is adaptable to different 3D pre-trained segmentation encoders and coarse-to-fine registration decoding architectures to improve model performance in medical image registration tasks. It effectively addresses the problems of existing registration methods, such as strong dependence on intensity similarity, insufficient robustness under limited data conditions, and weak generalization ability under domain offset conditions, demonstrating strong practicality and application prospects.

[0007] The specific technical solution for implementing this invention is: a transfer adaptation method based on a frozen pre-trained segmentation encoder.

[0008] Medical image registration methods are characterized by transferring large-scale pre-trained 3D segmentation encoders to medical image registration.

[0009] In the quasi-network, it serves as the backbone of shared frozen features, extracting multi-scale anatomical perception features in the shared feature space.

[0010] And it is connected to a coarse-to-fine residual pyramid registration decoder to predict the deformation field step by step; wherein, for the frozen encoding

[0011] The output feature design of the instrument incorporates a difference-product interactive fusion module to extract the difference and correlation features required for registration, generating an interactive representation for deformation field estimation. After completing the deformation field prediction at each layer, the module further utilizes the difference-product interactive fusion module to extract the difference and correlation features required for registration, generating an interactive representation for deformation field estimation.

[0012] The layer deformation field performs spatial transformation on the features of the image to be registered and calculates multi-scale features with the features of the target image.

[0013] Spatial similarity loss, thus enabling a registration network to be jointly optimized along with image similarity loss and deformation regularization loss.

[0014] Specifically, the following steps are included:

[0015] Step a: Input the image to be registered and the target image, and select the 3D segmentation encoder that has been pre-trained on a large-scale 3D medical image segmentation task as the shared frozen feature backbone network.

[0016] Step b: Using the freezing parameters from step a, extract multi-scale features of the image to be registered and the target image, construct a multi-scale feature pyramid in a shared feature space, and connect the multi-scale features to the coarse-to-fine residual pyramid registration decoder to predict the initial deformation update amount starting from the coarsest layer.

[0017] Step c: In each level, transform the features of the image to be registered after the current deformation estimation. Corresponding layer target image features The input is a difference-product interaction fusion module designed for frozen encoded features. Specifically, it constructs difference features using the difference between the two features. This is used to characterize local mismatch information; and the element-wise product of the two is used to construct product features. To characterize local matching and correlation information; then according to and The joint features generate corresponding channel weights, and the difference features and product features are weighted and fused to obtain an interactive representation for predicting the incremental deformation field of the current layer. This allows the anatomical sensing features output by the frozen encoder to be further transformed into differential and correlation information that directly serves deformation field estimation.

[0018] Step d: The interaction representation obtained in step c Input the residual flow field estimator to predict the incremental deformation field of the current layer, and combine it with the prior deformation field passed from the previous layer. Repeat the above process to obtain the corresponding deformation field of each layer and the final deformation field.

[0019] Step e: After obtaining the deformation fields of each layer, spatial transformation is performed on the corresponding layer's image features to be registered using the deformation fields of each layer, and multi-scale feature space similarity loss is calculated with the corresponding layer's target image features. This transforms the anatomical prior information provided by the frozen pre-trained segmentation encoder into structural consistency constraints during the registration process. The multi-scale feature space similarity loss is expressed by the following formula:

[0020] .

[0021] in, For the first Layer of image features to be registered For the first Layer target image features, For the first Layered deformation field, For the first The set of spatial locations of layer feature maps; Indicates the effect of the deformation field. For the first The total number of spatial locations in the layer feature map. This indicates the calculation of the cosine similarity between two feature vectors.

[0022] Step f: Use the final deformation field to perform spatial transformation on the image to be registered to obtain the registration result image. Then, use the multi-scale feature spatial similarity loss, image similarity loss, and deformation regularization loss as training objectives to optimize the registration network, thereby obtaining the optimized medical image registration network and achieving accurate registration of the image to be registered to the target image.

[0023] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0024] 1) This invention migrates a large-scale pre-trained 3D segmentation encoder to a medical image registration network as a shared frozen feature backbone, which can provide a stable anatomical perception feature base for the registration task and improve feature representation ability and cross-dataset generalization ability.

[0025] 2) This invention designs a difference-product interactive fusion module, which can simultaneously extract local mismatch information and local matching information, and transform them into an interactive representation that directly serves deformation field estimation, thereby improving the accuracy and stability of deformation field prediction.

[0026] 3) This invention introduces multi-scale feature space similarity loss, which can transform the anatomical prior information provided by the pre-trained segmentation encoder into structural consistency constraints in the registration process, thereby improving registration accuracy and robustness without relying on segmentation labels.

[0027] 4) This invention reduces the dependence of existing methods on grayscale consistency constraints by using a collaborative design of frozen feature backbone, interactive fusion modeling, and feature space constraints, thereby improving registration performance under complex deformation and domain offset conditions.

[0028] 5) This invention has good adaptability and transferability to a variety of different pre-trained 3D segmentation encoders, and can be used as a general paradigm design for medical image registration. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a pre-trained feature transfer framework. Figure 2 This is a schematic diagram illustrating the feature fusion and feature similarity constraint process. Figure 3This is a schematic diagram comparing the performance of the present invention with existing representative medical image registration methods. Detailed Implementation

[0030] See Figure 1 The present invention specifically includes the following steps:

[0031] Step a: Input the image to be registered and the target image, and use the frozen pre-trained 3D segmentation encoder to extract multi-scale anatomical perception features in the shared feature space.

[0032] Step b: Connect the multi-scale features to the coarse-to-fine residual pyramid registration decoder, and predict the incremental deformation field step by step from the coarsest layer to obtain the final deformation field.

[0033] Step c: In each level, input the deformed features of the image to be registered and the corresponding level's target image features into the difference-product interactive fusion module to generate an interactive representation for the deformation field prediction of the current level.

[0034] Step d: After obtaining the deformation fields of each layer, spatial transformation is performed on the features of the image to be registered using the corresponding layer deformation fields, and multi-scale feature space similarity loss is calculated with the features of the target image.

[0035] Step e: Use the multi-scale feature space similarity loss, image similarity loss, and deformation regularization loss as training objectives to optimize the medical image registration network.

[0036] Step f: After the model training is complete, fix the network parameters and output the final registration result from the image to be registered to the target image on the test set.

[0037] This invention directly integrates a frozen pre-trained 3D segmentation encoder into a medical image registration network as a shared feature backbone. Since the pre-trained segmentation encoder learns multi-scale anatomical representations for voxel-level structure prediction in segmentation tasks, its output hierarchical features simultaneously include organ boundaries, spatial layout, and contextual information. Furthermore, its U-Net-style multi-scale pyramid structure corresponds to the multi-level feature input format required by the coarse-to-fine registration decoder. Therefore, it can be directly transferred to the registration network as a replacement feature extraction part of the original de novo training registration encoder, providing a foundation for learning the correspondence between the image to be registered and the target image in a shared feature space.

[0038] See Figure 2 This invention mainly includes two key designs. Firstly, in... Figure 2In the interactive fusion section on the right, the deformed features of the image to be registered and the corresponding layer's target image features are input into the difference-product interactive fusion module. The difference between the two is used to construct difference features to represent local mismatch information; the element-wise product of the two is used to construct product features to represent local matching and correlation information; then, combined with the joint feature generation channel weights, the difference features and product features are weighted and fused to obtain the interactive representation used for deformation field prediction of the current layer. Secondly, after the deformation field prediction of each layer is completed, the corresponding layer's deformation field is used to perform spatial transformation on the features of the image to be registered in each layer, and multi-scale feature space similarity loss is calculated with the corresponding layer's target image features. This transforms the anatomical prior information provided by the frozen pre-trained segmentation encoder into structural consistency constraints in the registration process.

[0039] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0040] Example 1

[0041] This embodiment uses SAT-Pro, SAT-Nano, STU-Net-Base, and STU-Net-Large as frozen pre-trained 3D segmentation encoders, respectively, and performs training and testing on an abdominal CT dataset. During training, the Adam optimizer is used with a learning rate set to 1×10⁻⁶. -4 The batch size is set to 1, and the number of training iterations is set to 100k.

[0042] This embodiment selects the CorrMLP, IIRP-Net, and SACB-Net network architectures as comparison methods, and the comparison results are detailed in Table 1 below:

[0043] Table 1. Performance Comparison of Different Network Architectures with Existing Representative Medical Image Registration Methods

[0044]

[0045] Table 1 selects CorrMLP, IIRP-Net, and SACB-Net as comparison methods and presents the results of this invention under different pre-training bases. It can be seen that the proposed method significantly outperforms the existing methods in Dice metrics on the FLARE and AMOS datasets, and also has a lower TRE metric on the DIR-QA dataset, indicating that the proposed frozen pre-trained segmentation encoder transfer adaptation scheme can effectively improve medical image registration performance. Taking the FLARE dataset as an example, the Dice of CorrMLP, IIRP-Net, and SACB-Net are 67.2%, 67.5%, and 64.5%, respectively, while the proposed FSE-Reg(Pro) achieves 76.8% and FSE-Reg(B) achieves 76.4%. On the AMOS pure external test dataset, the proposed FSE-Reg(Pro) and FSE-Reg(B) achieve 50.4% and 50.3%, respectively, also outperforming existing comparative methods. On the DIR-QA dataset, which uses the target registration error TRE as an indicator, the proposed FSE-Reg(Pro) achieves a TRE of 6.4 mm, which is better than CorrMLP's 12.0 mm, IIRP-Net's 8.9 mm, and SACB-Net's 11.2 mm. These results demonstrate that the proposed method exhibits good registration accuracy and generalization ability on both in-domain and external datasets.

[0046] See Figure 3 This invention achieves registration and visualization results superior to existing representative methods, effectively utilizing the anatomical prior information learned by the pre-trained segmentation encoder to reduce mismatches and misallocations of organs and tissue structures.

[0047] The above embodiments are merely illustrative of the present invention and are not intended to limit the scope of the present invention. All equivalent implementations of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A medical image registration method based on frozen pre-trained segmentation encoder transfer adaptation, characterized in that, A method employing a large-scale pre-trained 3D segmentation encoder, transferred to a medical image registration network as a shared frozen feature backbone, extracts multi-scale anatomical perception features in a shared feature space from the image to be registered and the target image. These features are then fed into a coarse-to-fine residual pyramid registration decoder to predict the deformation field level by level. The corresponding layer deformation field is used to perform spatial transformation on the features of the image to be registered. Multi-scale feature space similarity loss is calculated between this loss and the target image features. This loss, along with image similarity loss and deformation regularization loss, optimizes the registration network, enabling registration of the image to be registered to the target image. The specific steps include: Step a: Input the image to be registered and the target image, and select a 3D segmentation encoder pre-trained on a large-scale 3D medical image segmentation task as the shared frozen feature backbone network; Step b: Extract multi-scale features of the image to be registered and the target image using the freezing parameters, construct a multi-scale feature pyramid in the shared feature space, and connect the multi-scale features to the coarse-to-fine residual pyramid registration decoder to predict the initial deformation update amount starting from the coarsest layer; Step c: In each level, transform the features of the image to be registered after the current deformation estimation. Corresponding layer target image features The input difference-product interaction fusion module constructs difference features using the difference between the two. To characterize local mismatch information; and to construct product features using the element-wise product of the two. To characterize local matching and correlation information; based on and The joint features generate corresponding channel weights, and the differential features are used to generate the channel weights. Sum of product features Weighted fusion is performed to obtain an interactive representation for predicting the incremental deformation field of the current layer. This allows the anatomical sensing features output by the freeze encoder to be transformed into differential and correlational information in the deformation field estimation; Step d: Represent the interaction Input the residual flow field estimator to predict the incremental deformation field of the current layer, and combine it with the prior deformation field passed from the previous layer. Repeat the above process to obtain the corresponding deformation field of each layer and the final deformation field. Step e: Utilize the deformation fields of each layer to perform spatial transformations with the corresponding layer's image features to be registered, and calculate the multi-scale feature space similarity loss with the corresponding layer's target image features. This transforms the anatomical prior information provided by the frozen pre-trained segmentation encoder into structural consistency constraints during the registration process. The multi-scale feature space similarity loss is expressed by the following formula: ； in, For the first Features of the image to be registered; For the first Layer target image features; For the first Layered deformation field; For the first The set of spatial locations of layer feature maps; This indicates the composition of functions; For the first The total number of spatial locations in the layer feature map; This indicates the calculation of the cosine similarity between two feature vectors; Step f: Use the final deformation field to perform spatial transformation on the image to be registered to obtain the registration result image. Use the multi-scale feature spatial similarity loss, image similarity loss and deformation regularization loss as training objectives to optimize the registration network and obtain the optimized 3D medical image registration network. Then, register the image to be registered to the target image.

Images