Prototype-enhanced medical image registration method, device, server and medium

Abstract

The application discloses a medical image registration method and device based on prototype enhancement, a server and a medium, and belongs to the technical field of image registration. The method comprises the following steps: after three-dimensional medical moving and fixed images are extracted into multi-scale features by a pre-training encoder, the images are input into an APE dissection prototype enhancement module to obtain moving and fixed projection features of each scale, and the similarity is calculated with corresponding dissection prototypes, and a prototype allocation map is obtained through Softmax; three-dimensional space smoothing and prototype weighting are performed to reconstruct moving and fixed prototype reconstruction features; prototype sets of each scale are spliced, and the reconstructed features are input into an attention enhancement module to perform prototype attention interaction and context enhancement, so that enhanced moving and fixed features are obtained, a multi-scale residual deformation field module is input to estimate a target deformation field, and a registration image is obtained by resampling the moving image. Through multi-scale feature coding, dissection prototype enhancement, attention interaction and multi-scale residual deformation field estimation, the registration of three-dimensional medical images is realized.

Description

Technical Field

[0001] This invention relates to the field of image registration technology, and in particular to a medical image registration method, apparatus, server, and medium based on prototype enhancement. Background Technology

[0002] Three-dimensional medical image registration can map medical images from different individuals, at different times, or under different imaging conditions to a unified space, providing important support for clinical applications such as brain image analysis, disease follow-up, surgical navigation, radiotherapy planning, and multimodal fusion.

[0003] Current mainstream learning-based registration methods rely on convolutional neural networks and pyramid networks to directly predict dense deformation fields, offering a significant advantage in inference speed compared to traditional iterative optimization schemes. Most existing 3D medical image registration networks use grayscale intensity features and conventional convolutional features from moving and stationary images as input, and complete deformation field prediction through feature concatenation, difference operations, correlation calculations, or residual decoders.

[0004] However, existing technologies are prone to problems such as mismatch of anatomical structures and unreasonable tissue deformation, which not only reduce the image registration effect, but also interfere with lesion identification and tissue localization, seriously restricting the actual effect of various clinical applications. Summary of the Invention

[0005] This invention provides a medical image registration method, device, server, and medium based on prototype enhancement to solve the technical problems of anatomical structure mismatch, unreasonable tissue deformation, and insufficient registration accuracy in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a medical image registration method based on prototype enhancement, comprising: The three-dimensional medical moving images and three-dimensional medical stationary images are processed by a pre-trained encoder to extract the moving image features and stationary image features at each scale; The moving and fixed image features at each scale are input into the APE anatomical prototype enhancement module. The moving and fixed projection features at each scale are obtained through projector operation. Under the adjustment of temperature parameters, the similarity between the moving and fixed projection features and the corresponding anatomical prototypes is calculated, and the prototype assignment map is obtained through softmax processing. The anatomical prototype is a feature template that characterizes the anatomical structure of the medical image. The prototype allocation map is smoothed in three dimensions and combined with anatomical prototype weighting to reconstruct the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. The anatomical prototypes are spliced ​​together to generate a prototype set at the corresponding scale. The mobile prototype reconstruction features and fixed prototype reconstruction features at each scale are input into the attention enhancement module along with the prototype set at the corresponding scale. Prototype attention interaction and context feature enhancement are performed to obtain enhanced mobile features and enhanced fixed features at each scale. The enhanced moving features and enhanced fixed features at each scale are input into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and displacement-corrected to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and displacement-corrected and applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field. The three-dimensional medical moving image is resampled using the target deformation field to obtain a registered image.

[0007] Secondly, embodiments of the present invention also provide a prototype-enhanced medical image registration device, comprising: The extraction module is used to process 3D medical moving images and 3D medical stationary images through a pre-trained encoder to extract moving image features and stationary image features at various scales. The calculation module is used to input the moving image features and fixed image features at each scale into the APE anatomical prototype enhancement module. The moving projection features and fixed projection features at each scale are obtained through the projector operation. Under the adjustment of the temperature parameter, the moving projection features and fixed projection features are compared with the anatomical prototypes at the corresponding scales. The prototype assignment map is obtained by softmax processing. The anatomical prototype is a feature template that represents the anatomical structure of the medical image. The reconstruction module is used to perform three-dimensional spatial smoothing and anatomical prototype weighted combination on the prototype allocation map to reconstruct the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. An enhancement module is used to stitch together the anatomical prototypes to generate a prototype set at the corresponding scale. The moving prototype reconstruction features and fixed prototype reconstruction features at each scale are input into the attention enhancement module along with the prototype set at the corresponding scale to perform prototype attention interaction and context feature enhancement, thereby obtaining enhanced moving features and enhanced fixed features at each scale. The deformation prediction module is used to input the enhanced movement features and enhanced fixed features at each scale into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and displacement corrected to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and displacement corrected and then applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field. The registration module is used to resample the three-dimensional medical moving image using the target deformation field to obtain a registered image.

[0008] Thirdly, embodiments of the present invention also provide a server, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the prototype-enhanced medical image registration method provided in the above embodiments.

[0009] Fourthly, embodiments of the present invention also provide a medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the prototype-enhanced medical image registration method provided in the above embodiments.

[0010] The medical image registration method, apparatus, server, and medium based on prototype enhancement provided in this invention process 3D moving and fixed medical images through a pre-trained encoder to extract moving and fixed image features at each scale. These features are then input into an APE anatomical prototype enhancement module. A projector operation yields moving and fixed projection features at each scale. Under temperature parameter adjustment, the moving and fixed projection features are compared with corresponding anatomical prototypes at different scales for similarity calculation, and then processed using softmax to obtain a prototype allocation map. The anatomical prototype serves as a feature template representing the anatomical structure of the medical image. The prototype allocation map is then subjected to 3D spatial smoothing and anatomical prototype weighted combination to reconstruct moving and fixed prototype reconstruction features at each scale. Finally, the anatomical prototype is stitched together to generate... A prototype set corresponding to a given scale is used. The reconstructed features of the moving prototypes and the reconstructed features of the fixed prototypes at each scale are input into the attention enhancement module along with the prototype set at the corresponding scale. Prototype attention interaction and context feature enhancement are performed to obtain enhanced moving features and enhanced fixed features at each scale. The enhanced moving features and enhanced fixed features at each scale are then input into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and shifted to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and shifted and then applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field. The target deformation field is then used to resample the three-dimensional medical moving image to obtain the registered image. By introducing anatomical prototypes to complete feature reconstruction and attention interaction enhancement, and combining multi-level residual structure to progressively predict deformation fields, we can achieve accurate 3D medical image registration with reasonable tissue deformation and matching of anatomical structures. Attached Figure Description

[0011] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0012] Figure 1 This is a flowchart of the medical image registration method based on prototype enhancement provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the APE anatomical prototype enhancement module of the medical image registration method based on prototype enhancement provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the attention enhancement module in the prototype-enhanced medical image registration method provided in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the structure of the multi-scale residual deformation field module of the medical image registration method based on prototype enhancement provided in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the structure of the medical image registration device based on prototype enhancement provided in Embodiment 2 of the present invention; Figure 6 This is a structural diagram of the server provided in Embodiment 3 of the present invention. Detailed Implementation

[0013] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0014] Example 1 Figure 1 This is a flowchart of a medical image registration method based on prototype enhancement provided in Embodiment 1 of the present invention. This embodiment is applicable to the registration of three-dimensional medical images of different individuals, at different times, and under different imaging conditions. Specifically, it includes the following steps: Step 110: The three-dimensional medical moving image and the three-dimensional medical fixed image are processed by a pre-trained encoder to extract the moving image features and fixed image features at each scale.

[0015] In 3D medical image registration tasks, moving 3D medical images and stationary 3D medical images are paired input data. The stationary 3D medical image serves as the reference benchmark for the registration process; its spatial position and anatomical coordinates remain constant, making it the standard for determining the accuracy of the registration result. The moving 3D medical image is the image to be registered, and it differs from the stationary image in terms of spatial offset and tissue deformation. Therefore, a registration algorithm is needed to transform its coordinates to align it with the spatial coordinate system of the stationary image.

[0016] For example, encoding and feature extraction operations are performed on the input 3D medical moving images and 3D medical stationary images respectively. Since a 2D pre-trained medical image encoder is used for processing, each 3D image is first decomposed along the depth direction and a slice sequence is constructed. For the z-th slice in the depth dimension, its adjacent z-1, z, and z+1 slices are selected and combined into a three-channel image. The boundary positions of the image edges are filled with duplicate data to ensure the integrity of the slice sequence. Optionally, the assembled three-channel slice sequence is then input into a MedSAM2 pre-trained encoder with frozen parameters, and the encoder outputs 2D feature maps at multiple scales. Finally, the 2D feature maps at different scales are reassembled along the depth dimension to construct a 3D feature pyramid, ultimately outputting the moving image features and stationary image features at the corresponding scales. Using the pre-trained encoder to perform feature transformation on the original 3D medical image effectively extracts texture information and anatomical structure features at different levels of the image.

[0017] Step 120: Input the moving image features and fixed image features at each scale into the APE anatomical prototype enhancement module. Obtain the moving projection features and fixed projection features at each scale through projector operation. Under the adjustment of temperature parameters, calculate the similarity between the moving projection features and fixed projection features and the corresponding anatomical prototypes at the same scale. Then, obtain the prototype assignment map through softmax processing. The anatomical prototype is a feature template that characterizes the anatomical structure of the medical image.

[0018] Anatomical prototypes are learnable feature templates defined for typical anatomical structures in 3D medical images. Each processing scale corresponds to an independently configured set of prototypes. These prototypes are dynamically optimized through model training, enabling them to capture common feature patterns of anatomical structures such as organs and tissues at different levels. This provides structured anatomical prior information for the registration process, unlike traditional methods that rely solely on pixel grayscale or general convolutional features. This approach can effectively guide subsequent feature enhancement and deformation estimation processes.

[0019] The moving image features and fixed image features at various scales output by the pre-trained encoder are subjected to projection transformation operations through a projector, for example, such as... Figure 2 As shown, for the first Three-dimensional input features at each scale In other words, the projector first transforms and adapts the feature channels to dimensions using a 3D convolution Conv3D operation, then constrains the feature distribution using a normalization Norm operation, and finally introduces a nonlinear expression through the ReLU activation function to obtain the projected features at the corresponding scale. This refers to either moving projection features or fixed projection features. This process maps the original image features to a feature space that fits the anatomical prototype, providing standardized feature inputs for subsequent prototype similarity calculations and feature reconstruction.

[0020] The temperature parameter, in this embodiment, is a configurable hyperparameter used to adjust the similarity distribution characteristics between voxel features and anatomical prototypes. Its core function is to control the amplification or compression of similarity differences. When the temperature parameter value is small, the similarity differences between voxels and different anatomical prototypes are amplified, and voxels tend to be classified as the single best-matching anatomical prototype. When the temperature parameter value is large, the similarity differences are smoothly compressed, and voxels can simultaneously obtain the weights of multiple related anatomical prototypes, adapting to the allocation requirements of the number of anatomical prototypes at different scales and ensuring the rationality of semantic matching between voxels and anatomical structures.

[0021] Under the influence of temperature parameters, similarity calculations and prototype assignment operations are performed on the moving and fixed projection features at each scale. For example, learnable anatomical prototypes are randomly initialized for different processing scales. The number of anatomical prototypes varies for each scale: 24 for the coarse scale, 10 for the medium scale, and 6 for the fine scale, forming prototype libraries for each scale. Each anatomical prototype serves as a feature template for a corresponding typical anatomical structure (such as the brainstem, cerebellum, and hippocampus), used to characterize common anatomical patterns in three-dimensional medical images. Secondly, regarding temperature parameters... Under the moderating effect, the similarity between the projection features of each voxel position v and the anatomical prototypes at that scale is calculated: for the moving projection features or fixed projection features at the i-th scale. The normalized dot product similarity formula is used to calculate its similarity with the k-th anatomical prototype. The similarity is calculated using the formula: ,in, This is a feature normalization operation used to eliminate the influence of differences in feature distribution; temperature parameter. The distribution characteristics used to adjust similarity control the smoothness of subsequent prototype assignments. The smaller the value, the more the similarity difference between the voxel and different prototypes is amplified, and the more the subsequent allocation is biased towards a single prototype. The larger the value, the more smoothly the similarity differences are compressed, resulting in a more even distribution and adaptability to different scales of prototype quantity configurations. Subsequently, along the prototype dimension, i.e., the i-th scale... The anatomical prototype dimension corresponds to 24, 10, and 6 prototype dimensions at three scales, affecting the similarity results. After performing softmax normalization, the similarity is converted into the probability of voxel position v belonging to each anatomical prototype, resulting in a prototype assignment map. The moving projection feature generates a moving prototype allocation map, and the fixed projection feature generates a fixed prototype allocation map. Each voxel position v in the allocation map corresponds to a set of values ​​of length [missing information]. The probability vector is represented by each component, which indicates the weight of the voxel belonging to the corresponding anatomical prototype, thus completing the semantic association between voxel features and anatomical prototypes.

[0022] Step 120 achieves accurate semantic matching between voxel projection features and anatomical prototypes at various scales through normalized dot product similarity calculation and softmax normalization under temperature parameter adjustment, adapting to the representation requirements of anatomical structures at different scales. The generated prototype assignment map clarifies the assignment weight of each voxel to each anatomical prototype, providing structured prior support for subsequent feature reconstruction based on anatomical prototypes and effectively enhancing the anatomical structural consistency of the features.

[0023] Step 130: Perform three-dimensional spatial smoothing and anatomical prototype weighted combination on the prototype allocation map to reconstruct the moving prototype reconstruction features and fixed prototype reconstruction features at each scale.

[0024] The purpose of performing 3D spatial smoothing and anatomical prototype weighted combination on the prototype assignment map is to eliminate the problem of inconsistent voxel assignment caused by local noise and discretization response during the prototype assignment process. At the same time, it integrates the semantic information of multiple anatomical prototypes to improve the consistency of the anatomical structure of the reconstructed features and provide robust structured feature input for subsequent processes.

[0025] For example, the APE anatomical prototype enhancement module performs 3D spatial smoothing and anatomical prototype weighted combination operations sequentially on the moving prototype assignment maps and fixed prototype assignment maps at each scale. First, 3D spatial smoothing is performed on the prototype assignment maps at each scale corresponding to the moving and fixed projection features. This 3D spatial filtering suppresses local noise and discretized responses generated during prototype assignment, making the probability distribution of voxels assigned to anatomical prototypes more consistent with the spatial continuity of anatomical structures in 3D medical images, avoiding isolated prototype assignment points, and ensuring spatial consistency of the assignment results. Then, using the smoothed prototype assignment map, a weighted combination of anatomical prototypes at that scale is performed to calculate the prototype reconstruction features. The specific formula is as follows: ,in, This is the current voxel position. Indicates the first An anatomical prototype. For the first Number of anatomical prototypes at each scale For the prototype assignment map, i.e., the smoothed voxel positions For the first The probability of assignment to each anatomical prototype. Let be the k-th anatomical prototype at the i-th scale. Through this weighted summation operation, the reconstructed features of each voxel location are composed of the weighted sum of its assignment probability to each anatomical prototype and the corresponding prototype features. Specifically, the moving prototype assignment map and the corresponding scale's anatomical prototype are weightedly combined to obtain the moving prototype reconstruction features, and the fixed prototype assignment map and the corresponding scale's anatomical prototype are weightedly combined to obtain the fixed prototype reconstruction features, thus completing the feature reconstruction process based on anatomical prototypes.

[0026] It should be noted that the APE anatomical prototype enhancement module introduces a multi-task loss function during the training phase to constrain and optimize each anatomical prototype and the prototype allocation process, in order to ensure the consistency of anatomical semantics and spatial continuity of feature reconstruction. Specifically, this includes four core constraints, the first being the prototype reconstruction loss, formulated as follows: ,in, For scale quantity Indicates the first The set of all voxel locations at each scale. For the i-th scale of the voxel position, consider the moving or fixed projection feature. For the moving prototype reconstruction features and fixed prototype reconstruction features at the i-th scale of the voxel location, the mean square error constraint between the reconstructed features and the original projected features ensures that the anatomical prototype effectively covers the input feature space, avoiding the prototype becoming a free parameter without actual semantics. Secondly, cross-scale prototype consistency loss is used, firstly by assigning the prototype map from the previous scale... Upsample to the spatial dimensions of the current scale to obtain the aligned distribution map. Then, the number of prototype channels is reduced from [previous number] using a convolution operation. Mapped to Then, normalization is performed on the prototype channel dimension to obtain the predicted allocation map. The final loss is defined as ,in, For scale quantity For predicting the allocation map, Assign a prototype map to the i-th scale. Cross-scale prototype consistency loss maintains semantic consistency by constraining prototype responses between adjacent scales, reducing semantic drift between different scales, and enhancing the overall consistency of multi-scale anatomical features.

[0027] Thirdly, spatial smoothing loss, firstly, the voxel at the i-th scale is... The assignment probabilities for all anatomical prototypes are represented in vector form. ,in For the prototype assignment map, i.e., the smoothed voxel positions For the first The probability of assigning each anatomical prototype is then determined using the formula. Define the loss, where, For scale quantity, Indicates the first The set of all voxel locations at each scale. For the first voxels at individual scales The probability of assignment to all anatomical prototypes. For the first voxels at individual scales The probability of assignment to all anatomical prototypes. Given the set of adjacent voxel offsets in the depth, height, and width directions, when When the image boundary is exceeded, the neighborhood term is not included in the calculation. This loss is calculated by calculating the L1 distance between the prototype assignment vectors of adjacent voxels, which constrains the spatially adjacent positions to have similar anatomical prototype responses, making the prototype assignment map more continuous and smooth in space, and suppressing the influence of local isolated responses and noisy assignments on the subsequent registration process.

[0028] Fourth, there is the label-guided loss, which involves resampling the segmentation labels to the first... Each scale is used to obtain a binary mask corresponding to each anatomical label, and then the soft Dice similarity formula is applied. Calculate the similarity between each anatomical label region and the prototype assignment map, where, To prevent constants with zero denominators, a one-to-one matching process is then performed between anatomical labels and prototypes based on similarity to obtain a matching set. Four sub-losses are then defined based on the matching results to cover the loss. Constrain the prototype to cover the corresponding anatomical label area as much as possible, with mutual exclusion loss. To avoid matching prototype responses to other label regions, voxel-level prototype classification loss is used. Voxels in the labeled regions are assigned to corresponding prototype numbers to enhance classification consistency; mismatched prototypes suppress loss. To avoid unmatched prototypes occupying the label area, the final... Anatomical tag-guided loss at each scale is The overall loss across multiple scales is ,in For the first Anatomical guidance weights at various scales. Indicates the first The set of all voxel locations at each scale. To resample to the first After the first scale, the second The binary mask corresponding to the anatomy label. For the first voxels at various scales Belongs to the The probability of assignment to each anatomical prototype. For the first A set of one-to-one matching anatomical labels and anatomical prototypes at each scale. For the first Anatomical tags and the first Each anatomical prototype is a semantic matching pair. For the first Union mask of all selected anatomical label regions at each scale For the first A set of voxels within a selected anatomical label region at each scale. For the first voxels at various scales It belongs to its corresponding real prototype The probability of allocation, For the first A collection of anatomical prototypes at various scales that have not been matched with any anatomical labels. , , and These are the weight coefficients for the coverage loss, mutual exclusion loss, voxel-level classification loss, and unmatched prototype suppression loss, respectively. This loss can guide the learnable prototype to establish a clear correspondence with the real anatomical structural region, improving the anatomical semantic interpretability and structural expressiveness of the prototype.

[0029] Step 140: Concatenate the anatomical prototypes to generate a prototype set at the corresponding scale. Input the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale and the prototype set at the corresponding scale into the attention enhancement module to perform prototype attention interaction and context feature enhancement, thereby obtaining enhanced mobile features and enhanced fixed features at each scale.

[0030] For example, a layered stitching approach is used to generate prototype sets for each scale. For the coarse scale, the coarse-scale anatomical prototypes are directly used as the coarse-scale prototype set, retaining only the coarse-scale prototype tokens. For the meso-scale, the meso-scale anatomical prototypes are stitched together with the coarse-scale anatomical prototypes to generate a meso-scale prototype set containing both meso- and coarse-scale prototype tokens. For the fine scale, the fine-scale, meso-scale, and coarse-scale anatomical prototypes are all stitched together to generate a fine-scale prototype set containing prototype tokens for all three scales. The core purpose of this layered stitching method is to reduce the memory overhead and computational cost of 3D attention calculations.

[0031] To address the issue that prototype reconstruction features only achieve static weighted fusion from voxels to anatomical prototypes and lack cross-prototype global context association and cross-image semantic interaction, it is necessary to use an attention enhancement module to perform prototype attention interaction and context feature enhancement, strengthen the consistency of the anatomical structure of the features, and provide robust structured feature input for subsequent multi-scale deformation field estimation.

[0032] For example, such as Figure 3 As shown, the attention enhancement module performs attention interaction and context enhancement on the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale, and the corresponding scale prototype set. First, the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale are used as voxel feature inputs, and then mapped through 1×1×1 convolutions. Generate query vectors to represent the query requirements for anatomical prototypes based on voxel features. Simultaneously, generate multi-scale prototype token sets corresponding to the specified scales. As prototype input, respectively through linear mapping and Key and value vectors are generated. Since the number of prototype tokens is significantly smaller than the number of 3D voxels, this design transforms the attention computation from a global correspondence search from voxel to voxel to a semantic retrieval from voxel to prototype token, reducing memory and computational overhead while preserving global anatomical semantic information. Subsequently, the moving and fixed query vectors are dot-producted with the transpose of the key vectors at the corresponding scales. After dividing by the square root of the dimension of a single attention head channel, softmax normalization is performed to obtain the moving and fixed attention weights for each scale. These weights represent the degree of attention each voxel location receives to different prototype tokens. Next, the attention weights are weighted and aggregated with the value vectors at the corresponding scales to obtain the moving and fixed anatomical context features for each scale. These features incorporate the semantic information of the global anatomical prototypes. Finally, the obtained anatomical context features are adjusted by output convolution mapping to make their channel number consistent with that of the original prototype reconstruction features. Then, they are multiplied by a learnable scaling factor, which is initialized to 0 and gradually increased during training to control the injection intensity of the prototype context. This is then added to the original moving or fixed prototype reconstruction features to obtain enhanced moving features and enhanced fixed features at each scale, thus completing the context feature enhancement process based on prototype attention.

[0033] The attention enhancement module models the semantic association between voxel features and global anatomical prototypes through attention interactions using a set of prototype tokens. This effectively supplements global contextual information across prototypes, significantly improving the anatomical semantic consistency and structural expressiveness of the features. Using prototype tokens as attention key-value pairs significantly reduces the memory overhead and computational cost of 3D attention calculations. While balancing local detail representation at different scales with global semantic modeling, it provides robust enhanced feature inputs for subsequent multi-scale deformation field estimation.

[0034] Step 150: Input the enhanced moving features and enhanced fixed features at each scale into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and displacement-corrected to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and displacement-corrected and applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field.

[0035] The deformation field is a core parameter used in 3D medical image registration to describe the voxel-level spatial position mapping relationship between moving and fixed images. It consists of a 3D displacement vector, with each voxel position corresponding to a set of displacement values ​​in the (x, y, z) directions, representing the coordinate transformation of the voxel from the moving image space to the fixed image space.

[0036] To address the challenges of single-scale deformation estimation in simultaneously aligning large global displacements with compensating for local anatomical details and inducing local registration errors and structural misalignments, a multi-scale residual deformation field module is needed. This module uses a three-tiered approach—coarse, medium, and fine—to estimate global displacements and local residual deformations, achieving accurate registration from coarse to fine.

[0037] For example, such as Figure 4 As shown, firstly, in the coarse-scale registration layer, the coarse-scale enhanced moving features and enhanced fixed features are input into the residual deformation decoder. The decoder first aggregates the dual-path feature information along the channel dimension through a splicing operation, and then performs shallow feature encoding on the three-dimensional volume data through a 3D convolution operation. Subsequently, it extracts high-dimensional features and suppresses gradient degradation through two residual convolution operations. Finally, it performs feature dimension transformation through a 3D convolution operation and predicts the initial residual deformation field.

[0038] Subsequently, the initial residual deformation field is upsampled and displacement corrected, with the displacement amplitude adjusted according to the scale ratios in the depth, height, and width directions to obtain the initial deformation field. This scale is primarily responsible for capturing the overall displacement of large-scale anatomical structures, resolving the global alignment problem. Next, the process proceeds to the mesoscale registration layer. First, the upsampled and corrected initial deformation field is used to perform a spatial pre-transformation (Warp operation) on the mesoscale enhanced motion features, generating mesoscale transformed enhanced motion features. Then, this transformed feature, along with the mesoscale enhanced fixed features, is input into the residual deformation decoder to generate the mesoscale residual deformation field. Next, the mesoscale residual deformation field is added to and fused with the initial deformation field, and upsampling and displacement correction operations are performed again to obtain the mesoscale deformation field. This step, after aligning the global displacement through pre-transformation, estimates the local displacement residuals of the mid-level anatomical structures, achieving compensation for mid-level details. Finally, in the fine-scale registration layer, the upsampled and corrected mesoscale deformation field is used to perform a spatial pre-transformation on the fine-scale enhanced motion features to generate fine-scale transformed enhanced motion features. These features, along with the fine-scale enhanced fixed features, are input into the residual deformation decoder to generate the fine-scale residual deformation field. The fine-scale residual deformation field is then added to and fused with the mesoscale deformation field to obtain the fine-scale deformation field.

[0039] Finally, upsampling and displacement correction operations are performed on the fine-scale deformation field to generate the final target deformation field. The three-layer registration estimation mode (coarse, medium, and fine) can not only align the overall spatial position of the image but also accurately correct minor offsets in local anatomical structures, effectively improving the overall registration accuracy. The deformation fields at each layer are upsampled and shifted, then progressively fused to ensure the continuity and smoothness of the displacement transformation. The output target deformation field can stably achieve accurate spatial transformation from a moving image to a fixed image.

[0040] Step 160: Resample the three-dimensional medical moving image using the target deformation field to obtain a registered image.

[0041] By employing a spatial transformer and utilizing the final estimated target deformation field, a resampling operation is performed on the 3D medical moving image to achieve voxel-level spatial coordinate mapping. Specifically, for each voxel in the moving 3D medical image, the corresponding grayscale value is sampled from the original moving image based on the 3D displacement vector at its corresponding position in the target deformation field. This resampling process can be viewed as a voxel position offset operation based on the displacement vector, causing the anatomical structures in the moving image to deform along the guidance of the target deformation field, gradually aligning them to the spatial coordinate system of the fixed image, ultimately resulting in a registered image whose spatial position is consistent with that of the anatomical structures in the fixed image. The resampling operation achieves precise voxel-level spatial mapping based on the target deformation field, enabling spatial alignment between the anatomical structures in the moving image and the fixed image, resolving the positional deviation problem between the moving and fixed images, and resulting in a registered image that is highly consistent with the fixed image in the spatial position of the anatomical structures.

[0042] This embodiment processes 3D moving and stationary medical images separately using a pre-trained encoder to extract moving and stationary image features at each scale. These features are then input into the APE anatomical prototype enhancement module. Projector operations generate moving and stationary projection features at each scale. Under temperature parameter adjustment, the moving and stationary projection features are compared with corresponding anatomical prototypes at their similarity scales, and softmax processing is used to obtain a prototype allocation map. The anatomical prototypes serve as feature templates representing the anatomical structures of the medical images. The prototype allocation map is then subjected to 3D spatial smoothing and weighted combination with the anatomical prototypes to reconstruct the moving and stationary prototype reconstruction features at each scale. Finally, the anatomical prototypes are stitched together to generate a prototype set at the corresponding scale. The reconstructed features of the moving prototype and the fixed prototype, along with the corresponding scale prototype sets, are input into the attention enhancement module for prototype attention interaction and contextual feature enhancement, resulting in enhanced moving features and enhanced fixed features at each scale. These enhanced moving and fixed features are then input into a multi-scale residual deformation field module to estimate the target deformation field. This module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and shifted to serve as the base deformation for the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and shifted, then applied to the fine-scale registration layer for residual deformation estimation and fusion to generate the target deformation field. The target deformation field is then used to resample the 3D medical moving image to obtain the registered image. By introducing anatomical prototypes to complete feature reconstruction and attention interaction enhancement, and combining this with a multi-level residual structure to progressively predict the deformation field, accurate anatomical structure matching and reasonable tissue deformation in 3D medical image registration are achieved.

[0043] Example 2 Figure 5 This is a schematic diagram of the structure of the medical image registration device based on prototype enhancement provided in Embodiment 2 of the present invention, as shown below. Figure 5 As shown, the device includes: The extraction module 210 is used to process the three-dimensional medical moving image and the three-dimensional medical fixed image through a pre-trained encoder to extract the moving image features and fixed image features at each scale. The calculation module 220 is used to input the moving image features and fixed image features of each scale into the APE anatomical prototype enhancement module, obtain the moving projection features and fixed projection features of each scale through the projector operation, and calculate the similarity between the moving projection features and fixed projection features and the corresponding anatomical prototypes under the adjustment of temperature parameters, and obtain the prototype allocation map through softmax processing. The anatomical prototype is a feature template that characterizes the anatomical structure of the medical image. The reconstruction module 230 is used to perform three-dimensional spatial smoothing and anatomical prototype weighted combination on the prototype allocation map to reconstruct the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. The enhancement module 240 is used to stitch together the anatomical prototype to generate a prototype set of the corresponding scale. The moving prototype reconstruction features and fixed prototype reconstruction features of each scale are input into the attention enhancement module along with the prototype set of the corresponding scale to perform prototype attention interaction and context feature enhancement, thereby obtaining enhanced moving features and enhanced fixed features of each scale. The deformation prediction module 250 is used to input the enhanced movement features and enhanced fixed features at each scale into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and displacement corrected to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and displacement corrected and then applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field. The registration module 260 is used to resample the three-dimensional medical moving image using the target deformation field to obtain a registered image.

[0044] The prototype-enhanced medical image registration device provided in this embodiment extracts motion and fixed image features at each scale by processing 3D moving and fixed medical images separately through a pre-trained encoder. These features are then input into the APE anatomical prototype enhancement module. A projector operation yields motion and fixed projection features at each scale. Under temperature parameter adjustment, the motion and fixed projection features are compared with corresponding anatomical prototypes for similarity calculation, and softmax processing is performed to obtain a prototype allocation map. The anatomical prototype serves as a feature template representing the anatomical structure of the medical image. The prototype allocation map is then subjected to 3D spatial smoothing and anatomical prototype weighted combination to reconstruct motion and fixed prototype reconstruction features at each scale. Finally, the anatomical prototypes are stitched together to generate corresponding scales. A prototype set is used to input the reconstructed features of the moving prototypes and the reconstructed features of the fixed prototypes at each scale, along with the prototype set at the corresponding scale, into an attention enhancement module. Prototype attention interaction and contextual feature enhancement are performed to obtain enhanced moving features and enhanced fixed features at each scale. These enhanced moving features and enhanced fixed features are then input into a multi-scale residual deformation field module to estimate the target deformation field. This multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and shifted to serve as the base deformation for the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and shifted, then applied to the fine-scale registration layer for residual deformation estimation and fusion to generate the target deformation field. The target deformation field is then used to resample the 3D medical moving image to obtain the registered image. By introducing anatomical prototypes to complete feature reconstruction and attention interaction enhancement, and combining multi-level residual structures to progressively predict the deformation field, accurate anatomical structure matching and reasonable tissue deformation in 3D medical image registration are achieved.

[0045] Based on the above embodiments, the computing module includes: An initialization unit is used to randomly initialize anatomical prototypes of different scales, wherein the number of prototypes corresponding to the different scales of anatomical prototypes is different; The similarity calculation unit is used to calculate the similarity between the moving projection features and the fixed projection features and the anatomical prototypes of the corresponding scale under the adjustment of temperature parameters. The normalization unit is used to perform softmax processing on the similarity results to obtain the prototype assignment map.

[0046] Based on the above embodiments, the enhancement module includes: The fine-scale prototype set generation unit is used to splice together fine-scale anatomical prototypes, mesoscale anatomical prototypes and coarse-scale anatomical prototypes to generate a fine-scale prototype set. The mesoscale prototype set generation unit is used to splice mesoscale and coarse-scale anatomical prototypes to generate a mesoscale prototype set. A coarse-scale prototype set generation unit is used to generate coarse-scale anatomical prototypes as a coarse-scale prototype set. The attention enhancement module input unit is used to input the mobile prototype reconstruction features and fixed prototype reconstruction features of each scale and the prototype set of the corresponding scale into the attention enhancement module. The query vector generation unit is used to perform convolution mapping on the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale, respectively, to generate mobile query vectors and fixed query vectors at each scale corresponding to the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. The key and value vector generation unit is used to perform linear mapping of keys and values ​​on the prototype set of the corresponding scale, respectively, to generate key vectors and value vectors of the corresponding scale. The attention weight generation unit is used to perform dot product operations on the moving query vector and the fixed query vector with the transpose of the key vector at the corresponding scale, and then divide by the square root of the dimension of a single attention head channel and normalize to obtain the moving attention weight and fixed attention weight at each scale. The context feature generation unit is used to weight and aggregate the mobile attention weights and fixed attention weights at each scale with the value vectors at the corresponding scales to obtain the mobile anatomical context features and fixed anatomical context features at each scale. The enhanced feature generation unit is used to perform convolutional mapping on the mobile anatomical context features and fixed anatomical context features at each scale, and then add them to the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale to obtain enhanced mobile features and enhanced fixed features at each scale.

[0047] Based on the above embodiments, the deformation prediction module includes: The initial residual deformation field generation unit is used to input the coarse-scale enhanced moving features and enhanced fixed features into the coarse-scale registration layer in the multi-scale residual deformation field module, and generate the initial residual deformation field after decoding by the residual deformation decoder. An initial deformation field generation unit is used to perform upsampling and displacement correction operations on the initial residual deformation field to generate the initial deformation field. The mesoscale transformation enhanced motion feature generation unit is used to perform spatial pre-transformation on the mesoscale enhanced motion feature using the initial deformation field to generate the mesoscale transformation enhanced motion feature. The mesoscale residual deformation field generation unit is used to input the mesoscale transformation enhanced moving features and the mesoscale enhanced fixed features into the mesoscale registration layer in the multi-scale residual deformation field module, and generate the mesoscale residual deformation field after decoding by the residual deformation decoder. The mesoscale deformation field generation unit is used to add the mesoscale residual deformation field and the initial deformation field, and generate the mesoscale deformation field by upsampling and displacement correction operations. The fine-scale transformation enhanced motion feature generation unit is used to perform spatial pre-transformation on the fine-scale enhanced motion feature using a mesoscale deformation field to generate the fine-scale transformation enhanced motion feature. The fine-scale residual deformation field generation unit is used to input the fine-scale transformation enhanced moving features and the fine-scale enhanced fixed features into the fine-scale registration layer in the multi-scale residual deformation field module, and generate the fine-scale residual deformation field after decoding by the residual deformation decoder. The target deformation generation unit is used to add the fine-scale residual deformation field and the mesoscale deformation field to obtain the fine-scale deformation field, and then perform an upsampling operation on the fine-scale deformation field to generate the target deformation field.

[0048] The prototype-enhanced medical image registration device provided in this embodiment of the invention can execute the prototype-enhanced medical image registration method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0049] Example 3 Figure 6 This is a schematic diagram of the structure of a server provided in Embodiment 3 of the present invention. Figure 6 A block diagram is shown of an exemplary server 12 suitable for implementing embodiments of the present invention. Figure 6 The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0050] like Figure 6 As shown, server 12 is presented in the form of a general-purpose computing server. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0051] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0052] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.

[0053] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0054] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0055] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing server, display 24, etc.), and with one or more servers that enable users to interact with server 12, and / or with any server (e.g., network card, modem, etc.) that enables server 12 to communicate with one or more other computing servers. This communication can be performed via input / output (I / O) interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, server drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0056] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the prototype-enhanced medical image registration method provided in the embodiments of the present invention.

[0057] Example 4 Embodiment 4 of the present invention also provides a medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the prototype-enhanced medical image registration method as described in any of the above embodiments.

[0058] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0059] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0060] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0061] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0062] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A medical image registration method based on prototype enhancement, characterized in that, include: The three-dimensional medical moving images and three-dimensional medical stationary images are processed by a pre-trained encoder to extract the moving image features and stationary image features at each scale; The moving and fixed image features at each scale are input into the APE anatomical prototype enhancement module. The moving and fixed projection features at each scale are obtained through projector operation. Under the adjustment of temperature parameters, the similarity between the moving and fixed projection features and the corresponding anatomical prototypes is calculated, and the prototype assignment map is obtained through softmax processing. The anatomical prototype is a feature template that characterizes the anatomical structure of the medical image. The prototype allocation map is smoothed in three dimensions and combined with anatomical prototype weighting to reconstruct the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. The anatomical prototypes are spliced ​​together to generate a prototype set at the corresponding scale. The mobile prototype reconstruction features and fixed prototype reconstruction features at each scale are input into the attention enhancement module along with the prototype set at the corresponding scale. Prototype attention interaction and context feature enhancement are performed to obtain enhanced mobile features and enhanced fixed features at each scale. The enhanced moving features and enhanced fixed features at each scale are input into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and displacement-corrected to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and displacement-corrected and applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field. The three-dimensional medical moving image is resampled using the target deformation field to obtain a registered image.

2. The method according to claim 1, characterized in that, The projector operation is used for: The moving image features and fixed image features at each scale are sequentially processed through 3D convolution, normalization, and activation function operations to obtain the moving projection features and fixed projection features at each scale.

3. The method according to claim 1, characterized in that, Under the adjustment of temperature parameters, the similarity between the moving projection features and the fixed projection features and the corresponding anatomical prototypes is calculated, and a prototype assignment map is obtained through softmax processing, including: Anatomical prototypes of different scales are randomly initialized, and the number of prototypes corresponding to the different scales of anatomical prototypes is different; Under the adjustment of temperature parameters, the similarity between the moving projection features and the fixed projection features and the corresponding anatomical prototypes is calculated. The similarity calculation formula is as follows: ； in, For the i-th scale of the voxel position, consider the moving or fixed projection feature. For feature normalization operation, This represents the k-th anatomical prototype at the i-th scale. For temperature parameters, Let be the similarity between the projection feature corresponding to the voxel position v at the i-th scale and the k-th anatomical prototype; The similarity results are processed using softmax to obtain the prototype assignment map.

4. The method according to claim 1, characterized in that, The process of assembling the anatomical prototypes to generate a prototype set at a corresponding scale includes: The fine-scale anatomical prototypes, mesoscale anatomical prototypes, and coarse-scale anatomical prototypes are spliced ​​together to generate a set of fine-scale prototypes. The mesoscale and coarse-scale anatomical prototypes are spliced ​​together to generate a mesoscale prototype set. Coarse-scale anatomical prototypes are used as a coarse-scale prototype set.

5. The method according to claim 1, characterized in that, The process involves inputting the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale, along with the prototype set at the corresponding scale, into the attention enhancement module for prototype attention interaction and context feature enhancement, resulting in enhanced mobile features and enhanced fixed features at each scale. The mobile prototype reconstruction features and fixed prototype reconstruction features at each scale, along with the prototype set at the corresponding scale, are input into the attention enhancement module. The mobile prototype reconstruction features and fixed prototype reconstruction features at each scale are convolved and mapped to generate mobile query vectors and fixed query vectors at each scale corresponding to the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. Linear mapping of keys and values ​​is performed on the prototype sets of the corresponding scales to generate key vectors and value vectors of the corresponding scales; The mobile query vector and the fixed query vector are respectively multiplied by the transpose of the key vector at the corresponding scale, and then divided by the square root of the single attention head channel dimension and normalized to obtain the mobile attention weights and fixed attention weights at each scale. The mobile attention weights and fixed attention weights at each scale are weighted and aggregated with the value vectors of the corresponding scales to obtain the mobile anatomical context features and fixed anatomical context features at each scale. After performing convolution mapping on the mobile anatomical context features and fixed anatomical context features at each scale, they are added to the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale to obtain the enhanced mobile features and enhanced fixed features at each scale.

6. The method according to claim 1, characterized in that, The target deformation field is estimated by inputting the enhanced moving features and enhanced fixed features at each scale into the multi-scale residual deformation field module. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer, including: The enhanced movement features and enhanced stationary features at each scale include coarse-scale enhanced movement features and enhanced stationary features, medium-scale enhanced movement features and enhanced stationary features, and fine-scale enhanced movement features and enhanced stationary features. The coarse-scale enhanced moving features and enhanced fixed features are input into the coarse-scale registration layer in the multi-scale residual deformation field module, and after decoding by the residual deformation decoder, the initial residual deformation field is generated. The initial residual deformation field is upsampled and displacement corrected to generate the initial deformation field; The initial deformation field is used to perform a spatial pre-transformation on the mesoscale enhanced movement feature to generate a mesoscale transformed enhanced movement feature. The mesoscale transformation enhanced moving features and the mesoscale enhanced fixed features are input into the mesoscale registration layer in the multi-scale residual deformation field module, and then decoded by the residual deformation decoder to generate the mesoscale residual deformation field. The mesoscale residual deformation field and the initial deformation field are added together, and then subjected to upsampling and displacement correction operations to generate the mesoscale deformation field. The fine-scale enhanced motion feature is spatially pre-transformed using a mesoscale deformation field to generate a fine-scale transformed enhanced motion feature. The fine-scale transformation enhanced moving features and the fine-scale enhanced fixed features are input into the fine-scale registration layer in the multi-scale residual deformation field module, and then decoded by the residual deformation decoder to generate the fine-scale residual deformation field. The fine-scale residual deformation field and the mesoscale deformation field are added together to obtain the fine-scale deformation field. The fine-scale deformation field is then upsampled and displacement corrected to generate the target deformation field.

7. The method according to claim 6, characterized in that, The residual deformation decoder decodes, including: The splicing operation is used to aggregate dual-path feature information along the channel dimension; 3D convolution operations are used for shallow feature encoding of three-dimensional volume data; Two residual convolution operations are used to extract high-dimensional features and suppress gradient degradation; 3D convolution operations are used to transform feature dimensions and predict residual deformation fields.

8. A medical image registration device based on prototype enhancement, characterized in that, include: The extraction module is used to process 3D medical moving images and 3D medical stationary images through a pre-trained encoder to extract moving image features and stationary image features at various scales. The calculation module is used to input the moving image features and fixed image features at each scale into the APE anatomical prototype enhancement module. The moving projection features and fixed projection features at each scale are obtained through the projector operation. Under the adjustment of the temperature parameter, the moving projection features and fixed projection features are compared with the anatomical prototypes at the corresponding scales. The prototype assignment map is obtained by softmax processing. The anatomical prototype is a feature template that represents the anatomical structure of the medical image. The reconstruction module is used to perform three-dimensional spatial smoothing and anatomical prototype weighted combination on the prototype allocation map to reconstruct the mobile prototype reconstruction features and fixed prototype reconstruction features at each scale. An enhancement module is used to stitch together the anatomical prototypes to generate a prototype set at the corresponding scale. The moving prototype reconstruction features and fixed prototype reconstruction features at each scale are input into the attention enhancement module along with the prototype set at the corresponding scale to perform prototype attention interaction and context feature enhancement, thereby obtaining enhanced moving features and enhanced fixed features at each scale. The deformation prediction module is used to input the enhanced movement features and enhanced fixed features at each scale into the multi-scale residual deformation field module to estimate the target deformation field. The multi-scale residual deformation field module includes a coarse-scale registration layer, a meso-scale registration layer, and a fine-scale registration layer. The initial residual deformation field generated by the coarse-scale registration layer is upsampled and displacement corrected to serve as the base deformation of the meso-scale registration layer. The meso-scale residual deformation field generated by the meso-scale registration layer is upsampled and displacement corrected and then applied to the fine-scale registration layer to perform residual deformation estimation and fusion to generate the target deformation field. The registration module is used to resample the three-dimensional medical moving image using the target deformation field to obtain a registered image.

9. A server, characterized in that, The server includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the prototype-enhanced medical image registration method as described in any one of claims 1-7.

10. A medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the prototype-enhanced medical image registration method as described in any one of claims 1-7.

Images