Medical image registration method based on large model robust features

Abstract

The invention discloses a medical image registration method based on large model robust features, and the method comprises the following steps: constructing a large model registration network which comprises a structure perception feature encoder module and a pyramid deformation field prediction module; acquiring a medical input image, and performing feature extraction on the medical input image through a structure perception feature encoder module to obtain image extraction features; performing deformation and registration on image extraction features based on a pyramid deformation field prediction module to obtain a medical registration image; and training based on the complete loss function pair to optimize the registration model. According to the method, the registration network SAMIR is utilized, universal visual features across anatomical regions can be effectively extracted, the registration accuracy is improved, the anatomical rationality of a deformation field is improved, the migration potential of a natural image pre-training model to a medical registration task is proved in the absence of medical priori knowledge, a feature-level loss function is proposed, and the medical registration efficiency is improved. And the registration consistency is further enhanced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a medical image registration method based on large model robust features. Background Art

[0002] Paired image registration holds significant application value in medical image processing. Its core task is to geometrically align moving and fixed images through spatial transformations to achieve precise matching of multi-source medical data. This technology has a wide range of applications in clinical medicine: in time-series imaging analysis, registering medical images of the same patient at different time points can accurately assess lesion evolution trends or treatment efficacy; in multicenter clinical studies, registration technology can eliminate variations caused by different scanning devices or imaging parameters, enabling standardized data analysis. Therefore, developing efficient and robust medical image registration methods is crucial for advancing precision medicine.

[0003] Traditional registration methods rely on iterative optimization to minimize the distance between a fixed image and a deformed, moving image, such as normalized cross-correlation (NCC) or mean squared error (MSE), which is a very time-consuming process. Deep learning-based registration algorithms significantly accelerate this process, achieving registration through a single forward inference after training. However, these methods often assume consistent lighting conditions and noise-free images, resulting in poor performance in real-world scenarios. Current deep learning methods enhance feature learning in registration networks through various architectures, such as incorporating transformers to capture long-range correlations. Previous studies have shown that incorporating anatomical structure priors, such as segmentation masks or landmarks, into network training can significantly improve registration performance. These methods, also known as weakly supervised methods, typically utilize segmentation masks / landmarks as a loss function during network training or directly provide them as input to the network. However, weak labels such as segmentation masks are not always available, limiting the practical application of such methods.

[0004] The Segment Anything Model (SAM) has recently significantly advanced the field of image segmentation. One of its key advantages is that by training on diverse datasets with extreme lighting, complex occlusions, and boundary uncertainty, the model maintains its performance in the face of variations or noise in the input data, exhibiting strong robustness and achieving modality-independent generalization. This enables SAM to achieve impressive zero-shot generalization performance across various tasks, often matching or exceeding fully supervised results. While SAM has achieved success in many downstream tasks, its application in image registration remains limited.

[0005] Therefore, how to design a more robust and accurate medical image registration method is a technical problem to be solved. SUMMARY

[0006] Therefore, in order to solve the problems in the prior art, the present application proposes a new medical image registration method to achieve robust and more accurate image registration. The present application aims to solve the problems of robust feature learning and large deformation in the network, and combines a hybrid loss to align the moving and fixed images in multiple spaces. The medical image registration method based on large model robust features of the present application comprises the following steps: S1: constructing a large model registration network, wherein the large model registration network comprises a structure perception feature encoder module and a pyramid deformation field prediction module; S2: obtaining a three-dimensional medical input image pair, extracting features of the medical input image through the structure perception feature encoder module to obtain image extraction features; the three-dimensional medical input image pair comprises a moving image and a fixed image; S3: deforming and registering the image extraction features based on the pyramid deformation field prediction module to obtain a highest resolution displacement field; S4: obtaining a medical registration image based on the medical input image and the highest resolution displacement field; S5: evaluating the highest resolution displacement field based on a complete loss function to optimize the large model registration network; the complete loss function comprises an image similarity loss function, a feature similarity loss function, a deformation field smoothness loss function and a segmentation mask similarity loss function.

[0007] Preferably, the structure perception encoder module comprises an image preprocessing module, a SAM feature encoder and a three-dimensional convolution module.

[0008] Preferably, step S2 comprises: S21: preprocessing the moving image and the fixed image of the three-dimensional medical input image pair respectively; S22: slicing the preprocessed three-dimensional medical input image pair along the depth direction to obtain a slice image; S23: sequentially inputting the slice image into the SAM feature encoder to obtain initial medical image features of the slice image; S24: stacking the initial medical image features along the depth direction to obtain an initial three-dimensional image feature pair; S25: upsampling and image splicing the initial three-dimensional image feature pair to obtain an intermediate three-dimensional image feature; S26: performing three-dimensional convolution processing on the intermediate three-dimensional image feature to obtain image extraction features.

[0009] Preferably, the pyramid deformation field prediction module comprises a multi-scale downsampling unit and a multi-scale displacement field combination block, and step S3 comprises: S31: performing multi-scale downsampling on the image extraction features to obtain multiple feature levels, each feature level including a pair of moving image features and fixed image features of the same resolution; S32: Use the Multi-scale Displacement Field Composition (MDFC) block to perform preliminary registration of the lowest resolution feature level and calculate the large-scale deformation field; S33: For feature levels other than the lowest resolution feature level, perform the following operations: interpolate and upsample the deformation field of the previous feature level to a higher resolution, use the result as the initial value of the next feature level, and input it into the MDFC module to obtain a local deformation field; S34: Based on the large-scale deformation field and the local deformation field, a highest-resolution displacement field is obtained.

[0010] Preferably, step S5 includes: S51: After obtaining the medical registration image, calculating a complete loss function; S52: passing the gradient error back to each layer of the large model registration network through the back propagation algorithm and performing parameter update based on the Adam optimizer; S53: When the set number of training rounds is reached, the training stops, and the training weights of the large model registration network are output and saved.

[0011] Preferably, the complete loss function is expressed by the following formula: (1); in, For complete loss, is the image similarity loss, is the feature similarity loss, is the segmentation mask similarity loss, is the smoothness loss of the deformation field, 、 、 and are the hyperparameters used for balancing.

[0012] Preferably, the feature similarity loss It is expressed by the following formula: (2); in, 、 They are the feature pyramid Hierarchical moving image features and fixed image features, is the deformation function, is the highest resolution displacement field, is the number of layers of the pyramid.

[0013] Preferably, the image similarity loss It is expressed by the following formula: (3); in, 、 Respectively The moving and fixed images of the layers are obtained by downsampling the original images in the same multi-scale as when obtaining the feature pyramid. For the Hierarchical moving images The mean gray value of For the Hierarchical fixed images The mean gray value of .

[0014] Preferably, the deformation field smoothness loss It is expressed by the following formula: (4); Preferably, the segmentation mask similarity loss It is expressed by the following formula: (5); in, is the mask of the deformed moving image, is the mask of the fixed image; Displacement field at each feature level It is expressed by the following formula: (6); in, represents 2x trilinear upsampling scaling, is the displacement field of the previous level, is the preliminary displacement field predicted by the flow field estimator at the current level. The specific process is: (1) the flow field estimator constructed with dual 3D convolution blocks predicts the feature matching velocity field u from the moving image to the fixed image; (2) the velocity field u is integrated to obtain .

[0015] Compared with the prior art, the present application has the following beneficial effects: the present application uses a general registration network SAMIR (Segment Anything Model Image Registration), which can effectively extract general visual features across anatomical regions, significantly improving the anatomical rationality of the deformation field and the accuracy of registration. Especially in the absence of medical prior knowledge, the natural image pre-training model proves the migration potential to the medical registration task, and a feature-level loss function HFC loss is proposed to further enhance the registration consistency. In addition, using a SAM-based robust feature encoder, the model converges quickly and can complete the training process in a short time, which is conducive to the use in practical situations. BRIEF DESCRIPTION OF DRAWINGS

[0016] Exemplary embodiments of the present application can be more fully understood with reference to the following description when taken in connection with the following drawings, in which like reference numerals refer to like elements throughout. The drawings are included to provide a further understanding of the embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and serve to explain the present application, and do not limit the present application. In the drawings, like reference numerals refer to the same or similar elements throughout.

[0017] Figure 1 A flowchart of a medical image registration method based on a large model robust feature provided by an embodiment of the present application; Figure 2 A structure diagram of a large model registration network provided by an embodiment of the present application; Figure 3 An effect diagram of a medical image registration method based on a large model robust feature provided by an embodiment of the present application for abdominal image registration. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0019] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0020] In addition, the technical features involved in the different embodiments of the application described below can be combined with each other as long as they do not conflict with each other.

[0021] With reference to Figure 1 The embodiment discloses a medical image registration method based on large model robust features, comprising the following steps: S1: constructing a large model registration network, the large model registration network comprising a structure perception feature encoder module and a pyramid deformation field prediction module; S2: obtaining a three-dimensional medical input image pair, extracting features of the image by the structure perception feature encoder module, and obtaining image extraction features; the three-dimensional medical input image pair comprises a moving image and a fixed image; Specifically, the structure perception encoder module is a SAM-based image encoder, which is specially designed for the characteristics of three-dimensional medical images and contains an image preprocessing module, a SAM feature encoder and a three-dimensional convolution module. Among them, the image preprocessing module is used to enhance the image resolution.

[0022] The encoder part in the SAM network framework is used in this step, and the provided pre-training weight is loaded. Specifically, step S2 comprises: S21: preprocessing the moving image and the fixed image of the three-dimensional medical input image pair respectively; S22: slicing the preprocessed three-dimensional medical input image pair along the depth direction to obtain a slice image; S23: sequentially inputting the slice image into the SAM feature encoder to obtain the initial medical image features of the slice image; S24: stacking the initial medical image features along the depth direction to obtain an initial three-dimensional image feature pair; S25: upsampling and image splicing the initial three-dimensional image feature pair to obtain an intermediate three-dimensional image feature; S26: performing three-dimensional convolution processing on the intermediate three-dimensional image feature to obtain the image extraction features.

[0023] In the preprocessing stage using the structure-aware encoder, the input medical image is first upsampled and then fed into the SAM pre-trained encoder with frozen parameters (i.e., parameters are not updated) for feature extraction. The size of the input image is denoted as , where D represents the number of slices, and each slice is Resolution of 2D image. Since the output feature map size of the SAM encoder is 1 / 16 of the input image, directly using the original resolution will result in the feature map being too small, affecting the accuracy of subsequent deformation field prediction. Therefore, we perform upsampling preprocessing on the input image, specifically using bilinear interpolation to enlarge each 2D slice to , and then input it into the SAM pre-trained encoder to obtain image features. At this time, the image feature size In order to make the feature map size consistent with the original input, the image is scaled using trilinear interpolation. In order to integrate the two-dimensional features into a three-dimensional representation, we stack all the slice features along the depth direction to construct a three-dimensional feature volume. This step retains the complete spatial information and avoids the registration error caused by the low resolution of the feature map. , the fixed image is The features obtained are , , the calculation process of this step can be expressed as: (5); (6); in, Indicates that the encoder is pre-trained by SAM, Indicates 2x bilinear upsampling scaling, Indicates 2x trilinear upsampling scaling.

[0024] However, because the SAM encoder processes each 2D slice independently, the features it extracts lack continuity information across slices. To address this, we designed a lightweight 3D convolutional module to enhance feature correlation in the depth dimension. The input to this module consists of the concatenation of the moving image and the fixed image (single channel) with their respective SAM-extracted features (256 channels) along the channel dimension, resulting in feature data of size 257 × H × W × D. This data is then separated using the chunk function. The computational process for this step can be expressed as: (7); In the entire feature extraction architecture, moving images and fixed images are processed in the same way, thereby obtaining the final image features of moving images and fixed images.

[0025] S3: deforming and registering the image extraction features based on the pyramid deformation field prediction module to obtain the highest resolution displacement field; Specifically, step S3 includes: S31: performing multi-scale downsampling on the image extraction features to obtain multiple feature levels, each feature level including a pair of moving image features and fixed image features of the same resolution; Multi-scale downsampling of moving image and fixed image features is performed separately to generate five feature levels from coarse to fine (from low resolution to high resolution). Each level contains a pair of moving image and fixed image features of the same resolution. The bottom level is the original resolution, and the resolution of each upper level is reduced to 1 / 2 of the original image. S32: Use the multi-scale displacement field combination block to perform preliminary registration of the lowest resolution feature level and calculate the large-scale deformation field; At the lowest resolution layer (i.e., the top layer), a preliminary registration is performed using the Multi-scale Displacement Field Composition (MDFC) block to calculate large-scale deformations. S33: For feature levels other than the lowest resolution feature level, perform the following operations: upsample the deformation field of the previous feature level to a higher resolution, use it as the initial value of the next feature level, and input it into the MDFC module to gradually optimize finer local deformations; S34: Based on the large-scale deformation field and the local deformation field, a highest-resolution displacement field is obtained.

[0026] In order to handle small and large deformations, pyramid feature-based coarse-to-fine registration is an effective solution. First, the image features obtained in S2 are downsampled using trilinear interpolation to generate multi-scale feature maps. In our experiment, we performed four downsamplings to obtain a five-layer feature pyramid with a resolution of half the size. Each layer of the pyramid is a pair of moving image features and fixed image features with the same resolution. According to the change in resolution from large to small, each layer of the pyramid is called the first to fifth levels, and the moving and fixed images are the first and fifth levels respectively. The hierarchical features are represented as and Subsequently, the displacement field is gradually optimized by processing from the fifth level to the first level.

[0027] At each level, a multi-scale displacement field composition (MDFC) block is used to predict the deformation field. Specifically, the input of the MDFC block is the moving image and fixed image features of the current level resolution (all levels), as well as the displacement field estimate of the previous level (except the fifth level).

[0028] In the MDFC block, for the fifth level, the velocity field u of the moving image features registered to the fixed image features is predicted by a flow field estimator composed of two 3D convolution blocks, and then the displacement field is obtained by integration. .

[0029] For the fourth to first level, the displacement field of the previous level is Upsample to the same resolution as the current layer and use it to deform the moving image features of the current layer , and then combined with the fixed image features of the current level Perform the same operation as the fifth level to obtain , and finally use right Perform progressive refinement to obtain the final displacement field at this level , the progressive refinement formula is: (3); in, is the preliminary displacement field of the current level, is the displacement field of the previous level of upsampling: (4); in, represents 2x trilinear upsampling and scaling, represents the deformation function. This yields the highest-resolution displacement field for registering the moving image to the fixed image. Using this displacement field, the moving image is deformed to obtain the registered moving image. This design captures large global deformations at a coarse level (low resolution) and optimizes local details at a fine level (high resolution), achieving accurate global registration. The number of pyramid levels n directly affects registration accuracy, with larger deformations requiring more pyramid levels.

[0030] In order to make those skilled in the art better understand the technical solution of the present invention, Figure 2 The present invention is further described in detail. The method includes two steps: (1) inputting the moving image and the fixed image into a structure-aware feature encoder to obtain the moving image features and the fixed image features; and (2) inputting the moving image features and the fixed image features into a pyramid-shaped deformation field prediction module to achieve the generation and progressive optimization of the deformation field.

[0031] Specifically, if Figure 2As shown in (a), in the structure-aware feature encoder, a 3D moving image and a fixed image are input, each of which undergoes upsampling preprocessing to increase image resolution. These images are then fed into the SAM image encoder (consisting of Patch Embedding and multiple Transformer blocks) to generate preliminary image feature pairs. Because the image features obtained by the SAM encoder are reduced to 1 / 16 of the image size before entering the encoder, these preliminary 3D medical image feature pairs are then upsampled to restore them to their original size, resulting in 3D image feature pairs of the same size as the original input images. These 3D image features are then concatenated with their corresponding input images to compensate for information lost during feature extraction. The concatenation is then fed into a 3D convolutional module (the Conv module in the figure), consisting of three layers of 3D convolutions, to supplement the depthwise image features. These features are then split using the chunk() function (a PyTorch function for splitting tensors into multiple sub-tensors along a specified dimension) to generate the final moving and fixed image features.

[0032] Specifically, if Figure 2 As shown in (b), in the pyramidal deformation field prediction module, each column represents a feature level. Each level contains a pair of features from the moving and fixed images at the same resolution, with the first column representing the original resolution features. First, the features output by the structure-aware feature encoder (i.e., the original resolution features) are downsampled by a factor of 2 to obtain features from the second through fifth levels, halving the image size. Progressive registration is then performed, starting with the fifth level (i.e., the lowest resolution level). Within each level, a multi-scale displacement field composition (MDFC) block is used to predict the deformation field.

[0033] Specifically, the input of the MDFC block is the moving image and fixed image features of the current level resolution (all levels), and the displacement field estimation of the previous level (except the fifth level). In the MDFC block, for the fifth level, there is no , so firstly the velocity field u of the moving image feature is predicted by the flow field estimator to be registered to the fixed image feature, and then the displacement field is obtained by integration .

[0034] For the fourth to first level, the displacement field of the previous level is Upsample to the same resolution as the current layer, i.e. Then use it to deform the moving image features of the current level to obtain , and then combined with the fixed image features of the current level Smooth estimation is performed , and finally use right Transform, and add , and get the final displacement field of this level .

[0035] Thus, by optimizing from the fifth level to the first level, a progressively better registration is achieved. This design captures large global deformations at the coarse level (low resolution) and optimizes local details at the fine level (high resolution), achieving accurate global registration. The number of pyramid levels n directly affects registration accuracy, with tasks with larger deformations requiring more pyramid levels.

[0036] S4: Obtain a medical registration image based on the medical input image and the highest resolution displacement field.

[0037] S5: Evaluate the results of the highest resolution displacement field based on the complete loss function to optimize the large model registration model; the complete loss function includes image similarity loss function, feature similarity loss function, deformation field smoothness loss function and segmentation mask similarity loss function.

[0038] Specifically, step S5 includes: S51: After obtaining the medical registration image, calculate the complete loss function; S52: passing the gradient error back to each layer of the large model registration network through the back propagation algorithm and performing parameter update based on the Adam optimizer; S53: When the set number of training rounds is reached, the training stops, and the training weights of the large model registration network are output and saved.

[0039] Specifically, a hybrid loss function consisting of image similarity loss, feature similarity loss, segmentation mask similarity loss, and deformation field smoothness loss is used to jointly optimize the model. The loss function includes two aspects: dissimilarity loss and regularization loss. The former measures the distance between the deformed moving image and the fixed image at each pyramid level, while the latter ensures a smooth deformation field to avoid unrealistic deformations. To improve consistency, losses are calculated for all pyramid levels, where the moving image is downsampled and deformed by the deformation field of each level to measure the difference from the downsampled fixed image. In SAMIR, the dissimilarity loss utilizes image similarity and feature similarity loss functions to simultaneously capture image-level and feature-level dissimilarity.

[0040] For the weakly supervised version, segmentation masks are used in network training and Dice loss is also used. Calculate regional differences. Dice loss corresponds to the aforementioned segmentation mask similarity loss. The calculated regions are different for each dataset and are the most important organs and tissues in each type of image. For example, for cardiac MRI images, they are the left ventricle, right ventricle, left atrium, and right atrium; for abdominal images, they are the various organs in the abdominal cavity, liver, kidneys, etc. Figure Three Several areas of the stroke in the middle. Using the diffusion regularization term To constrain the spatial smoothness of the deformation field. The complete loss function for training the network is expressed as follows: (1); in, For complete loss, is the image similarity loss, is the feature similarity loss, is the segmentation mask similarity loss, is the smoothness loss of the deformation field, 、 、 and are the hyperparameters used for balancing.

[0041] At the image level, we use normalized cross correlation (NCC) to evaluate the similarity between the deformed moving image and the fixed image. At the feature level, we construct a hierarchical feature consistency loss , to further enhance the registration consistency. The moving image and the fixed image in the feature pyramid The hierarchical features are represented as and At each level In the middle, the layer-wise feature consistency loss is calculated as follows: (2); in, 、 Respectively Moving image features and fixed image features of feature pyramid, is the deformation function, is the highest resolution displacement field, is the number of layers of the pyramid.

[0042] Among them, image similarity loss It is expressed by the following formula: (3); in, 、 Respectively The moving and fixed images of the layers are obtained by downsampling the original images in the same multi-scale as when obtaining the feature pyramid. For the Layered moving images The mean gray value of For the Hierarchical fixed images The mean gray value of .

[0043] Among them, the segmentation mask similarity loss It is expressed by the following formula: (4); in, is the mask of the deformed moving image, is the mask of the fixed image. This loss is used in the weakly supervised case, that is, the masks of the moving image and the fixed image are provided in the training data for auxiliary training.

[0044] Among them, the deformation field smoothness loss It is expressed by the following formula: (5); In each training round, the loss function is calculated, the gradient error is propagated back to each layer of the network using the backpropagation algorithm, and the parameters are updated using the Adam optimizer. When the set number of training rounds is reached, the training weights of the large model registration network are output and saved to optimize the next image registration result.

[0045] Specifically, the Adam (Adaptive Moment Estimation) optimizer adaptively updates all learnable parameters of a neural network during training. Specifically, it optimizes two types of parameters: weights and biases, as well as other learnable parameters (such as normalization layers and embedding layers).

[0046] This application utilizes a universal registration network, SAMIR, which effectively extracts universal visual features across anatomical regions (such as organ topology and boundary continuity). This network significantly improves the anatomical plausibility of deformation fields, demonstrating the potential for transferring pre-trained models from natural images to medical registration tasks, particularly in the absence of prior medical knowledge. Furthermore, the use of a robust SAMIR-based feature encoder allows for rapid model convergence, enabling rapid training and facilitating practical application.

[0047] It should be noted that the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0048] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0049] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0050] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0051] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0052] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and description of the present application.

Claims

1. A medical image registration method based on large model robust features, characterized in that: The steps include: S1: Constructing a large model registration network, wherein the large model registration network includes a structure-aware feature encoder module and a pyramidal variation field prediction module; S2: Obtain a three-dimensional medical input image pair, and perform feature extraction on the medical input image using the structure-aware feature encoder module to obtain image extraction features; the three-dimensional medical input image pair includes a moving image and a fixed image; S3: deforming and registering the image extraction features based on the pyramid deformation field prediction module to obtain the highest resolution displacement field; S4: obtaining a medical registration image based on the medical input image and the highest resolution displacement field; S5: Evaluate the highest resolution displacement field based on a complete loss function to optimize the large model registration network; the complete loss function includes an image similarity loss function, a feature similarity loss function, a deformation field smoothness loss function, and a segmentation mask similarity loss function.

2. The method according to claim 1, characterized in that The structure-aware encoder module includes an image preprocessing module, a SAM feature encoder, and a 3D convolution module. Step S2 includes: S21: Preprocessing the moving image and the fixed image of the three-dimensional medical input image pair respectively; S22: Slicing the preprocessed three-dimensional medical input image along the depth direction to obtain a slice image; S23: inputting the slice images into a SAM feature encoder in sequence to obtain initial medical image features of the slice images; S24: stacking the initial medical image features along the depth direction to obtain an initial three-dimensional image feature pair; S25: performing upsampling and image stitching on the initial three-dimensional image feature pair to obtain an intermediate three-dimensional image feature; S26: Performing three-dimensional convolution processing on the intermediate three-dimensional image features to obtain image extraction features.

3. The method according to claim 2, characterized in that The pyramidal displacement field prediction module includes a multi-scale downsampling unit and a multi-scale displacement field combination block. Step S3 includes: S31: performing multi-scale downsampling on the image extraction features to obtain multiple feature levels, each feature level including a pair of moving image features and fixed image features of the same resolution; S32: Use the scale displacement field combination block to perform preliminary registration of the lowest resolution feature level and calculate the large-scale deformation field; S33: For feature levels other than the lowest resolution feature level, perform the following operations: interpolate and upsample the deformation field of the previous feature level to a higher resolution, use the result as the initial value of the next feature level, and input it into the MDFC module to obtain a local deformation field; S34: Based on the large-scale deformation field and the local deformation field, a highest-resolution displacement field is obtained.

4. The method according to claim 3, characterized in that Step S5 includes: S51: After obtaining the medical registration image, calculating a complete loss function; S52: passing the gradient error back to each layer of the large model registration network through the back propagation algorithm and performing parameter update based on the Adam optimizer; S53: When the set number of training rounds is reached, the training stops, and the training weights of the large model registration network are output and saved.

5. The method according to claim 4, characterized in that The complete loss function is expressed by the following formula: （1）； in, For complete loss, is the image similarity loss, is the feature similarity loss, is the segmentation mask similarity loss, is the smoothness loss of the deformation field, 、 、 and are the hyperparameters used for balancing.

6. The method according to claim 5, characterized in that Feature similarity loss It is expressed by the following formula: （2）； in, 、 They are the feature pyramid Hierarchical moving image features and fixed image features, is the deformation function, is the highest resolution displacement field, is the number of layers of the pyramid.

7. The method according to claim 6, characterized in that Image similarity loss It is expressed by the following formula: （3）； in, 、 Respectively The moving and fixed images of the layers are obtained by downsampling the original images in the same multi-scale as when obtaining the feature pyramid. For the Hierarchical moving images The mean gray value of For the Hierarchical fixed images The mean gray value of .

8. The method according to claim 7, characterized in that Deformation field smoothness loss It is expressed by the following formula: （4）。 9. The method according to claim 8, characterized in that Segmentation mask similarity loss It is expressed by the following formula: （5）； in, is the mask of the deformed moving image, is the mask of the fixed image; Displacement field at each feature level It is expressed by the following formula: （6）； in, represents 2x trilinear upsampling scaling, is the displacement field of the previous level, is the preliminary displacement field predicted by the flow field estimator at the current level. The specific process is: (1) the flow field estimator constructed with dual 3D convolution blocks predicts the feature matching velocity field u from the moving image to the fixed image; (2) the velocity field u is integrated to obtain .

Images