A medical image registration method based on large model robust features

Abstract

The application discloses a medical image registration method based on large model robust features, comprising the following steps: constructing a large model registration network, the large model registration network comprising a structure perception feature encoder module and a pyramid deformation field prediction module; acquiring a medical input image, extracting features of the medical input image through the structure perception feature encoder module to obtain image extraction features; deforming and registering the image extraction features based on the pyramid deformation field prediction module to obtain a medical registration image; and training based on a complete loss function to optimize the registration model. The application uses the registration network SAMIR to effectively extract general visual features across anatomical regions, improves the accuracy of registration, improves the anatomical rationality of the deformation field, proves the migration potential of the natural image pre-training model to the medical registration task in the absence of medical prior knowledge, and proposes a feature-level loss function to further enhance registration consistency.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision, and particularly relates to a medical image registration method based on robust features of a large model. BACKGROUND

[0002] Pairwise image registration has important application value in the field of medical image processing. Its core task is to geometrically align the moving image with the fixed image through spatial transformation, so as to realize accurate matching of multi-source medical data. This technology has a wide range of application scenarios in clinical medicine: in time series image analysis, by registering medical images of the same patient at different time points, the evolution trend or treatment effect of the lesion can be accurately evaluated; in multi-center clinical research, the registration technology can eliminate the differences caused by different scanning devices or imaging parameters, and realize standardized data analysis. Therefore, it is of great significance to study efficient and robust medical image registration methods for promoting the development of precision medicine.

[0003] Traditional registration methods rely on iterative optimization to minimize the distance between the fixed image and the deformed moving image, such as normalized cross correlation (NCC) or mean square error (MSE), which is very time-consuming. Deep learning-based registration algorithms significantly speed up this process, achieving registration through a single forward inference after training. However, this method usually assumes consistent lighting conditions and noise-free images, resulting in poor performance in real-world scenarios. Current deep learning methods enhance feature learning in the registration network through various architectures, such as adding transformers to capture long-range correlations. Previous studies have shown that incorporating anatomical structure priors, such as segmentation masks or landmarks, into network training can significantly improve registration performance. These methods are also known as weakly supervised methods, which usually use segmentation masks / landmarks as loss functions during network training, or directly use them as network inputs. However, weak labels such as segmentation masks are not always available, limiting the practical application of such methods.

[0004] Recently, the Segment Anything Model (SAM) has significantly advanced the field of image segmentation. One of its key advantages is that by training on different datasets with extreme lighting, complex occlusions, and uncertain boundaries, the model can still maintain performance when faced with changes or noise in input data, demonstrating strong robustness and modality-agnostic generalization ability. This enables SAM to achieve impressive zero-shot generalization performance in various tasks, often reaching or exceeding fully supervised results. Although SAM has been successful in many downstream tasks, its application in image registration is still limited.

[0005] Therefore, how to design a more robust and accurate medical image registration method is a technical problem to be solved. SUMMARY

[0006] Therefore, in order to solve the problems in the prior art, the present application proposes a new medical image registration method to achieve robust and more accurate image registration. The present application aims to solve the problems of robust feature learning and large deformation in the network, and combines a hybrid loss to align the moving and fixed images in multiple spaces. The medical image registration method based on large model robust features of the present application comprises the following steps:

[0007] S1: constructing a large model registration network, wherein the large model registration network comprises a structure perception feature encoder module and a pyramid deformation field prediction module;

[0008] S2: obtaining a pair of three-dimensional medical input images, extracting features of the medical input images by using the structure perception feature encoder module to obtain image extraction features; the pair of three-dimensional medical input images comprises a moving image and a fixed image;

[0009] S3: deforming and registering the image extraction features based on the pyramid deformation field prediction module to obtain a highest resolution displacement field;

[0010] S4: obtaining a medical registration image based on the medical input images and the highest resolution displacement field;

[0011] S5: evaluating the results of the highest resolution displacement field based on a complete loss function to optimize the large model registration network; the complete loss function comprises an image similarity loss function, a feature similarity loss function, a deformation field smoothness loss function, and a segmentation mask similarity loss function.

[0012] Preferably, the structure perception encoder module comprises an image preprocessing module, a SAM feature encoder, and a three-dimensional convolution module.

[0013] Preferably, step S2 comprises:

[0014] S21: preprocessing the moving image and the fixed image of the pair of three-dimensional medical input images respectively;

[0015] S22: slicing the pair of preprocessed three-dimensional medical input images along the depth direction to obtain slice images;

[0016] S23: inputting the slice images into the SAM feature encoder in sequence to obtain initial medical image features of the slice images;

[0017] S24: stacking the initial medical image features along the depth direction to obtain a pair of initial three-dimensional image features;

[0018] S25: upsampling and image stitching the initial three-dimensional image feature pair to obtain an intermediate three-dimensional image feature;

[0019] S26: performing three-dimensional convolution processing on the intermediate three-dimensional image feature to obtain image extraction features.

[0020] Preferably, the pyramid deformation field prediction module comprises a multi-scale down-sampling unit and a multi-scale displacement field composition block, and step S3 comprises:

[0021] S31: performing multi-scale down-sampling on the image extraction features to obtain a plurality of feature levels, each feature level comprising a pair of moving image features and fixed image features of the same resolution;

[0022] S32: performing preliminary registration on the feature level of the lowest resolution by using a multi-scale displacement field composition block (MDFC) to calculate a large-scale deformation field;

[0023] S33: for the feature levels other than the feature level of the lowest resolution, performing the following operations: interpolating and up-sampling the deformation field of the previous feature level to a high-level resolution as an initial value of the next feature level and inputting the initial value into the MDFC module to obtain a local deformation field;

[0024] S34: obtaining a highest-resolution displacement field based on the large-scale deformation field and the local deformation field.

[0025] Preferably, step S5 comprises:

[0026] S51: after obtaining the medical registration image, calculating a complete loss function;

[0027] S52: transmitting gradient errors back to each layer of the large model registration network by using a back propagation algorithm and updating parameters based on an Adam optimizer;

[0028] S53: when a set number of training rounds is reached, stopping training, outputting and saving the training weight of the large model registration network.

[0029] Preferably, the complete loss function is represented by the following formula:

[0030] (1) ;

[0031] wherein, is a complete loss, is an image similarity loss, is a feature similarity loss, is a segmentation mask similarity loss, is a deformation field smoothness loss, , , and are hyperparameters for balancing.

[0032] Preferably, the feature similarity loss is represented by the following equation:

[0033] (2).

[0034] wherein, , are the moving image features and the fixed image features of the th level of the feature pyramid, is the warping function, is the highest resolution displacement field, is the number of levels of the pyramid.

[0035] Preferably, the image similarity loss is represented by the following equation:

[0036] (3).

[0037] wherein, , are the moving image and the fixed image of the th level, obtained from the original image by the same multi-scale down-sampling as when obtaining the feature pyramid, is the mean of the gray values of the moving image of the th level, is the mean of the gray values of the fixed image of the th level.

[0038] Preferably, the warping field smoothness loss is represented by the following equation:

[0039] (4).

[0040] Preferably, the segmentation mask similarity loss is represented by the following equation:

[0041] (5).

[0042] wherein, is the mask of the deformed moving image, is the mask of the fixed image;

[0043] The displacement field of each feature level is represented by the following equation:​​

[0044] (6);

[0045] wherein, denotes a 2x trilinear upscaling, is the displacement field of the previous level, is the preliminary displacement field of the current level predicted by the flow field estimator, the specific process is (1) using a flow field estimator constructed by a double 3D convolution block to predict the feature matching velocity field u of the moving image to the fixed image; (2) performing integral operation on the velocity field u to obtain .

[0046] Compared with the prior art, the present application has the following beneficial effects: the present application uses a general registration network SAMIR (Segment Anything Model Image Registration), which can effectively extract general visual features across anatomical regions, significantly improving the anatomical rationality of the deformation field and the accuracy of registration. Especially in the absence of medical prior knowledge, the migration potential of the natural image pre-training model to the medical registration task is proved, and a feature-level loss function HFC loss is proposed to further enhance the registration consistency. In addition, using a robust feature encoder based on SAM, the convergence speed of the model is fast, and the training process can be completed in a short time, which is conducive to the use in practical occasions. BRIEF DESCRIPTION OF DRAWINGS

[0047] The exemplary embodiments of the present application can be more fully understood by reference to the following drawings. The drawings are provided to provide a further understanding of the embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and serve to explain the present application, and do not constitute a limitation of the present application. In the drawings, the same reference numerals refer to the same components or steps throughout the several views.

[0048] Figure 1 A flowchart of a medical image registration method based on robust features of large models provided by the embodiments of the present application;

[0049] Figure 2 A structure diagram of a large model registration network provided by the embodiments of the present application;

[0050] Figure 3 An effect diagram of a medical image registration method based on robust features of large models provided by the embodiments of the present application for abdominal image registration. DETAILED DESCRIPTION

[0051] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms without being limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be thoroughly and completely understood, and so that the scope of the present disclosure can be conveyed completely to those skilled in the art.

[0052] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0053] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict between them.

[0054] With reference to Figure 1 The embodiment discloses a medical image registration method based on large model robust features, comprising the following steps:

[0055] S1: constructing a large model registration network, the large model registration network comprising a structure perception feature encoder module and a pyramid deformation field prediction module;

[0056] S2: obtaining a pair of three-dimensional medical input images, extracting features of the medical input images through the structure perception feature encoder module to obtain image extraction features; the pair of three-dimensional medical input images comprises a moving image and a fixed image;

[0057] Specifically, the structure perception encoder module is a SAM-based image encoder, which is specially optimized and designed for the characteristics of three-dimensional medical images, and comprises an image preprocessing module, a SAM feature encoder, and a three-dimensional convolution module. Among them, the image preprocessing module is used to enhance the image resolution.

[0058] The encoder part in the SAM network framework is used in this step, and the provided pre-training weight is loaded. Specifically, step S2 comprises:

[0059] S21: preprocessing the moving image and the fixed image of the pair of three-dimensional medical input images respectively;

[0060] S22: slicing the pair of three-dimensional medical input images after preprocessing along the depth direction to obtain slice images;

[0061] S23: sequentially inputting the slice images into a SAM feature encoder to obtain initial medical image features of the slice images;

[0062] S24: stacking the initial medical image features along a depth direction to obtain initial three-dimensional image feature pairs;

[0063] S25: performing upsampling and image splicing on the initial three-dimensional image feature pairs to obtain intermediate three-dimensional image features;

[0064] S26: performing three-dimensional convolution processing on the intermediate three-dimensional image features to obtain image extraction features.

[0065] In the preprocessing stage using the structure-aware encoder, first, the input medical image is upsampled, and then fed into the SAM pre-trained encoder with frozen parameters (i.e., the parameters are not updated) for feature extraction. The size of the input image is denoted as , where D represents the number of slices, and each slice is a two-dimensional image with a resolution of . Since the output feature map size of the SAM encoder is 1 / 16 of the input image, directly using the original resolution will result in a too small feature map, affecting the accuracy of subsequent deformation field prediction. Therefore, we perform upsample preprocessing on the input image, specifically using the bilinear interpolation method to enlarge each two-dimensional slice to , and then input it into the SAM pre-trained encoder to obtain image features, at this time the image feature size is . To make the feature map size consistent with the original input in proportion, we use trilinear interpolation to scale the image. To integrate the two-dimensional features into a three-dimensional representation, we stack all the slice features along the depth direction to construct a three-dimensional feature volume . This step preserves the complete spatial information and avoids registration errors caused by low feature map resolution. Denote the moving image as , the fixed image as , and the obtained features as , , the calculation process of this step can be represented as:

[0066] (5) ;

[0067] (6) ;

[0068] where denotes the SAM pre-trained encoder, denotes 2x bilinear upscaling, denotes 2x trilinear upscaling.

[0069] However, since the SAM encoder processes each two-dimensional slice independently, the extracted features lack inter-slice continuity information. To this end, we design a lightweight three-dimensional convolution module to enhance the correlation of features in the depth dimension. The input of this module is formed by concatenating the moving image and the fixed image (single channel) with the respective SAM-extracted features (256 channels) along the channel dimension, resulting in a feature data concatenation of size 257xHxWxD. Finally, the chunk function is applied to separate them. The calculation process of this step can be represented as:

[0070] (7);

[0071] In the entire feature extraction architecture, the moving image and the fixed image are processed in the same way. Thus, the final image features of the moving image and the fixed image are obtained.

[0072] S3: deforming and registering the image extraction features based on the pyramid-shaped deformation field prediction module to obtain a highest resolution displacement field;

[0073] Specifically, step S3 includes:

[0074] S31: performing multi-scale down-sampling on the image extraction features to obtain a plurality of feature levels, each feature level containing a pair of moving image features and fixed image features of the same resolution;

[0075] The moving image and fixed image features are respectively down-sampled at multiple scales to generate 5 feature levels from coarse to fine (from low resolution to high resolution), wherein each level contains a pair of moving image and fixed image features of the same resolution, and the bottom layer is the original resolution, and the resolution of each upper layer is reduced to 1 / 2 of the original image;

[0076] S32: using a multi-scale displacement field composition block to preliminarily register the lowest resolution feature level to calculate a large-scale deformation field;

[0077] In the lowest resolution layer (i.e., the top layer), a multi-scale displacement field composition block (MDFC) is used for preliminary registration to calculate a large-scale deformation;

[0078] S33: for other feature levels except the lowest resolution feature level, performing the following operations: interpolating and up-sampling the deformation field of the previous feature level to a high layer resolution as the initial value of the next feature level and inputting it into the MDFC module to gradually optimize finer local deformation;

[0079] S34: based on the large-scale deformation field and the local deformation field, obtaining a highest resolution displacement field.

[0080] To handle small and large deformations, a coarse-to-fine registration based on pyramid features is an effective solution. First, the image features obtained in S2 are down-sampled using trilinear interpolation respectively to generate multi-scale feature maps. In our experiments, we down-sample four times to obtain five layers of feature pyramids, with the resolution of each layer being halved successively. Each layer of the pyramid is a pair of moving image features and fixed image features with the same resolution. According to the change in resolution from large to small, the pyramid is divided into the first to fifth levels, and the moving and fixed image features of each level are denoted as and respectively. Then, from the fifth level to the first level, the displacement field is optimized step by step.

[0081] In each level, a Multi-scale Displacement Field Composition (MDFC) block is used to predict the deformation field. Specifically, the input of the MDFC block is the moving image and fixed image features at the current level resolution (all levels), and the displacement field estimate of the previous level (except for the fifth level).

[0082] In the MDFC block, for the fifth level, the velocity field u that aligns the moving image features to the fixed image features is predicted by a flow field estimator composed of two 3D convolution blocks, and then the displacement field is obtained by integration .

[0083] For the fourth to first levels, the displacement field of the previous level is up-sampled to the same resolution as the current level, and is used to deform the moving image features of the current level to obtain , which is then operated with the fixed image features of the current level in the same way as the fifth level to obtain . Finally, the progressive refinement of is used to obtain the final displacement field of the level , and the progressive refinement formula is:

[0084] (3);

[0085] where is the preliminary displacement field of the current level, is the up-sampled displacement field of the previous level:

[0086] (4);

[0087] where denotes 2x trilinear up-sampling and scaling, ​​denotes the deformation function. Thus, the highest resolution displacement field of moving image registration to fixed image is obtained, and the moving image is deformed using the displacement field to obtain the registered moving image. This design captures global large deformation at the coarse level (low resolution) and optimizes local details at the fine level (high resolution), achieving global accurate registration. The number of pyramid layers n directly affects the registration accuracy, and the task with greater deformation requires more pyramid layers.

[0088] In order to enable those skilled in the art to better understand the technical solutions of the present application, the technical solutions of the present application will be further described in detail below with reference to the accompanying drawings. Figure 2 The present application is further described in detail. The method includes two steps: (1) inputting the moving image and the fixed image into the structure-aware feature encoder to obtain the moving image features and the fixed image features; (2) inputting the moving image features and the fixed image features into the pyramid type deformation field prediction module to realize the generation and progressive optimization of the deformation field.

[0089] Specifically, as shown in Figure 2 (a), in the structure-aware feature encoder: input the three-dimensional moving image and the fixed image, and then perform upsampling preprocessing to improve the image resolution. Then input into the SAM image encoder (including patch embedding and multiple transformer blocks), to obtain the preliminary image feature pair. Since the image feature size obtained by the SAM encoder will be reduced to 1 / 16 of the image size before inputting the encoder, the obtained preliminary three-dimensional medical image features are then upsampled to restore the original image size, to obtain a three-dimensional image feature pair with the same size as the initial input image. Then the three-dimensional image features are respectively spliced with their corresponding input images to supplement the information lost in the feature extraction process, and then the two are spliced and input into the three-dimensional convolution module (Conv module in the figure) composed of three 3D convolutions to supplement the image features in the depth direction, and use the chunk() function (a function in PyTorch for splitting tensors, mainly used to split tensors into multiple sub-tensors according to the specified dimension) to split them to obtain the final moving image and fixed image features.

[0090] Specifically, as shown in Figure 2(b) As shown, in the pyramid deformation field prediction module: take each column as a hierarchical feature, each hierarchy contains a pair of moving image and fixed image features with the same resolution, and the first column is the original resolution feature. First, the features output by the structure-aware feature encoder (i.e., the original resolution features) are down-sampled by 2 to obtain the second to fifth level features, and the picture size is halved in turn. Then, starting from the fifth level (i.e., the lowest resolution level), progressive registration is performed. In each level, a multi-scale displacement field composition block (MDFC) is used to predict the deformation field.

[0091] Specifically, the input of the MDFC block is the moving image and fixed image features of the current level resolution (all levels), and the displacement field estimation of the previous level (except the fifth level). In the MDFC block, for the fifth level, there is no , so first, the velocity field u of the moving image feature registration to the fixed image feature is predicted by the flow field estimator, and then the displacement field is integrated.

[0092] For the fourth to first levels, the displacement field of the previous level is up-sampled to the same resolution as the current level, i.e. Then, the moving image feature of the current level is deformed using it to obtain , and then the smoothness estimation is performed with the fixed image feature of the current level to obtain , and finally, the deformation is performed on using , and then added to , to obtain the final displacement field of this level .

[0093] Thus, from the fifth level to the first level, progressive optimization registration is achieved in the process. This design captures global large deformations at a coarse level (low resolution) and optimizes local details at a fine level (high resolution), achieving global accurate registration. The number of pyramid layers n directly affects the registration accuracy, where tasks with greater deformation require more pyramid layers.

[0094] S4: Obtain a medical registration image based on the medical input image and the highest resolution displacement field.

[0095] S5: Perform result evaluation on the highest resolution displacement field based on a complete loss function to optimize the large model registration model; the complete loss function includes an image similarity loss function, a feature similarity loss function, a deformation field smoothness loss function, and a segmentation mask similarity loss function.

[0096] In particular, step S5 comprises:

[0097] S51: after obtaining the medical registration image, calculate the complete loss function;

[0098] S52: pass the gradient error back to each layer of the large model registration network through the back propagation algorithm and update the parameters based on the Adam optimizer;

[0099] S53: when the set number of training rounds is reached, the training stops, and the trained weights of the large model registration network are output and saved.

[0100] In particular, a hybrid loss function comprising an image similarity loss function, a feature similarity loss function, a segmentation mask similarity loss function and a deformation field smoothness loss function is used to jointly optimize the model. The loss function includes two aspects of dissimilarity loss and regularization loss. The former measures the distance between the deformed moving image and the fixed image at each pyramid layer, while the latter ensures a smooth deformation field to avoid unrealistic deformation. In order to improve consistency, the loss of all pyramid layers is calculated, in which the moving image is down-sampled and deformed by the deformation field of each layer to measure the dissimilarity with the down-sampled fixed image. In SAMIR, the dissimilarity loss simultaneously captures image-level and feature-level dissimilarity using image similarity and feature similarity loss functions.

[0101] For the weakly supervised version, segmentation masks are used in network training, and Dice loss is also used The Dice loss and the aforementioned segmentation mask similarity loss correspond to each other, and the calculated regions are different for each data set, and are the most important organ tissues in each image, such as the left ventricle, right ventricle, left atrium, right atrium for heart MRI images; for abdominal images, it is each organ in the abdominal cavity, liver, kidney, etc. It can be seen that Figure Three several regions are outlined. A diffusion regularization term is used to constrain the spatial smoothness of the deformation field. The complete loss function used to train the network is represented by the following formula:

[0102] (1);

[0103] wherein, is the complete loss, is the image similarity loss, is the feature similarity loss, is the segmentation mask similarity loss, is the deformation field smoothness loss, , , and are hyperparameters for balancing, respectively.

[0104] At the image level, we use normalized cross correlation (NCC) to evaluate the similarity between the warped moving image and the fixed image. At the feature level, we construct a hierarchical feature consistency loss to further enhance the registration consistency. The hierarchical features of the moving image and the fixed image at the th level of the feature pyramid are denoted as and respectively. In each level , the hierarchical feature consistency loss is calculated as follows:

[0105] (1).

[0106] where , are the moving image features and the fixed image features of the th feature pyramid respectively, is the warping function, is the highest resolution displacement field, is the number of levels of the pyramid.

[0107] where the image similarity loss is represented by the following equation:

[0108] (2).

[0109] where , are the moving image and the fixed image at the th level respectively, which are obtained by the same multi-scale down-sampling as when obtaining the feature pyramid from the original image, is the mean of the gray values of the th level moving image , is the mean of the gray values of the th level fixed image .

[0110] where the segmentation mask similarity loss is represented by the following equation:

[0111] (3).

[0112] where is the mask of the warped moving image, is the mask of the fixed image. This loss is used in the weakly supervised case, i.e., the masks of the moving image and the fixed image are provided in the training data to assist the training.

[0113] wherein the deformation field smoothness loss is expressed by the following equation:

[0114] (5);

[0115] The loss function is calculated in each training round, the gradient error is transmitted back to each layer of the network using the back propagation algorithm, and the parameters are updated using the Adam optimizer. When the set number of training rounds is reached, the training is ended, and the trained weights of the large model registration network are output and saved to optimize the image registration results in the next time.

[0116] Specifically, the Adam (Adaptive Moment Estimation) optimizer performs adaptive updates on all learnable parameters of the model when training the neural network. Specifically, it optimizes the following two types of parameters: weights, biases, and other learnable parameters (such as normalization layers, embedding layers, etc.).

[0117] The present application uses a general registration network SAMIR, which can effectively extract general visual features across anatomical regions (such as organ topology and boundary continuity), significantly improving the anatomical rationality of the deformation field. Especially in the absence of medical prior knowledge, the migration potential of natural image pre-training models to medical registration tasks is proved. In addition, using a robust feature encoder based on SAM, the model converges quickly and can complete the training process in a short time, which is conducive to the use in practical situations.

[0118] It should be noted that the flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the system, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order from that noted in the drawings. For example, two consecutive blocks can actually be executed in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0120] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. The apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, and for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some communication interfaces, apparatuses or units, which can be electrical, mechanical or other forms.

[0121] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0122] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0123] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0124] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the description of the present application.

Claims

1. A medical image registration method based on large model robust features, characterized in that, The method comprises the following steps: S1: constructing a large model registration network, the large model registration network comprising a structure-aware feature encoder module and a pyramid deformation field prediction module; S2: obtaining a pair of three-dimensional medical input images, extracting features of the medical input images by the structure-aware feature encoder module to obtain image extraction features; the pair of three-dimensional medical input images comprising a moving image and a fixed image; The structure-aware feature encoder module comprises an image preprocessing module, a SAM feature encoder and a three-dimensional convolution module; step S2 comprises: S21: preprocessing the moving image and the fixed image of the pair of three-dimensional medical input images respectively; S22: slicing the pair of preprocessed three-dimensional medical input images along the depth direction to obtain slice images; S23: inputting the slice images into the SAM feature encoder in sequence to obtain initial medical image features of the slice images; S24: stacking the initial medical image features along the depth direction to obtain a pair of initial three-dimensional image features; S25: upsampling and splicing the pair of initial three-dimensional image features to obtain intermediate three-dimensional image features; S26: performing three-dimensional convolution processing on the intermediate three-dimensional image features to obtain image extraction features; S3: deforming and registering the image extraction features based on the pyramid deformation field prediction module to obtain a highest-resolution displacement field; S4: obtaining a medical registration image based on the medical input images and the highest-resolution displacement field; S5: evaluating the highest-resolution displacement field based on a complete loss function to optimize the large model registration network; the complete loss function comprising an image similarity loss function, a feature similarity loss function, a deformation field smoothness loss function and a segmentation mask similarity loss function.

2. The method of claim 1, wherein, The pyramid deformation field prediction module comprises a multi-scale down-sampling unit and a multi-scale displacement field combination block, and step S3 comprises: S31: performing multi-scale down-sampling on the image extraction features to obtain a plurality of feature levels, each feature level comprising a pair of moving image features and fixed image features of the same resolution; S32: performing preliminary registration on the feature level of the lowest resolution by using the scale displacement field combination block to calculate a large-scale deformation field; S33: for the other feature levels except the feature level of the lowest resolution, performing the following operations: interpolating and up-sampling the deformation field of the previous feature level to a high-level resolution as an initial value of the next feature level and inputting the initial value into the MDFC module to obtain a local deformation field; S34: obtaining the highest-resolution displacement field based on the large-scale deformation field and the local deformation field.

3. The method of claim 2, wherein, Step S5 comprises: S51: after obtaining the medical registration image, calculating the complete loss function; S52: transmitting gradient errors back to each layer of the large model registration network by using a back propagation algorithm and updating parameters based on an Adam optimizer; S53: when a set number of training rounds is reached, stopping training, and outputting and saving the training weights of the large model registration network.

4. The method of claim 3, wherein, The complete loss function is represented by the following formula: （1）； wherein, is a complete loss, is an image similarity loss, is a feature similarity loss, is a segmentation mask similarity loss, is a deformation field smoothness loss, , , and are hyperparameters for balancing, respectively.

5. The method of claim 4, wherein, Feature similarity loss is expressed by the following equation: （2）； in, , The feature pyramids are respectively the first Hierarchical moving image features and fixed image features, For deformation function, For the highest resolution displacement field, This refers to the number of levels in the pyramid.

6. The method of claim 5, wherein, Image similarity loss is expressed by the following equation: （3）； in, , The first The moving and stationary images at different levels are obtained by downsampling the original image at the same scale as when the feature pyramid was obtained. For the first Layered moving images The mean gray value, For the first Hierarchical fixed image The average grayscale value.

7. The method of claim 6, wherein, Deformation field smoothness loss is expressed by the following equation: （4）。 8. The method of claim 7, wherein, Segmentation mask similarity loss is expressed by the following equation: （5）； wherein, is a mask of the morphed moving image, is a mask of the fixed image; displacement field of each feature level is expressed by the following equation: （6）； wherein, denotes a 2x trilinear upscaling, is the displacement field of the previous level, is the preliminary displacement field predicted by the flow field estimator for the current level, the specific process is (1) using the flow field estimator constructed by double 3D convolution block to predict the feature matching velocity field u of the moving image to the fixed image; (2) performing integration operation on the velocity field u to obtain .

Images