A medical image registration method, system, medium, device and product

Abstract

The present application relates to the technical field of medical image processing and deep learning, and provides a medical image registration method, system, medium, equipment and product. The medical image registration method comprises: obtaining a medical image to be registered, using a trained image registration model to register the medical image, and obtaining a medical image registration result, wherein the medical image to be registered comprises a moving image and a fixed image; generating a deformation field based on the moving image and the fixed image; based on the moving image, the fixed image and the deformation field, using a high-risk deformation region perception mechanism module, introducing gradient information of the deformation field, dynamically adjusting the weight of the deformation field, extracting fixed image features, moving image features and deformation field features; based on the fixed image features, the moving image features and the deformation field features, using a three-view hierarchical feature fusion module to obtain a final fused feature map; and based on the final fused feature map, using a spatial transformation network to obtain a final deformation field for registering the medical image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing and deep learning, in particular to a medical image registration method, system, medium, equipment and product. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] In medical image registration, the prior art often has difficulty in generating accurate and reasonable deformation fields for regions containing complex anatomical structures and large displacement deformations, such as soft tissue or tumor regions. This is mainly because traditional registration methods rely more on static feature matching between fixed and moving images when learning features, ignoring the dynamic characteristics of the deformation region. In particular, in complex anatomical scenarios, the model has difficulty in effectively dealing with registration errors caused by large displacement. In addition, due to the inherent local perception characteristics of convolutional neural networks, they perform poorly in capturing long-range dependencies and global context information, failing to fully learn the global semantic information of the image that is crucial for registration, resulting in anatomically unreasonable deformations in the deformation field (such as deformation field line crossing), further reducing the clinical applicability of registration.

[0004] In addition, most existing methods fail to adequately consider the topological properties of complex anatomical structures, especially in dynamic deformation regions, where the preservation of topological structure is often overlooked, which can lead to anatomical consistency of organs or tissues being disrupted in the registration results. At the same time, due to the lack of adaptive attention mechanisms for deformation regions when extracting features, the registration model is often disturbed by data noise or irrelevant information, resulting in deformation fields that lack robustness and accuracy, especially under diverse clinical data distributions. SUMMARY

[0005] To solve the technical problems existing in the background art, the present application provides a medical image registration method, system, medium, equipment and product, which designs an unsupervised deformation perception adaptive learning framework (UDA 2 L) for medical image registration. The framework adopts a high-risk deformation region perception mechanism (HDA 2 M), a three-view hierarchical feature fusion module (THFI) and a cross-consistency recurrent learning (C 3 ) module to improve the registration accuracy of medical images, maintain topological consistency and enhance the robustness of the model.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] The first aspect of the present application provides a medical image registration method.

[0008] A medical image registration method, comprising:

[0009] Obtaining a medical image to be registered, using a trained image registration model to register the medical image, obtaining a medical image registration result, the medical image to be registered including a moving image and a fixed image; generating a deformation field based on the moving image and the fixed image;

[0010] Wherein, the image registration model includes a high-risk deformation area perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, the fixed image and the deformation field, the high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field and dynamically adjust the weight of the deformation field, and the fixed image features, the moving image features and the deformation field features are extracted; based on the fixed image features, the moving image features and the deformation field features, the three-view hierarchical feature fusion module is adopted to obtain the final fusion feature map; based on the final fusion feature map, the spatial transformation network is adopted to obtain the final deformation field, which is used to register the medical image.

[0011] Further, based on the moving image, the fixed image and the deformation field, the high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field and dynamically adjust the weight of the deformation field, and the fixed image features, the moving image features and the deformation field features are extracted; the method comprises:

[0012] The fixed image and the moving image are spliced, linearly projected and layer normalized to obtain a query vector and a key vector; based on the deformation field, the Gradient3D network is adopted to calculate the gradient coefficient of the extracted deformation field; the gradient coefficient of the deformation field is integrated into the query vector and the key vector respectively to obtain an enhanced query vector and an enhanced key vector; the enhanced query vector and the enhanced key vector are subjected to attention mechanism processing to obtain the deformation field features;

[0013] The moving image and the fixed image are subjected to attention mechanism processing to obtain the fixed image features and the moving image features respectively.

[0014] Further, the gradient coefficient of the extracted deformation field is calculated using the following formula:

[0015]

[0016] Wherein, The gradient coefficient of the deformation field φ extracted by Gradient3D is represented as The absolute values of the partial derivatives of the deformation field φ in the x, y and z directions are represented as

[0017] Further, the three-view hierarchical feature fusion module is adopted based on the fixed image feature, the moving image feature and the deformation field feature to obtain a final fusion feature map; the method comprises:

[0018] The three-view hierarchical feature fusion module comprises four-layer decoders and a THFI module, each layer decoder decodes the fixed image feature, the moving image feature and the deformation field feature, and the convolution block of the THFI module is aggregated to generate a fusion feature map; the generated fusion feature map is input to a deformation field estimation module of the THFI module to predict a residual deformation field; the predicted residual deformation field is combined with the deformation field of the previous layer to generate an optimized deformation field; after the processing of the four-layer decoders and the THFI module, a final fusion feature map is obtained.

[0019] Further, in the image registration model training process, a cross-consistency cycle learning module is further included, which is used to calculate a deformation field φ X→Y of the image X to the image Y through a registration network X→Y , to obtain a registration image X'; the same registration network is used to calculate a deformation field φ Y→X from the image Y to the image X Y→X ; the deformation field φ Y→X is applied to the image Y to obtain a registration image Y'; the inverse deformation field -φ X→Y is used to deform the registration image X' to obtain a cycle-consistent image X"; the inverse deformation field -φ cross is used to deform the registration image Y' to obtain a cycle-consistent image Y"; a comprehensive loss function is constructed to optimize the learning parameters of the image registration model.

[0020] Further, the comprehensive loss function is expressed by the following formula:

[0021] L sim =L reg +αL cycle

[0022] L sim =-NCC(X′,Y)-NCC(Y′,X)

[0023]

[0024] L cycle =||X″-X||1+||Y″-Y||1

[0025] Wherein, L cross represents the comprehensive loss function, L sim represents the similarity loss, L reg represents the regularization loss, and L cycledenotes the cycle-consistency loss, NCC denotes the normalized cross correlation, and || ||1denotes the L1 norm.

[0026] A second aspect of the present application provides a medical image registration system.

[0027] A medical image registration system comprises:

[0028] A prediction module is configured to: acquire medical images to be registered, register the medical images using a trained image registration model to obtain a medical image registration result, the medical images to be registered including a moving image and a fixed image; and generate a deformation field based on the moving image and the fixed image.

[0029] The image registration model comprises a high-risk deformation region perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, the fixed image and the deformation field, the high-risk deformation region perception mechanism module is adopted to introduce gradient information of the deformation field and dynamically adjust a weight of the deformation field, and fixed image features, moving image features and deformation field features are extracted; based on the fixed image features, the moving image features and the deformation field features, the three-view hierarchical feature fusion module is adopted to obtain a final fusion feature map; and based on the final fusion feature map, a spatial transformation network is adopted to obtain a final deformation field for registering the medical images.

[0030] A third aspect of the present application provides a computer-readable storage medium.

[0031] A computer-readable storage medium has a computer program stored thereon, and the program, when executed by a processor, implements the steps in the medical image registration method of the first aspect described above.

[0032] A fourth aspect of the present application provides a computer device.

[0033] A computer device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor implements the steps in the medical image registration method of the first aspect described above when executing the program.

[0034] A fifth aspect of the present application provides a computer program product or a computer program.

[0035] The present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the steps in the medical image registration method of the first aspect described above.

[0036] Compared with the prior art, the application has the beneficial effects that:

[0037] The application provides an unsupervised deformation perception adaptive learning framework (UDA 2 L). By introducing a high-risk deformation area perception mechanism (HDA 2 M), significant features in the deformation area are dynamically captured, and a multi-level feature integration module (THFI) is combined to fuse local details and global context information, so that the performance of the registration model in a complex anatomical scene is effectively improved.

[0038] The application further improves the model's ability to maintain topological consistency by constructing cross-consistency constraints (C 3 ), and ensures that the deformation field generated in the registration process has anatomical rationality and can meet the diversified needs of complex clinical scenes. BRIEF DESCRIPTION OF DRAWINGS

[0039] The drawings accompanying the specification of the application form part of the application and serve to provide a further understanding of the application, the illustrative embodiments thereof, and their description serve to explain the application without imposing undue limitations on the application.

[0040] Figure 1 is a flowchart of the medical image registration method shown in the application;

[0041] Figure 2 is a network framework diagram of the high-risk deformation area perception mechanism (HDA 2 M) shown in the application;

[0042] Figure 3 is a structure diagram of the first three decoding layers in the three-view hierarchical feature fusion module (THFI) shown in the application;

[0043] Figure 4 is a structure diagram of the last decoding layer in the three-view hierarchical feature fusion module (THFI) shown in the application;

[0044] Figure 5 is a comparison diagram of registration on the IXI dataset shown in the application;

[0045] Figure 6 is a framework diagram of the unsupervised deformation perception adaptive learning framework (UDA 2 L) shown in the application. DETAILED DESCRIPTION

[0046] The application will be further described below in conjunction with the drawings and examples.

[0047] It should be noted that the following detailed description is intended to provide further description of the application. Unless defined otherwise, all technical and scientific terms and any acronyms used herein have the same meanings as commonly understood by one of ordinary skill in the art to which this application belongs.

[0048] It is also important to note that the term "or" as used herein is intended to mean an inclusive "or," such that "A or B" means any or all of the items listed with no options required to be selected with the item. As used herein, the term "and / or" comprises any and all combinations of one or more of the associated listed items. Unless defined otherwise, all technical and scientific terms and any acronyms used herein have the same meanings as commonly understood by one of ordinary skill in the art to which this application belongs.

[0049] It should be noted that the flow diagrams and block diagrams in the drawings are representative of the architectural, functional, and operational aspects of possible implementations of methods and systems according to various embodiments of the present disclosure. It should be noted that each block in the flow diagrams and block diagrams can represent a module, a segment, or a portion of code, which includes one or more executable instructions for implementing the specified logical functions (s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession can in fact be executed substantially concurrently or can sometimes be executed in reverse order, depending on the functionality involved. It will also be noted that each block and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and

[0050] Embodiment One

[0051] As Figure 1As shown, the embodiment provides a medical image registration method. The embodiment takes the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server and is realized through interaction of the terminal and the server. The server can be a physical server, a server cluster composed of multiple physical servers or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, security services CDN, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like, but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application. In the embodiment, the method includes the following steps:

[0052] Obtaining a medical image to be registered, using a trained image registration model to register the medical image, and obtaining a medical image registration result, the medical image to be registered including a moving image and a fixed image; generating a deformation field based on the moving image and the fixed image;

[0053] The image registration model includes a high-risk deformation area perception mechanism module and a three-view hierarchical feature fusion module. Based on the moving image, the fixed image and the deformation field, the high-risk deformation area perception mechanism module is used to introduce gradient information of the deformation field and dynamically adjust the weight of the deformation field, and the fixed image features, the moving image features and the deformation field features are extracted. Based on the fixed image features, the moving image features and the deformation field features, the three-view hierarchical feature fusion module is used to obtain the final fusion feature map. Based on the final fusion feature map, the spatial transformation network is used to obtain the final deformation field for registering the medical image.

[0054] The present application provides an unsupervised deformation perception adaptive learning framework (UDA 2 L) for medical image registration. The framework adopts a high-risk deformation area perception mechanism (HDA 2 M), a three-view hierarchical feature fusion module (THFI) and a cross-consistency cycle learning (C 3 ) module to improve the registration accuracy of medical images, maintain topological consistency and enhance the robustness of the model. Each module is described in detail as follows:

[0055] 1. High-risk deformation area perception mechanism (HDA 2 M)

[0056] Medical images typically contain numerous intricate and complex anatomical structures. Furthermore, the gradient fluctuations in the deformation field within these critical regions are significant, leading to a substantial increase in registration difficulty. To address these issues, this invention proposes a High-Risk Deformation Region Sensing Mechanism (HDA). 2 M, such as Figure 2 (As shown). HDA 2 M introduces gradient information of the deformation field based on a multi-head local attention mechanism and dynamically adjusts the weight of the deformation field, so that high-risk and highly deformed regions receive higher attention weights, thereby improving the accuracy of feature extraction and registration.

[0057] Assume we have a fixed image F, a moving image M, and a matching 3D deformation field. Where H, W, and L represent height, width, and length, respectively, and 3 represents the x, y, and z directions in three-dimensional space.

[0058] In HDA 2 In M, an image patch is formed by connecting features F and M (see...). Figure 2 After linear projection (LP) and layer normalization (LN) processing, query (Q) and key (K) vectors are generated, and the formulas are as follows:

[0059] Q = LN(LP(concat(F,M))) (1)

[0060] K = LN(LP(concat(F,M))) (2)

[0061] Here, concat() represents feature concatenation. It can be decomposed into HDA 2 M has multiple attention heads, namely {Q 1 Q 2 ,…,Q S} and {K 1 ,K 2 ,…,K S}, where S represents the number of attention modules and c represents the number of channels.

[0062] Meanwhile, the Gradient3D operation combines a convolutional neural network (CNN) and a differential operator to extract the gradient map of the deformable field. This operation calculates the absolute value of the gradient map and normalizes it, with each partial derivative approximated by convolution with the Sobel operator. The formula for calculating the gradient of the deformable field is as follows:

[0063]

[0064] in, This represents the gradient coefficient of the deformation field φ extracted by Gradient3D; respectively, are the partial derivatives of the deformation field in x, y, z directions; max represents the maximum value function.

[0065] Subsequently, the gradient information of the deformation field is integrated into the enhanced query vector and the enhanced key vector to strengthen the model's attention to the key deformation area, and the specific steps are as follows:

[0066]

[0067] where Q enhanced represents the enhanced query vector, K enhanced represents the enhanced key vector.

[0068] The attention mechanism formula of the high-risk deformation area is:

[0069]

[0070] where Attention() represents the attention mechanism, softmax() represents the softmax function, V represents the corresponding value vector (Value), d k represents the dimension of the query vector and the key vector, represents the transpose of the enhanced key vector.

[0071] Based on the above formula and feature extraction method, the HDA 2 M not only considers the feature correlation in the attention mechanism, but also specially introduces the gradient information of the deformation field, making the attention mechanism more sensitive to the deformation and motion areas contained in the input image, thereby significantly improving the accuracy and robustness of registration.

[0072] The present application designs a high-risk deformation area perception mechanism (HDA 2 M) for dynamically adjusting the weight of the high-risk area in the feature extraction stage to improve the model's attention to complex anatomical structures and large deformation areas, thereby enhancing the accuracy of the registration results.

[0073] 2, three-view hierarchical feature fusion module (THFI)

[0074] Many researchers have tried to improve the model's ability to capture complex image details through feature enhancement methods such as pyramid structure, multi-resolution registration, cascade structure, wavelet transform, and edge information. However, these methods often fail to fully utilize the features of the image itself, but rely on other means to improve the model performance.

[0075] To solve this problem, the present application proposes a three-view hierarchical feature fusion module (THFI). The module integrates the features of fixed images, moving images and deformation fields by recursion, and optimizes the deformation process layer by layer, so as to better handle complex anatomical structure changes and large deformation regions. The THFI module maximizes the compensation of the complementary information lost in the network layer processing, further improving the accuracy and efficiency of registration.

[0076] The entire decoder contains four decoding layers (l = 4, 3, 2, 1), which predict the deformation field from coarse to fine layer by layer. From the deepest layer l4 to the shallowest layer l1, each layer of the decoder generates a specific deformation field φ l The calculation process is shown in Figure 3 、 Figure 4 . First, the features of the fixed image and the moving image are extracted by the lth layer of the decoder, and they are normalized to the same size by downsampling processing. The deformation field φ l+1 predicted by the l+1th layer is used to spatially transform the features of the moving image to obtain the distorted moving features. The above features M' l are passed to the concatenation and convolution (CC) block for feature map aggregation, thereby generating the fused feature map C l , as shown in Figure 3 . Subsequently, the fused feature map C l is input to the deformation field estimation module (Def) to predict the residual deformation field (as shown in Figure 3 ). Finally, the deformation fusion operation (DF) combines the newly predicted residual field with the deformation field φ l+1 of the previous layer to generate the optimized deformation field φ l . When l = 4, this layer does not need to supplement the deformation field from the previous layer, so the THFI structure does not contain the distorted feature or deformation field fusion component (as shown in Figure 4 ).

[0077] The THFI module helps to learn complex deformations by performing necessary feature fusion operations at each decoding layer. In addition, it can integrate and propagate high-level semantic information in each decoding layer, thereby expanding the receptive field at a finer resolution level and enabling shallow feature maps to obtain deep semantic information.

[0078] In the 4th layer decoder, the feature maps and are input to the CC block to generate the fused feature map C4:

[0079]

[0080] where F l and M l: The fixed image and the moving image are input to the encoder, and the first layer generates feature maps F1 and M1 after convolution (Conv), instance normalization (InsNorm) and LeakyRelu (LRelu). represents the fixed image feature output in the fourth layer encoder. represents the moving image feature output by the fourth layer encoder. The CC block is composed of two groups of convolution + instance normalization (InsNorm) + Leaky ReLU (LRelu) operations. Then, C4 is sent to Def to generate the first residual deformation field of the deepest layer That is, φ1.

[0081] After the initial fusion feature map is generated in the fourth layer, the following operations are as follows Figure 3 as shown in the formula:

[0082]

[0083] wherein, represents the deformation field φ l+1 under the action of the previously obtained deformation field φ

[0084] Finally, the pyramid decoder generates a semantic-rich deformation field φ, which is then used to deform the moving image to align with the fixed image.

[0085] In order to solve the problem of information loss in the process of processing multi-level features, the application proposes a three-view hierarchical feature fusion module (THFI), which integrates the fixed image, the moving image and the deformation field feature by recursion, optimizes the deformation process layer by layer, and realizes efficient and accurate image registration.

[0086] 3, cross-consistency cycle learning (C 3 )

[0087] In order to maintain the reversibility and topological consistency of the deformation field when registering single-modal brain magnetic resonance images, the application proposes a C 3 learning, which can ensure the consistency of forward and reverse deformation and the reversibility of the deformation field.

[0088] C 3Learning using a shared registration network N, which handles both forward and inverse registration tasks simultaneously during training. Specifically, the parameters of network N are shared between forward and inverse registration. When the input is (X, Y), network N computes the deformation field (φ X→Y ) from source image X to target image Y; when the input is (Y, X), network N computes the deformation field φ Y→X from target image Y to source image X. The specific steps are as follows.

[0089] Forward registration: registration network N is used to compute the deformation field φ X→Y from image X to image Y:

[0090] φ X→Y = N(X, Y) (11)

[0091] Apply deformation field φ X→Y to image X to get the registered image X':

[0092] X' = θ(X, φ X→Y ) (12)

[0093] where the θ operation represents a spatial transformation of image X by deformation field φ X→Y .

[0094] Inverse registration: using the same registration network N to compute the deformation field φ Y→X from image Y to image X:

[0095] φ Y→X = N(Y, X) (13)

[0096] Apply deformation field φ Y→X to image Y to get the registered image Y':

[0097] Y' = θ(Y, φ Y→X ) (14)

[0098] To ensure the invertibility of the deformation field, the present application establishes a C 3 constraint to ensure the invertibility of the deformation field and the consistency of the topological structure. Among them, the inverse deformation field -φ Y→X is used to deform the registered image X' to get the cyclic consistent image X":

[0099] X" = θ(X', -φ Y→X ) (15)

[0100] where -φ Y→X represents the inverse deformation field used in the inverse deformation step.

[0101] Using the inverse deformation field -φ X→YThe registered image Y' is deformed to obtain a cycle-consistent image Y":

[0102] Y" = Θ(Y', -φ X→Y ) (16)

[0103] Finally, to realize unsupervised learning, the application designs a comprehensive loss function L cross . The function constrains the correspondence between the registered image and the target image, the smoothness of the deformation field, and the consistency of the cycle registration process under three parts: similarity loss L sim , regularization loss L reg , and cycle consistency loss L cycle . These constraints can ensure that the model learns effectively without supervision labels, thereby achieving accurate and robust registration results.

[0104] L cross = L sim + αL reg + βL cylce (17)

[0105] Wherein, α and β represent weight coefficients, controlling the relative importance between similarity loss, regularization loss and cycle consistency loss. According to the experimental results, adjust these weights to optimize the training of the model.

[0106] The similarity function uses the normalized cross correlation (NCC) between (X', Y) and (Y', X) to reduce sensitivity to contrast changes:

[0107] L sim = -NCC(X', Y) - NCC(Y', X) (18)

[0108] Where NCC is:

[0109]

[0110] Wherein, Ω is the entire three-dimensional volume, and are the local mean values of the volumes X(v) and Y(v), respectively, v i represents the i-th voxel position in the image volume. X(v i ) and Y(v i ) represent the pixel values of image X and image Y at voxel position v i .

[0111] The regularization function penalizes the smoothness of the deformation field to prevent unrealistic deformations. This can be achieved by the L2 norm of the deformation field gradient φ:

[0112]

[0113] where φ X→Y is the deformation vector field produced by network N with input X and Y, φ Y→X is the deformation vector field obtained by network N with input Y and X.

[0114] The gap between the initial image and the cycle-consistent image is determined by the following factors:

[0115] L cycle =||X″-X||1+||Y″-Y||1 (21)

[0116] where || ||1 represents the L1 norm.

[0117] In summary, by introducing C 3 learning, the model can effectively maintain the topological structure and reversibility of the deformation field during registration. This innovative approach not only improves registration accuracy, but also enhances the robustness and stability of the network in handling complex deformations and single-modality brain magnetic resonance imaging data.

[0118] The present invention constructs a shared network structure through the cross-cycle consistency learning (C 3 ) mechanism, simultaneously completing the forward and reverse registration tasks to ensure the reversibility and topological consistency of the deformation field.

[0119] 4. Introduction of data set and experimental parameter setting

[0120] To evaluate the effectiveness of the proposed model, the present invention conducted experiments on three brain MRI data sets. LPBA40 (40 T1W MR images), IXI (576 images), and OASIS (414 images). For LPBA40 and IXI, atlas-to-patient registration was performed, while OASIS focused on obtaining high-precision registration for structures of different shapes and sizes for different patients. All magnetic resonance scans were preprocessed by FreeSurfer, including skull stripping, spatial normalization, affine transformation, and automatic structure segmentation. OASIS and IXI scans were cropped to 160x192x224, and LPBA40 scans were cropped to 160x192x160. The evaluation used 56 structures (LPBA40) and 30 structures (IXI) segmentation maps.

[0121] To comprehensively evaluate the method proposed in the present invention, it was compared with seven leading traditional methods and deep learning methods.

[0122] All models were trained for 500 epochs with an adaptive moment estimation (Adam) optimizer with a batch size of 1. The method of the present application utilizes both LNCC and MSE loss, with the coefficient of the L2 regularization term optimized to 4 for the LNCC loss and 0.04 for the MSE loss. The learning rate for both losses was set to 0.0004.

[0123] The evaluation metrics in the experiment are as follows:

[0124] DSC (Dice Similarity Coefficient): used to measure the volume overlap of the anatomical segmentation results. The higher the DSC, the better the registration effect, and the standard deviation (STD) is used to evaluate the stability.

[0125]

[0126] where A and B are the voxel sets in the fixed image and the moving image segmentation, respectively.

[0127] Jacobian matrix: J φ (v)≤0} : represents the set of voxels whose Jacobian matrix is less than or equal to zero during the deformation process, i.e. the voxels that have undergone folding (inverse deformation). The size of this set reflects the number of voxels that have experienced topological errors or irreversible deformation during the entire deformation process.

[0128]

[0129] Jacobian matrix (Jacobian matrix), representing the voxel transformation gradient under the deformation field φ. It describes the local spatial distortion in image deformation. Its value reflects the changes in shape and volume of voxels during deformation. |V| represents the total number of voxels in the volume. φ (v)≤0} : represents the set of voxels whose Jacobian matrix is less than or equal to zero during the deformation process, i.e. the voxels that have undergone folding (inverse deformation). The size of this set reflects the number of voxels that have experienced topological errors or irreversible deformation during the entire deformation process.

[0130] Wilcoxon rank sum test: Wilcoxon rank sum test is used to evaluate the significant difference between two groups.

[0131] The method of the present application is compared with other methods.

[0132]

[0133] where X i represents the samples in the first group, n i represents the number of samples contained in the first group.

[0134] On the LPBA40 dataset (see Table 1), UDA 2 L obtained the highest DSC value, indicating that UDA 2L can effectively learn and capture the basic features and structural information of images, even on small-scale datasets. This effect is mainly attributed to UDA 2 The L model effectively integrates multi-view and multi-level features, ensuring more accurate and robust image registration results. In addition, the network of the present application has almost no folding in the deformation field (|J φ The percentage of |J

[0135] On the IXI dataset, as shown in Table 1, the proposed method obtained the highest DSC value with the lowest Jacobian determinant value compared with all other methods. This performance benefits from the UDA 2 The efficient feature extraction and integration mechanism of the L model, as well as its unique training constraint condition, not only improves the performance of the model, but also ensures the stability and reversibility of the registration process. This fully embodies the advantages of the proposed method in achieving accurate and consistent image registration results.

[0136] On the OASIS dataset, the method of the present application obtained the highest DSC value (0.818), indicating that the anatomical structures in the aligned image are most similar to those in the fixed image. This result is mainly attributed to the adaptive learning strategy of UDA 2 L, which effectively solves the complexity and variability of anatomical structures in the OASIS dataset and dynamically adjusts the deformation field according to the specific characteristics of the data, thereby achieving excellent performance.

[0137] Figure 5 Visual comparisons of different registration methods on the IXI dataset are shown. The first column includes the fixed image and the moving image. In addition to the first column, the results of VoxelMorph, CycleMorph, Vit-V-Net, TransMorph, TransMatch, PAN and the proposed method are shown from left to right. The warped moving image, estimated deformation (represented in RGB), estimated deformation (represented in mesh), and the absolute difference between the warped moving image and the ground truth fixed image are shown from top to bottom in the figure.

[0138] As shown in Table 1, the higher the Dice score, the more accurate the registration result. |J φ |≤0 represents the average percentage of folding in the deformation field. The smaller the value, the higher the smoothness. STD(|J φ |) represents the standard deviation of the Jacobian value.

[0139] Table 1 Qualitative comparison of the model of the present application and other methods on LPBA40, IXI and OASIS datasets

[0140]

[0141] According to Figure 5 the second and third rows of Table 1, the method of the present application produces smoother deformation than the other six deep learning methods, with almost no folding. The grid plot of deformation field in the third row shows that the method of the present application has less intersection than the other six methods. In the last row of Table 1, the present application visualizes the absolute difference map between the warped image and the fixed image obtained by different methods. The red box shows that the method of the present application generates the difference map with the lightest color and the smallest difference, which is closest to the fixed difference map. Figure 5

[0142] Figure 6 UDA 2 L is the invented model of the present application 2 L is based on a shared bidirectional network N, which simultaneously processes forward and backward registration tasks during training. UDA 2 L's encoder divides the three inputs (moving image M, fixed image F, and deformation field (obtained by symmetric normalization)) into non-overlapping three-dimensional image blocks. The gray value of each block is tiled into a 1x64 vector and converted into a new feature representation through linear projection. After completing the linear projection step, the image blocks are sent to the image block merger, and the merged image blocks are sent to the feature extraction module. During feature learning, HDA 2 M will perform multi-level identification, weighting and adjustment on high-risk areas to ensure that UDA 2 L can adaptively focus on these areas during feature extraction, thereby improving the accuracy and robustness of the registration result. For the decoder of UDA 2 L, the present application proposes a THFI module that uses a pyramid structure and convolutional blocks to perform multi-level recursive extraction, merging and refinement of feature maps obtained from three viewpoints (fixed image, moving image and deformation field), gradually refining the deformation, improving the efficiency of the registration process, and enabling UDA 2 L to achieve comprehensive and detailed image registration. To facilitate the model to learn the topological structure between images and improve the reversibility of the registration process, the present application performs C 3 learning between the original image and the deformation field, and ensures the reversibility of the registration process and the consistency of the topological structure by enforcing multi-level constraints.

[0143] Embodiment Two

[0144] The present embodiment provides a medical image registration system.

[0145] A medical image registration system comprises:

[0146] ​The prediction module is configured to: acquire a medical image to be registered, register the medical image by using a trained image registration model, and obtain a medical image registration result, the medical image to be registered including a moving image and a fixed image; and generate a deformation field based on the moving image and the fixed image.

[0147] The image registration model includes a high-risk deformation region perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, the fixed image and the deformation field, the high-risk deformation region perception mechanism module is adopted to introduce gradient information of the deformation field and dynamically adjust a weight of the deformation field, and fixed image features, moving image features and deformation field features are extracted; based on the fixed image features, the moving image features and the deformation field features, the three-view hierarchical feature fusion module is adopted to obtain a final fusion feature map; and based on the final fusion feature map, a spatial transformation network is adopted to obtain a final deformation field, which is used to register the medical image.

[0148] In some embodiments, the prediction module is further configured to: perform stitching, linear projection and layer normalization processing on the fixed image and the moving image to obtain a query vector and a key vector; based on the deformation field, a Gradient3D network is adopted to calculate gradient coefficients of the extracted deformation field; the gradient coefficients of the deformation field are integrated into the query vector and the key vector respectively to obtain an enhanced query vector and an enhanced key vector; attention mechanism processing is performed on the enhanced query vector and the enhanced key vector to obtain deformation field features; and attention mechanism processing is performed on the moving image and the fixed image to obtain fixed image features and moving image features respectively.

[0149] In some embodiments, the gradient coefficients of the extracted deformation field are calculated by using the following formula:

[0150]

[0151] wherein, denotes gradient coefficients of the deformation field φ extracted by Gradient3D; denote absolute values of partial derivatives of the deformation field φ in x, y and z directions respectively; and max denotes a maximum value function.

[0152] In some embodiments, the prediction module is further configured to: the three-view hierarchical feature fusion module includes four decoders and a THFI module, each decoder decodes the fixed image features, the moving image features and the deformation field features, and the decoded features are aggregated by a convolution block of the THFI module to generate a fusion feature map; the generated fusion feature map is input to a deformation field estimation module of the THFI module to predict a residual deformation field; the predicted residual deformation field is combined with a deformation field of a previous layer to generate an optimized deformation field; and finally, the final fusion feature map is obtained through processing of the four decoders and the THFI module.

[0153] In some embodiments, the system further includes a training module configured to: during the image registration model training process, include a cross-consistency cyclic learning module for calculating the deformation field φ from image X to image Y through the registration network. X→Y Apply a deformation field φ to image X X→Y The registered image X' is obtained; the deformation field φ from image Y to image X is calculated using the same registration network. Y→X ; the deformation field φ Y→X Applied to image Y, the registered image Y′ is obtained; using the inverse deformation field -φ Y→X The registered image X' is deformed to obtain a cyclically consistent image X”; the inverse deformation field -φ is then used. X→Y The registered image Y' is deformed to obtain a cyclically consistent image Y”; a comprehensive loss function is constructed to optimize the learning parameters of the image registration model.

[0154] In some embodiments, the comprehensive loss function is expressed by the following formula:

[0155] L cross =L sim +αL reg +βL cycle

[0156] L sim =-NCC(X′,Y)-NCC(Y′,X)

[0157]

[0158] L cycle =||X″-X||1+||Y″-Y||1

[0159] Among them, L cross L represents the comprehensive loss function. sim L represents the similarity loss. reg L represents the regularization loss. cycle represents the cycle consistency loss, NCC represents the normalized cross-correlation, and |||1 represents the L1 norm.

[0160] Example 3

[0161] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the medical image registration method as described in Embodiment 1 above.

[0162] Example 4

[0163] The embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the medical image registration method according to the embodiment one when executing the program.

[0164] Embodiment five

[0165] The embodiment provides a computer program product or computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the medical image registration method according to the embodiment one.

[0166] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including, but not limited to, disk storage and optical storage, etc.) containing computer usable program code.

[0167] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0168] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0169] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operational steps are performed on the computer or other programmable data processing device to generate a computer-implemented process, thus the instructions executed on the computer or other programmable data processing device provide the function of implementing the processes specified in the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the block Figure 1 one block or multiple blocks.

[0170] Those of ordinary skill in the art can understand that all or part of the flow of the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the flow of the above-mentioned embodiment of each method. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), and the like.

[0171] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A medical image registration method, characterized in that, include: The medical image to be registered is acquired, and the medical image is registered using a trained image registration model to obtain the medical image registration result. The medical image to be registered includes a moving image and a fixed image. A deformation field is generated based on the moving image and the fixed image. The image registration model includes a high-risk deformation region perception mechanism module and a three-view layered feature fusion module. Based on moving images, fixed images, and deformation fields, a high-risk deformation region perception mechanism module is employed. Gradient information from the deformation field is introduced, and its weights are dynamically adjusted to extract features from fixed images, moving images, and deformation fields. The method includes: Static and moving images are stitched together, linearly projected, and layer normalized to obtain query vectors and key vectors. Based on the deformation field, the gradient coefficients of the extracted deformation field are calculated using a Gradient3D network. The gradient coefficients of the deformation field are integrated into the query vector and key vector respectively to obtain enhanced query vectors and enhanced key vectors. Attention mechanisms are applied to the enhanced query vectors and enhanced key vectors to obtain deformation field features. The moving and fixed images are processed using an attention mechanism to obtain features for the fixed and moving images, respectively. Based on fixed image features, moving image features, and deformation field features, a three-view hierarchical feature fusion module is used to obtain the final fused feature map; based on the final fused feature map, a spatial transformation network is used to obtain the final deformation field, which is used for medical image registration.

2. The medical image registration method according to claim 1, characterized in that, The gradient coefficients of the extracted deformation field are calculated using the following formula: in, This represents the deformation field extracted using Gradient3D. The gradient coefficient; Deformation fields The absolute values ​​of the partial derivatives in the x, y, and z directions; max represents the maximum value function.

3. The medical image registration method according to claim 1, characterized in that, The method, based on fixed image features, moving image features, and deformation field features, employs a three-view hierarchical feature fusion module to obtain the final fused feature map; the method includes: The three-view hierarchical feature fusion module includes a four-layer decoder and a THFI module. Each layer decoder decodes fixed image features, moving image features, and deformation field features. These features are then aggregated by the convolutional blocks of the THFI module to generate a fused feature map. The generated fused feature map is input to the deformation field estimation module of the THFI module to predict the residual deformation field. The predicted residual deformation field is then merged with the deformation field of the previous layer to generate an optimized deformation field. After processing by the four-layer decoder and the THFI module, the final fused feature map is obtained.

4. The medical image registration method according to claim 1, characterized in that, The image registration model training process also includes a cross-consistency recurrent learning module, which is used to calculate the deformation field from image X to image Y through the registration network. Apply a deformation field to image X The registered image X' is obtained; the deformation field from image Y to image X is calculated using the same registration network. ; Deformation field Applying this to image Y yields the registered image. ; Using reverse deformation field The registered image X' is deformed to obtain a cyclically consistent image X''; the inverse deformation field is then used. The registered image Y' is deformed to obtain a cyclically consistent image Y''; a comprehensive loss function is constructed to optimize the learning parameters of the image registration model.

5. The medical image registration method according to claim 4, characterized in that, The comprehensive loss function is expressed by the following formula: in, Represents the comprehensive loss function. Represents similarity loss. Indicates the regularization loss. Indicates the loss of cycle consistency. Indicates normalized cross correlation. This represents the L1 norm.

6. A medical image registration system, characterized in that, include: The prediction module is configured to: acquire a medical image to be registered; register the medical image using a trained image registration model to obtain a medical image registration result; wherein the medical image to be registered includes a moving image and a fixed image; and generate a deformation field based on the moving image and the fixed image. The image registration model includes a high-risk deformation region perception mechanism module and a three-view layered feature fusion module. Based on moving images, fixed images, and deformation fields, a high-risk deformation region perception mechanism module is employed. Gradient information from the deformation field is introduced, and its weights are dynamically adjusted to extract features from fixed images, moving images, and deformation fields. The method includes: Static and moving images are stitched together, linearly projected, and layer normalized to obtain query vectors and key vectors. Based on the deformation field, the gradient coefficients of the extracted deformation field are calculated using a Gradient3D network. The gradient coefficients of the deformation field are integrated into the query vector and key vector respectively to obtain enhanced query vectors and enhanced key vectors. Attention mechanisms are applied to the enhanced query vectors and enhanced key vectors to obtain deformation field features. The moving and fixed images are processed using an attention mechanism to obtain features for the fixed and moving images, respectively. Based on fixed image features, moving image features, and deformation field features, a three-view hierarchical feature fusion module is used to obtain the final fused feature map; based on the final fused feature map, a spatial transformation network is used to obtain the final deformation field, which is used for medical image registration.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the medical image registration method as described in any one of claims 1-5.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the medical image registration method as described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the medical image registration method as described in any one of claims 1-5.

Images