Medical image registration method and system, medium, equipment and product

Abstract

The invention relates to the technical field of medical image processing and deep learning, and provides a medical image registration method and system, a medium, equipment and a product. The medical image registration method comprises the steps that medical images to be registered are acquired, a trained image registration model is used for registering the medical images, a medical image registration result is obtained, and the medical images to be registered comprise a moving image and a fixed image; generating a deformation field based on the moving image and the fixed image; based on the moving image, the fixed image and the deformation field, a high-risk deformation area sensing mechanism module is adopted, gradient information of the deformation field is introduced, the weight of the deformation field is dynamically adjusted, and fixed image features, moving image features and deformation field features are extracted; based on the fixed image features, the moving image features and the deformation field features, a three-view hierarchical feature fusion module is adopted to obtain a final fusion feature map; and based on the final fusion feature map, adopting a spatial transformation network to obtain a final deformation field for registering the medical image.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing and deep learning technology, and in particular to a medical image registration method, system, medium, equipment and product. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In medical image registration, existing technologies often have difficulty generating accurate and reasonable deformation fields for areas containing complex anatomical structures and large displacement deformations (such as soft tissue or tumor areas). This is mainly because traditional registration methods rely more on static feature matching between fixed and moving images during feature learning, while ignoring the dynamic characteristics of the deformed area. Especially in complex anatomical scenes, the model is difficult to effectively deal with registration errors caused by large displacements. In addition, due to the inherent local perception characteristics of convolutional neural networks, they are insufficient in capturing long-range dependencies and global contextual information, and cannot fully learn the global semantic information of the image that is crucial for registration, resulting in anatomically unreasonable deformations in the deformation field (such as crossing of deformation field lines), further reducing the clinical applicability of registration.

[0004] In addition, most existing methods fail to fully consider the topological characteristics of complex anatomical structures, especially in dynamically deformed regions, where the preservation of topological structures is often neglected, which may lead to the destruction of the anatomical consistency of organs or tissues in the registration results. At the same time, when extracting features, the registration model is often disturbed by data noise or irrelevant information due to the lack of an adaptive attention mechanism for the deformed region, resulting in the lack of robustness and accuracy of the generated deformation field, especially under the diverse distribution of clinical data. Summary of the invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a medical image registration method, system, medium, device and product. The present invention designs an unsupervised deformation-aware adaptive learning framework (UDA) for medical image registration. 2 L). The framework adopts the high risk deformation area perception mechanism (HDA) 2 M), three-view hierarchical feature fusion module (THFI) and cross-consistency loop learning (C 3 ) module is used to improve the registration accuracy of medical images, maintain topological consistency, and enhance the robustness of the model.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides a medical image registration method.

[0008] A medical image registration method, comprising:

[0009] Acquire a medical image to be registered, register the medical image using a trained image registration model, and obtain a medical image registration result, wherein the medical image to be registered includes a moving image and a fixed image; generate a deformation field based on the moving image and the fixed image;

[0010] Among them, the image registration model includes a high-risk deformation area perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, fixed image and deformation field, the high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field, and dynamically adjust the weight of the deformation field to extract fixed image features, moving image features and deformation field features; based on the fixed image features, moving image features and deformation field features, the three-view hierarchical feature fusion module is adopted to obtain the final fused feature map; based on the final fused feature map, the spatial transformation network is adopted to obtain the final deformation field for registering medical images.

[0011] Furthermore, based on the moving image, the fixed image and the deformation field, a high-risk deformation area perception mechanism module is adopted, the gradient information of the deformation field is introduced, and the weight of the deformation field is dynamically adjusted to extract the fixed image features, the moving image features and the deformation field features; the method includes:

[0012] The fixed image and the moving image are stitched, linearly projected and layer-normalized to obtain the query vector and the key vector. Based on the deformation field, the gradient coefficient of the extracted deformation field is calculated using the Gradient3D network. The gradient coefficient of the deformation field is respectively integrated into the query vector and the key vector to obtain the enhanced query vector and the enhanced key vector. The enhanced query vector and the enhanced key vector are processed by the attention mechanism to obtain the deformation field features.

[0013] The moving image and the fixed image are processed by the attention mechanism to obtain the fixed image features and the moving image features respectively.

[0014] Furthermore, the gradient coefficient of the calculated extracted deformation field is calculated using the following formula:

[0015]

[0016] in, represents the gradient coefficient of the deformation field φ extracted by Gradient3D; are the absolute values ​​of the partial derivatives of the deformation field φ in the x, y, and z directions respectively; max represents the maximum value function.

[0017] Furthermore, the method adopts a three-view hierarchical feature fusion module based on fixed image features, moving image features and deformation field features to obtain a final fused feature map; the method includes:

[0018] The three-view hierarchical feature fusion module includes a four-layer decoder and a THFI module. Each layer of the decoder decodes fixed image features, moving image features and deformation field features, aggregates them through the convolution block of the THFI module, and generates a fused feature map; the generated fused feature map is input into the deformation field estimation module of the THFI module to predict the residual deformation field; the predicted residual deformation field is merged with the deformation field of the previous layer to generate an optimized deformation field; after being processed by the four-layer decoder and the THFI module, the final fused feature map is finally obtained.

[0019] Furthermore, in the image registration model training process, a cross-consistency loop learning module is also included to calculate the deformation field φ from image X to image Y through the registration network. X→Y , apply a deformation field φ to the image X X→Y , get the registered image X'; use the same registration network to calculate the deformation field φ from image Y to image X Y→X ; The deformation field φ Y→X Applied to image Y, the registered image Y′ is obtained; using the inverse deformation field -φ Y→X Deform the registered image X' to obtain a cycle-consistent image X"; use the inverse deformation field -φ X→Y The registered image Y' is deformed to obtain a cycle-consistent image Y"; a comprehensive loss function is constructed to optimize the learning parameters of the image registration model.

[0020] Furthermore, the comprehensive loss function is expressed by the following formula:

[0021] L cross =L sim +αL reg +βL cycle

[0022] L sim =-NCC(X′,Y)-NCC(Y′,X)

[0023]

[0024] L cycle =||X″-X||1+||Y″-Y||1

[0025] Among them, L cross Represents the comprehensive loss function, L sim represents the similarity loss, L reg represents the regularization loss, L cyclerepresents cycle consistency loss, NCC represents normalized cross correlation, and || ||1 represents the L1 norm.

[0026] A second aspect of the present invention provides a medical image registration system.

[0027] A medical image registration system, comprising:

[0028] A prediction module is configured to: obtain a medical image to be registered, register the medical image using a trained image registration model, and obtain a medical image registration result, wherein the medical image to be registered includes a moving image and a fixed image; and generate a deformation field based on the moving image and the fixed image;

[0029] Among them, the image registration model includes a high-risk deformation area perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, fixed image and deformation field, the high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field, and dynamically adjust the weight of the deformation field to extract fixed image features, moving image features and deformation field features; based on the fixed image features, moving image features and deformation field features, the three-view hierarchical feature fusion module is adopted to obtain the final fused feature map; based on the final fused feature map, the spatial transformation network is adopted to obtain the final deformation field for registering medical images.

[0030] A third aspect of the present invention provides a computer-readable storage medium.

[0031] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the medical image registration method as described in the first aspect above.

[0032] A fourth aspect of the present invention provides a computer device.

[0033] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the medical image registration method as described in the first aspect above are implemented.

[0034] A fifth aspect of the present invention provides a computer program product or a computer program.

[0035] The present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the medical image registration method described in the first aspect above.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] This paper proposes an unsupervised deformation-aware adaptive learning framework (UDA) 2 L). By introducing the high risk deformation area perception mechanism (HDA) 2 M), dynamically captures salient features in the deformed region and combines it with a multi-level feature integration module (THFI) to fuse local details with global context information to effectively improve the performance of the registration model in complex anatomical scenes.

[0038] The present invention constructs a cross consistency constraint (C 3 ), further improving the model’s ability to maintain topological consistency and ensuring that the deformation field generated during the registration process is both anatomically reasonable and able to meet the diverse needs of complex clinical scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0040] Figure 1 is a flow chart of a medical image registration method shown in the present invention;

[0041] Figure 2 The high risk deformation area perception mechanism (HDA) shown in the present invention 2 M) Network framework diagram;

[0042] Figure 3 is a structural diagram of the first three decoding layers in the three-view hierarchical feature fusion module (THFI) shown in the present invention;

[0043] Figure 4 is a structural diagram of the last decoding layer in the three-view hierarchical feature fusion module (THFI) shown in the present invention;

[0044] Figure 5 It is a comparison diagram of the registration on the IXI data set shown in the present invention;

[0045] Figure 6 The unsupervised deformation-aware adaptive learning framework (UDA) shown in the present invention 2 L) framework diagram. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0047] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0049] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0050] Embodiment 1

[0051] like Figure 1As shown, this embodiment provides a medical image registration method. This embodiment uses the method applied to a server as an example for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a terminal, a server, and a system, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0052] Acquire a medical image to be registered, register the medical image using a trained image registration model, and obtain a medical image registration result, wherein the medical image to be registered includes a moving image and a fixed image; generate a deformation field based on the moving image and the fixed image;

[0053] Among them, the image registration model includes a high-risk deformation area perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, fixed image and deformation field, the high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field, and dynamically adjust the weight of the deformation field to extract fixed image features, moving image features and deformation field features; based on the fixed image features, moving image features and deformation field features, the three-view hierarchical feature fusion module is adopted to obtain the final fused feature map; based on the final fused feature map, the spatial transformation network is adopted to obtain the final deformation field for registering medical images.

[0054] The present invention provides an unsupervised deformation-aware adaptive learning framework (UDA) for medical image registration. 2 L). The framework adopts the high risk deformation area perception mechanism (HDA) 2 M), three-view hierarchical feature fusion module (THFI) and cross-consistency loop learning (C 3 ) modules are used to improve the registration accuracy of medical images, maintain topological consistency, and enhance the robustness of the model. The following is a detailed description of each module:

[0055] 1. High-risk deformation area perception mechanism (HDA) 2 M)

[0056] Medical images usually contain many complex anatomical structures. In addition, in these key areas, the gradient of the deformation field fluctuates greatly, which significantly increases the difficulty of registration. To solve these problems, this paper proposes a high-risk deformation area perception mechanism (HDA) 2 M, such as Figure 2 HDA 2 M introduces the gradient information of the deformation field based on the multi-head local attention mechanism and dynamically adjusts the weight of the deformation field so that high-risk and highly deformed areas obtain higher attention weights, thereby improving the accuracy of feature extraction and registration.

[0057] Assume there is a fixed image F, a moving image M and a matching 3D deformation field Where H, W, L represent height, width, and length respectively, and 3 represents the x, y, and z directions in three-dimensional space.

[0058] In HDA 2 In M, the image patch formed by connecting features F and M (see Figure 2 ), linear projection (LP) and layer normalization (LN) are processed to generate query (Q) and key (K) vectors, the formula of which is as follows:

[0059] Q=LN(LP(concat(F,M))) (1)

[0060] K=LN(LP(concat(F,M))) (2)

[0061] Among them, concat() represents feature concatenation. Can be decomposed into HDA 2 M's multiple attention heads, namely {Q 1 ,Q 2 ,…,Q S} and {K 1 ,K 2 ,…,K S}, where S represents the number of heads of the attention module and c represents the number of channels.

[0062] At the same time, the Gradient3D operation combines a convolutional neural network (CNN) and a differential operator to extract the gradient map of the deformation field. This operation calculates the absolute value of the gradient map and normalizes it, and each partial derivative is approximated by convolution with the Sobel operator. The gradient calculation formula for the deformation field is as follows:

[0063]

[0064] in, represents the gradient coefficient of the deformation field φ extracted by Gradient3D; are the absolute values ​​of the partial derivatives of the deformation field φ in the x, y, and z directions respectively; max represents the maximum value function.

[0065] Subsequently, the gradient information of the deformation field is integrated into the enhanced query vector and the enhanced key vector to strengthen the model's attention to the key deformation area. The specific steps are as follows:

[0066]

[0067] Among them, Q enhanced represents the enhanced query vector, K enhanced Represents the augmented key vector.

[0068] The attention mechanism formula for high-risk deformation areas is:

[0069]

[0070] Among them, Attention() represents the attention mechanism, softmax() represents the softmax function, V represents the corresponding value vector (Value), d k represents the dimensions of the query vector and key vector, Represents the transpose of the augmented key vector.

[0071] Based on the above formula and feature extraction method, HDA 2 M not only considers the feature correlation in the attention mechanism, but also specifically introduces the gradient information of the deformation field, making the attention mechanism more sensitive to the deformation and motion regions contained in the input image, thereby significantly improving the accuracy and robustness of the registration.

[0072] The present invention designs a high risk deformation area perception mechanism (HDA) 2 M), which is used to dynamically adjust the weights of high-risk areas in the feature extraction stage to increase the model's attention to complex anatomical structures and large deformation areas, thereby enhancing the accuracy of the registration results.

[0073] 2. Three-view hierarchical feature fusion module (THFI)

[0074] Many researchers have tried to improve the model's ability to capture complex image details through feature enhancement methods (such as pyramid structure, multi-resolution registration, cascade structure, wavelet transform, and edge information). However, these methods usually fail to fully utilize the features of the image itself, but rely on other means to improve model performance.

[0075] To solve this problem, the present invention proposes a three-view hierarchical feature fusion module (THFI). This module recursively integrates the features of fixed images, moving images, and deformation fields, and optimizes the deformation process layer by layer, so that it can better handle complex anatomical changes and large deformation areas. The THFI module maximizes the complementary information lost during the processing of the network layer, further improving the accuracy and efficiency of the registration.

[0076] The entire decoder contains four decoding layers (l = 4, 3, 2, 1), which predict the deformation field from coarse to fine layer by layer. From the deepest layer l4 to the shallowest layer l1, each layer of the decoder generates a specific deformation field φ l The calculation process is as follows Figure 3 , Figure 4 As shown in Figure 1. First, the features of the fixed image and the moving image are extracted by the decoder layer l, and they are normalized to the same size through downsampling. The deformation field φ predicted by the l+1 layer l+1 The cascaded features used to spatially transform the moving image are used to obtain the distorted moving features. The above feature M′ l It is passed to the concatenation and convolution (CC) block for feature map aggregation to generate a fused feature map C l ,like Figure 3 As shown. Then, the fusion feature map C l is input to the deformation field estimation module (Def) to predict the residual deformation field (e.g. Figure 3 Finally, the deformation fusion operation (DF) combines the newly predicted residual field The deformation field φ of the previous layer l+1 Merge to generate the optimized deformation field φ l When l = 4, this layer does not need to supplement the deformation field from the previous layer, so the THFI structure does not contain distortion features or deformation field fusion components (such as Figure 4 shown).

[0077] The THFI module helps to learn complex deformations by performing necessary feature fusion operations at each decoding layer. In addition, it is able to integrate and propagate high-level semantic information in each decoding layer, thereby expanding the receptive field at a finer resolution level and enabling shallow feature maps to obtain deep semantic information.

[0078] In the 4th layer decoder, the feature map and Is input to the CC block to generate the fused feature map C4:

[0079]

[0080] Among them, F l and M l:The fixed image and the moving image are input to the encoder, and the first layer generates feature maps F1 and M1 after convolution (Conv), instance normalization (InsNorm) and LeakyRelu (LRelu). Except for the first layer, which has no downsampling, the next three layers first downsample the feature maps through convolution with a stride of 2, and enter the residual module after downsampling in each layer). This will generate two sets of feature maps F1, F2, F3, F4 and M1, M2, M3, M4. Represents the fixed image features output in the 4th layer encoder. Represents the moving image features output by the 4th layer encoder. The CC block consists of two sets of convolution + instance normalization (InsNorm) + Leaky ReLU (LRelu) operations. Then, C4 is sent to Def to generate the first residual deformation field of the deepest layer. That is φ1.

[0081] After the initial fusion feature map of the 4th layer is generated, the following operations are performed Figure 3 As shown:

[0082]

[0083] in, represents the deformation field φ obtained previously l+1 The deformation of the moving image is performed by the spatial transformer network (STN).

[0084] Finally, the pyramid decoder generates a semantically rich deformation field φ, which is subsequently used to warp the moving image to align the fixed image.

[0085] In order to solve the problem of information loss in the process of multi-level feature processing, the present invention proposes a three-view hierarchical feature fusion module (THFI), which recursively integrates fixed image, moving image and deformation field features, optimizes the deformation process layer by layer, and thus achieves efficient and accurate image registration.

[0086] 3. Cross-consistency loop learning (C 3 )

[0087] In order to maintain the reversibility and topological consistency of the deformation field when registering single-modality brain magnetic resonance images, the present invention proposes C 3 Learning, which ensures the consistency of forward and reverse deformations and the reversibility of the deformation field.

[0088] C 3We learn to use a shared registration network N, which handles both forward and backward registration tasks during training. Specifically, the parameters of network N are shared between forward and backward registration. When the input is (X, Y), network N computes the deformation field (φ) from the source image X to the target image Y. X→Y ); when the input is (Y,X), the network N calculates the deformation field φ from the target image Y to the source image X Y→X The specific steps are as follows.

[0089] Forward registration: The registration network N is used to calculate the deformation field φ from image X to image Y X→Y :

[0090] φ X→Y =N(X,Y) (11)

[0091] Apply a deformation field φ to the image X X→Y , and get the registered image X':

[0092] X′=θ(X,φ X→Y ) (12)

[0093] Among them, the θ operation represents the deformation field φ X→Y Apply spatial transformation to image X.

[0094] Backward registration: Use the same registration network N to calculate the deformation field φ from image Y to image X Y→X :

[0095] φ Y→X =N(Y,X) (13)

[0096] The deformation field φ Y→X Applied to image Y, we get the registered image Y′:

[0097] Y′=θ(Y,φ Y→X ) (14)

[0098] In order to ensure the reversibility of the deformation field, the present invention establishes a C 3 Constraints are used to ensure the reversibility of the deformation field and the consistency of the topological structure. Y→X Deform the registered image X' to obtain a cycle-consistent image X":

[0099] X″=θ(X′,-φ Y→X ) (15)

[0100] Among them, -φ Y→X Represents the inverse deformation field used in the inverse deformation step.

[0101] Using the inverse deformation field -φ X→YDeform the registered image Y' to obtain a cycle-consistent image Y":

[0102] Y″=θ(Y′,-φ X→Y ) (16)

[0103] Finally, in order to achieve unsupervised learning, the present invention designs a comprehensive loss function L cross . This function is based on the similarity loss L sim , regularization loss L reg and cycle consistency loss L cycle Under the three parts, the correspondence between the registered image and the target image, the smoothness of the deformation field, and the consistency of the cyclic registration process are constrained. These constraints ensure that the model can be effectively learned without the need for supervised labels, thereby achieving accurate and robust registration results.

[0104] L cross =L sim +αL reg +βL cylce (17)

[0105] Among them, α and β represent weight coefficients, which control the relative importance between similarity loss, regularization loss and cycle consistency loss. Adjust these weights according to the experimental results to optimize the training of the model.

[0106] The similarity function uses the normalized cross correlation (NCC) between (X′,Y) and (Y′,X) to reduce sensitivity to contrast changes:

[0107] L sim =-NCC(X′,Y)-NCC(Y′,X) (18)

[0108] Where NCC is:

[0109]

[0110] where Ω is the entire 3D volume, and are the local means of volumes X(v) and Y(v), respectively, v i represents the position of the i-th voxel in the image volume. i ) and Y(v i ) represents image X and image Y at voxel position v i The pixel value of .

[0111] The regularization function penalizes the smoothness of the deformation field to prevent unrealistic deformations. This can be achieved by the L2 criterion of the deformation field gradient φ:

[0112]

[0113] Among them, φ X→Y is the deformation vector field generated by the network N with inputs X and Y, φ Y→X is the deformation vector field obtained by the network N with input Y and X.

[0114] The gap between the initial image and the cycle-consistent image is determined by the following factors:

[0115] L cycle =||X″-X||1+||Y″-Y||1 (21)

[0116] Here, || ||1 represents the L1 norm.

[0117] In short, by introducing C 3 The learning,model can effectively maintain the topological structure and reversibility of the deformation field during,registration.,This innovative approach not only improves the registration accuracy,but also enhances the robustness and stability of the network when,processing complex deformations and single-modality brain MRI data.

[0118] The present invention uses cross-consistency loop learning (C 3 ) mechanism, a shared network structure is constructed to simultaneously complete the forward and reverse registration tasks to ensure the reversibility and topological consistency of the deformation field.

[0119] 4. Dataset introduction and experimental parameter setting

[0120] To evaluate the effectiveness of the proposed model, experiments were performed on three brain MRI datasets: LPBA40 (40 T1W MR images), IXI (576 images), and OASIS (414 images). For LPBA40 and IXI, atlas-to-patient registration was performed, while OASIS focused on obtaining high-precision registration for structures of different shapes and sizes across different patients. All MRI scans were preprocessed using FreeSurfer, including methods such as skull stripping, spatial normalization, affine transformation, and automatic structure segmentation. OASIS and IXI scans were cropped to 160×192×224, and LPBA40 scans were cropped to 160×192×160. Segmentation maps of 56 structures (LPBA40) and 30 structures (IXI) were used for the evaluation.

[0121] To comprehensively evaluate the proposed method, it is compared with seven leading traditional methods and deep learning methods.

[0122] All models were trained for 500 epochs using an Adaptive Moment Estimation (Adam) optimizer with a batch size of 1. The method of the present invention utilized LNCC and MSE losses, where the coefficient of the L2 regularization term was optimized to 4 for the LNCC loss and to 0.04 for the MSE loss. The learning rate for both losses was set to 0.0004.

[0123] The evaluation indicators in the experiment are:

[0124] DSC Similarity Coefficient (DSC): It is used to measure the volume overlap of anatomical segmentation results. The higher the DSC, the better the registration effect. The standard deviation (STD) is used to evaluate the stability.

[0125]

[0126] Among them, A and B are the voxel sets in the fixed image and moving image segmentation, respectively.

[0127] Jacob Matrix: J φ Voxels with (v)≤0 are folding points. The smaller the ratio, the better the texture preservation effect.

[0128]

[0129] The Jacobian matrix represents the gradient of voxel transformation under the deformation field φ. It describes the degree of local spatial distortion in image deformation. Its value reflects the change of voxel shape and volume during deformation. |V| represents the total number of all voxels in the volume. φ (v)≤0}: represents the set of voxels whose Jacob matrix is ​​less than or equal to zero during the deformation process, that is, the voxels that are folded (reverse deformed). The size of this set reflects the number of voxels with topological errors or irreversible deformations that occurred during the entire deformation process.

[0130] Wilcoxon rank-sum test: The Wilcoxon rank-sum test was used to assess the significant differences between the two groups.

[0131] The method of the present invention is compared with other methods.

[0132]

[0133] Among them, X i represents the samples in the first group, n i Indicates the number of samples included in the first group.

[0134] On the LPBA40 dataset (see Table 1), UDA 2 L obtained the highest DSC value, indicating that UDA 2L can effectively learn and capture the basic features and structural information of images, even on small-scale datasets. This effect is mainly attributed to UDA 2 The L model effectively integrates multi-view and multi-level features, thereby ensuring more accurate and robust image registration results. In addition, the network of the present invention has almost no folding in the deformation field (|J φ |≤0 is only 0.1%), which indicates that its deformation process is stable and reliable.

[0135] On the IXI dataset, as shown in Table 1, the proposed method achieves the highest DSC value with the lowest Jacobian determinant value compared to all other methods. This performance is due to the UDA 2 The L model's efficient feature extraction and integration mechanism, as well as its unique training constraints, improve the model performance while ensuring the stability and reversibility of the registration process, which fully demonstrates the advantages of the proposed method in achieving accurate and consistent image registration results.

[0136] On the OASIS dataset, our method achieves the highest DSC value (0.818), indicating that the anatomical structures in the aligned images are most similar to those in the fixed images. This result is mainly attributed to the UDA 2 L’s adaptive learning strategy, which effectively addresses the complexity and variability of the anatomical structures in the OASIS dataset and dynamically adjusts the deformation field according to the specific characteristics of the data, thus achieving excellent performance.

[0137] Figure 5 The visual comparison of different registration methods on the IXI dataset is shown. The first column includes fixed images and moving images. Except for the first column, from left to right are the results of VoxelMorph, CycleMorph, Vit-V-Net, TransMorph, TransMatch, PAN and the method proposed in this invention. The figure shows the warped moving image, the estimated deformation (expressed in RGB), the estimated deformation (expressed in grid), and the absolute difference between the warped moving image and the ground truth fixed image from top to bottom.

[0138] As shown in Table 1, the higher the Dice score, the more accurate the registration result. φ |≤0 indicates the average percentage of folded bodies in the deformation field. The smaller the value, the higher the smoothness. STD(|J φ |) represents the standard deviation of the Jacobian value.

[0139] Table 1 Qualitative comparison of the proposed model with other methods on LPBA40, IXI and OASIS datasets

[0140]

[0141] according to Figure 5 In the second and third rows, the deformations produced by the proposed method are smoother and almost fold less than those produced by the other six deep learning methods. The deformation field grid diagram in the third row shows that the proposed method has fewer intersections than the other six methods. Figure 5 In the last row, the present invention visualizes the absolute difference maps between the distorted image and the fixed image obtained by different methods. The red box shows that the difference map generated by the method proposed in the present invention has the lightest color, the smallest difference, and is closest to the fixed difference map.

[0142] Figure 6 The UDA model of the present invention 2 Overall framework diagram of L. UDA 2 L is based on a shared bidirectional network N that handles both forward and backward registration tasks during training. 2 The encoder of L divides the three inputs (moving image M, fixed image F, and deformation field (obtained by symmetric normalization)) into non-overlapping 3D image blocks. The grayscale values ​​of each block are flattened into a 1×64 vector and transformed into a new feature representation through linear projection. After the linear projection step, the image block is fed into the image block merger, and the merged image block is fed into the feature extraction module. During the feature learning process, HDA 2 M will conduct multi-level identification, weighting and adjustment of high-risk areas to ensure UDA 2 L can adaptively focus on these areas during feature extraction, thereby improving the accuracy and robustness of the registration results. 2 The decoder of L, the present invention proposes a THFI module, which uses a pyramid structure and convolution blocks to perform multi-level recursive extraction, merging and refinement of feature maps obtained from three viewpoints (fixed map, moving map and deformation field), gradually refine the deformation, improve the efficiency of the registration process, and make UDA 2 L can achieve a comprehensive and detailed image registration effect. In order to facilitate the model to learn the topological structure between images and improve the reversibility of the registration process, the present invention performs C between the original image and the deformation field. 3 Learning ensures the reversibility of the registration process and the consistency of the topological structure by enforcing multiple levels of constraints.

[0143] Embodiment 2

[0144] This embodiment provides a medical image registration system.

[0145] A medical image registration system, comprising:

[0146] A prediction module is configured to: obtain a medical image to be registered, register the medical image using a trained image registration model, and obtain a medical image registration result, wherein the medical image to be registered includes a moving image and a fixed image; and generate a deformation field based on the moving image and the fixed image;

[0147] Among them, the image registration model includes a high-risk deformation area perception mechanism module and a three-view hierarchical feature fusion module; based on the moving image, fixed image and deformation field, the high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field, and dynamically adjust the weight of the deformation field to extract fixed image features, moving image features and deformation field features; based on the fixed image features, moving image features and deformation field features, the three-view hierarchical feature fusion module is adopted to obtain the final fused feature map; based on the final fused feature map, the spatial transformation network is adopted to obtain the final deformation field for registering medical images.

[0148] In some embodiments, the prediction module is further configured to: perform splicing, linear projection and layer normalization on the fixed image and the moving image to obtain a query vector and a key vector; based on the deformation field, use the Gradient3D network to calculate the gradient coefficient of the extracted deformation field; integrate the gradient coefficient of the deformation field into the query vector and the key vector respectively to obtain an enhanced query vector and an enhanced key vector; perform attention mechanism processing on the enhanced query vector and the enhanced key vector to obtain deformation field features; perform attention mechanism processing on the moving image and the fixed image to obtain fixed image features and moving image features, respectively.

[0149] In some embodiments, the gradient coefficient of the extracted deformation field is calculated using the following formula:

[0150]

[0151] in, represents the gradient coefficient of the deformation field φ extracted by Gradient3D; are the absolute values ​​of the partial derivatives of the deformation field φ in the x, y, and z directions respectively; max represents the maximum value function.

[0152] In some embodiments, the prediction module is further configured as follows: the three-view hierarchical feature fusion module includes a four-layer decoder and a THFI module, each layer of the decoder decodes the fixed image features, the moving image features and the deformation field features, and aggregates them through the convolution blocks of the THFI module to generate a fused feature map; the generated fused feature map is input into the deformation field estimation module of the THFI module to predict the residual deformation field; the predicted residual deformation field is merged with the deformation field of the previous layer to generate an optimized deformation field; after being processed by the four-layer decoder and the THFI module, the final fused feature map is finally obtained.

[0153] In some embodiments, the system further includes a training module, which is configured to: during the image registration model training process, further include a cross-consistency loop learning module for calculating the deformation field φ from image X to image Y through the registration network X→Y , apply a deformation field φ to the image X X→Y , get the registered image X'; use the same registration network to calculate the deformation field φ from image Y to image X Y→X ; The deformation field φ Y→X Applied to image Y, the registered image Y′ is obtained; using the inverse deformation field -φ Y→X Deform the registered image X' to obtain a cycle-consistent image X"; use the inverse deformation field -φ X→Y The registered image Y' is deformed to obtain a cycle-consistent image Y"; a comprehensive loss function is constructed to optimize the learning parameters of the image registration model.

[0154] In some embodiments, the comprehensive loss function is expressed by the following formula:

[0155] L cross =L sim +αL reg +βL cycle

[0156] L sim =-NCC(X′,Y)-NCC(Y′,X)

[0157]

[0158] L cycle =||X″-X||1+||Y″-Y||1

[0159] Among them, L cross represents the comprehensive loss function, L sim represents the similarity loss, L reg represents the regularization loss, L cycle represents cycle consistency loss, NCC represents normalized cross correlation, and || ||1 represents the L1 norm.

[0160] Embodiment 3

[0161] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the medical image registration method described in the first embodiment are implemented.

[0162] Embodiment 4

[0163] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the medical image registration method described in the first embodiment are implemented.

[0164] Embodiment 5

[0165] This embodiment provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the medical image registration method described in the first embodiment.

[0166] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0167] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0170] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A medical image registration method, characterized in that: include: Acquire a medical image to be registered, register the medical image using a trained image registration model, and obtain a medical image registration result, wherein the medical image to be registered includes a moving image and a fixed image; generate a deformation field based on the moving image and the fixed image; The image registration model includes a high-risk deformation region perception mechanism module and a three-view hierarchical feature fusion module; Based on moving images, fixed images and deformation fields, a high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field, and dynamically adjust the weight of the deformation field to extract fixed image features, moving image features and deformation field features; based on fixed image features, moving image features and deformation field features, a three-view hierarchical feature fusion module is adopted to obtain the final fused feature map; based on the final fused feature map, a spatial transformation network is used to obtain the final deformation field for medical image registration.

2. The medical image registration method according to claim 1, characterized in that: The method adopts a high-risk deformation area perception mechanism module based on a moving image, a fixed image and a deformation field, introduces gradient information of the deformation field, and dynamically adjusts the weight of the deformation field to extract fixed image features, moving image features and deformation field features; the method includes: The fixed image and the moving image are stitched, linearly projected and layer-normalized to obtain the query vector and the key vector. Based on the deformation field, the gradient coefficient of the extracted deformation field is calculated using the Gradient3D network. The gradient coefficient of the deformation field is respectively integrated into the query vector and the key vector to obtain the enhanced query vector and the enhanced key vector. The enhanced query vector and the enhanced key vector are processed by the attention mechanism to obtain the deformation field features. The moving image and the fixed image are processed by the attention mechanism to obtain the fixed image features and the moving image features respectively.

3. The medical image registration method according to claim 1, characterized in that: The gradient coefficient of the deformation field extracted by calculation is calculated using the following formula: in, represents the gradient coefficient of the deformation field φ extracted by Gradient3D; are the absolute values ​​of the partial derivatives of the deformation field φ in the x, y, and z directions respectively; max represents the maximum value function.

4. The medical image registration method according to claim 1, characterized in that: The method adopts a three-view hierarchical feature fusion module based on fixed image features, moving image features and deformation field features to obtain a final fused feature map; the method includes: The three-view hierarchical feature fusion module includes a four-layer decoder and a THFI module. Each layer of the decoder decodes fixed image features, moving image features and deformation field features, aggregates them through the convolution block of the THFI module, and generates a fused feature map; the generated fused feature map is input into the deformation field estimation module of the THFI module to predict the residual deformation field; the predicted residual deformation field is merged with the deformation field of the previous layer to generate an optimized deformation field; after being processed by the four-layer decoder and the THFI module, the final fused feature map is finally obtained.

5. The medical image registration method according to claim 1, characterized in that: In the image registration model training process, a cross-consistency loop learning module is also included to calculate the deformation field φ from image X to image Y through the registration network. X→Y , apply a deformation field φ to the image X X→Y , get the registered image X'; use the same registration network to calculate the deformation field φ from image Y to image X Y→X ; The deformation field φ Y→X Applied to image Y, the registered image Y′ is obtained; using the inverse deformation field -φ Y→X Deform the registered image X' to obtain a cycle-consistent image X"; Using the inverse deformation field -φ X→Y The registered image Y' is deformed to obtain a cycle-consistent image Y"; a comprehensive loss function is constructed to optimize the learning parameters of the image registration model.

6. The medical image registration method according to claim 5, characterized in that: The comprehensive loss function is expressed by the following formula: L cross =L sim +αL reg +βL cycle L sim =-NCC(X′,Y)-NCC(Y,X′) L cycle =||X″-X||1+||Y″-Y||1 Among them, L cross Represents the comprehensive loss function, L sim represents the similarity loss, L reg represents the regularization loss, L cycle represents cycle consistency loss, NCC represents normalized cross correlation, and || ||1 represents the L1 norm.

7. A medical image registration system, characterized in that: include: A prediction module is configured to: obtain a medical image to be registered, register the medical image using a trained image registration model, and obtain a medical image registration result, wherein the medical image to be registered includes a moving image and a fixed image; and generate a deformation field based on the moving image and the fixed image; The image registration model includes a high-risk deformation region perception mechanism module and a three-view hierarchical feature fusion module; Based on moving images, fixed images and deformation fields, a high-risk deformation area perception mechanism module is adopted to introduce the gradient information of the deformation field, and dynamically adjust the weight of the deformation field to extract fixed image features, moving image features and deformation field features; based on fixed image features, moving image features and deformation field features, a three-view hierarchical feature fusion module is adopted to obtain the final fused feature map; based on the final fused feature map, a spatial transformation network is used to obtain the final deformation field for medical image registration.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the medical image registration method according to any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the medical image registration method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in the medical image registration method according to any one of claims 1 to 6 are implemented.

Images