Multimodal medical image registration method and system based on multi-standard fusion

Abstract

The present invention provides a multimodal medical image registration method and system based on multi-standard fusion, which obtains a multimodal image; based on the multimodal image, the structural features of the image are extracted through an image disentanglement strategy, and the deformation field is estimated in the coding domain according to the structural features; based on the multimodal image, the deformation field is directly registered; the estimated deformation field and the deformation field are spliced, and the deformation field is secondary inferred in combination with the features of the source image to obtain the best registration result. The present invention can combine the advantages of multimodal images and image disentanglement structure coding, reduce the mismatch and error generated in the registration process of a single method, and improve the registration effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of image registration, and in particular relates to a multi-modal medical image registration method and system based on multi-standard fusion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Medical image registration is a key technology in image-assisted surgery and auxiliary diagnosis, and has a vital impact on the accuracy of surgery and diagnostic results. Nowadays, medical image registration is still a very challenging task, especially multimodal medical image registration, which is more complicated due to the complex and unknown correspondence between images of different modalities.

[0004] Most of the mainstream deep learning-based medical image registration algorithms are completed using weakly supervised or unsupervised algorithms. The registration method represented by VoxelMorph inputs a pair of image voxels through CNN, outputs a densely mapped deformation field, and applies the predicted deformation field to the floating image through STN to obtain the registered image. Limited by the current situation of small data volume and small or no labels, the image similarity between fixed images and floating images will be used to drive the optimization of deep learning networks. For multimodal medical image registration tasks, due to the large difference in distribution between fixed images and floating images, the correspondence between pixels is not obvious, and more complex similarity measurement methods are often required. This means that the accurate and efficient intensity-based similarity measurement method in the single-modality registration task will fail here. It is particularly important to find a method to evaluate the relationship between multimodal image pairs, which will be the key path to drive the optimization of multimodal registration networks.

[0005] One of the feasible multimodal registration schemes is the cross-modal similarity measurement method, which mainly includes two categories: information theory-based methods and structural feature representation methods. The information theory-based methods mainly include MI, CMI and other measures, which describe the correlation between images by the degree of mutual inclusion of information between them; the structural feature representation methods mainly include SSIM and MIND, etc. Such methods can extract the underlying structural information of the image. Since the structural information is mostly independent of the image modality, such methods can also be used to measure the similarity of multimodal images, that is, as a multimodal registration method. However, information theory methods mostly focus on the global relationship between images, and have poor measurement capabilities for spatial information. Structural feature methods may have different measurement performances in different modalities, making it difficult to generalize to any modality. The above factors will have an adverse effect on the final registration accuracy of the network. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a multimodal medical image registration method and system based on multi-standard fusion. The present invention can combine the advantages of both structural feature similarity measurement and image disentanglement structure coding, reduce the mismatch and error generated in the registration process of a single method, and improve the registration effect.

[0007] According to some embodiments, the present invention adopts the following technical solutions:

[0008] A multimodal medical image registration method based on multi-standard fusion includes the following steps:

[0009] Acquire multimodal images;

[0010] Based on the multimodal image, the structural features of the image are extracted through an image disentanglement strategy, and the deformation field is estimated in the coding domain according to the structural features;

[0011] Based on the multimodal images, the deformation field is directly obtained by direct registration using structural feature similarity measurement;

[0012] The deformation fields inferred under the above two standards are spliced, and the deformation fields are inferred again in combination with the characteristics of the source image to obtain the best registration result.

[0013] As an optional implementation, the structural features of the image are extracted through an image disentanglement strategy, and the specific process of estimating the deformation field in the coding domain based on the structural features includes: extracting corresponding structural feature codes based on the multimodal image through different feature disentanglement encoders, taking the corresponding structural feature codes as input, and using a registration network to output the deformation field and deformed image inferred based on the coding domain information.

[0014] As a further limitation, the mean square error between the structural feature encoding of the deformed image and the fixed image in the multimodal image is used as the similarity measure between the fixed image and the registered image, and the L2 norm of the velocity field gradient is used as a smoothing regularization term.

[0015] As an optional implementation, based on the multimodal image, using structural feature similarity measurement, the specific process of directly performing registration to obtain the deformation field includes using UNet, taking the multimodal image as input, and inferring the deformation field and the deformed image.

[0016] As a further limitation, MIND is used as the similarity measure between multimodal images, and the L2 norm of the velocity field gradient is used as a smoothing regularization term.

[0017] As an optional implementation, the specific process of performing secondary reasoning on the deformation field in combination with the features of the source image includes using a fusion network to perform secondary reasoning on the deformation field, and the fusion network shown includes three convolutions, and the final deformation field is generated by velocity field integration.

[0018] As a further limitation, the fusion network uses MIND as the similarity measure between multimodal images, and the L2 norm of the velocity field gradient is added as a smoothing regularization term.

[0019] A multi-modal medical image registration system based on multi-standard fusion, comprising:

[0020] An image acquisition module, configured to acquire multimodal images;

[0021] The coding domain sub-network module is configured to extract the structural features of the image based on the multimodal image through an image disentanglement strategy, and estimate the deformation field in the coding domain according to the structural features;

[0022] An image domain sub-network module is configured to directly perform registration based on the multimodal image using a structural feature similarity metric to obtain a deformation field;

[0023] The fusion network module is configured to splice the estimated deformation field and the deformation field, perform secondary reasoning on the deformation field in combination with the features of the source image, and obtain the best registration result.

[0024] An electronic device comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above method are completed.

[0025] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps in the above method are completed.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention comprises two registration sub-networks and a fusion network, a coding domain sub-network SN c Working in the feature encoding domain obtained by image disentanglement, the image domain subnetwork SN i It works directly in the multimodal image domain by using cross-modal similarity metrics; the two sub-networks first independently infer the deformation field with the highest similarity of the deformed images based on the input feature encoding and image information. With the assistance of the prior information provided by the two sub-networks, the fusion network performs secondary inference on the deformation field based on the features of the source image, combining the advantages of the sub-methods in the two domains to obtain the best registration result.

[0028] The present invention combines the advantages of multimodal images and image disentanglement structure coding to reduce the mismatch and error generated in the registration process of a single method; at the same time, the source image and the sub-deformation field are used as the input of the fusion network to infer the fused deformation field, which will help to focus on more details.

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 It is a schematic diagram of the overall principle of this embodiment;

[0032] Figure 2 is a schematic diagram of the MUNIT of this embodiment;

[0033] Figure 3 The coding domain sub-network SN of this embodiment c Schematic diagram of the structure;

[0034] Figure 4 The image domain sub-network SN of this embodiment i Schematic diagram of the structure;

[0035] Figure 5 This is a schematic diagram of the fusion network in this embodiment;

[0036] Figure 6 A comparison chart of the results of the method provided in this embodiment and the single domain method. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0039] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0040] In order to break through the limitation that a single method cannot accurately express the similarity relationship of multimodal images, based on the existing technology, the present invention proposes a fusion registration strategy, which combines the advantages of multimodal images and image disentanglement structure coding to reduce the mismatch and error generated in the registration process of a single method; and provides a more comprehensive fusion network structure, which uses the source image and the sub-deformation field as the input of the fusion network to infer the fused deformation field, which will help to pay attention to more details.

[0041] Specific as Figure 1 As shown in the figure, the method consists of two registration sub-networks and a fusion network, a total of three parts: the encoding domain sub-network SN c Working in the feature encoding domain obtained by image disentanglement, the image domain subnetwork SN i It works directly in the multimodal image domain by using cross-modal similarity metrics. The two sub-networks first independently infer the deformation field with the highest similarity of the deformed images based on the input feature encoding and image information. With the assistance of the prior information provided by the two sub-networks, the fusion network performs secondary inference on the deformation field based on the features of the source image, combining the advantages of the sub-methods in the two domains to obtain the best registration result.

[0042] Code Domain Subnetwork SN c The structural features of the image are extracted through the image disentanglement strategy, and then the deformation field φ is estimated in the encoding domain based on this structural feature. code ; At the same time, the image domain sub-network SN i Directly register the multimodal images to obtain the deformation field φimage.

[0043] This embodiment uses MUNIT as the image disentanglement encoder of the encoding subnetwork, and adds MIND loss to constrain better structural feature preservation during the encoding and decoding process. A pair of multimodal images can be decomposed into a content encoding independent of the modality and a style encoding that describes the modality information. Figure 2 As shown, a set of encoders E c and E s The image is decomposed into content and style codes in the form of coding, and the decoder G can restore any combination of content and style codes into an image of specified structure and modality. MUNIT decomposes and recombines the image in the form of image disentanglement to complete feature extraction and image translation tasks. c In the content encoder E c It is used to extract the structural features of the image, which are in the form of high-dimensional encoding and are completely unrelated to the modal information of the image.

[0044] SN c First, the input multimodal image I f and Im De-entangle the encoder by corresponding features and Extract the corresponding feature code C f and C m , the registration network is based on C f and C m As input, output is the deformation field φ inferred based on the encoding domain information code , and the deformed image like Figure 3 Deformed image The deformation field is applied to the floating image I through STN. m Since the coding domain processes complex high-dimensional coding features and the coding domain deformation field has a more complex correspondence than the deformation field inferred from the image, this embodiment uses UNet++ as the registration network structure of the coding domain.

[0045] The encoder and decoder of UNet++ implement skip-layer connections at multiple scales in a dense manner, which will help better semantic similarity preservation and information extraction.

[0046] This embodiment uses a deformed image and a fixed image I f The feature code C w and C f The MSE between the fixed image and the registered image is used as the similarity measure. This successfully simplifies the complex similarity measurement method of multimodal medical image registration into a simple single-modality problem.

[0047] and SN c The difference is that the image domain registration network (i.e., image domain subnetwork) SN i Directly with multimodal image I f and I m As input, the inferred deformation field φ image , and the deformed image like Figure 4 shown.

[0048] This embodiment uses the classic UNet as the registration network architecture in the image domain and MIND as the similarity measurement method between multimodal images.

[0049] In addition, in order to avoid unrealistic or undesirable deformations, this embodiment constructs a differential homeomorphism strategy to achieve topology preservation and transformation reversibility. The registration network first outputs a velocity field v, which is integrated and resampled to generate a deformation field φ. In order to drive the network to generate a smooth deformation field, the velocity field gradient is also added. The L2 norm of is used as a smoothing regularization term. In summary, the loss functions used to optimize the two sub-networks are defined as:

[0050]

[0051]

[0052] In the formula, λ c and λ i SN c and SN i Scaling factor for the smoothing regularization term in .

[0053] Although both code and image domain methods can independently register multimodal images, neither code nor image domain can achieve comprehensive registration. The MIND method measures image similarity by local image descriptors and lacks a global view, while the disentangled content code focuses more on the overall outline.

[0054] Therefore, a fusion network is proposed to perform fusion registration based on the preliminary reasoning of two independent methods. Figure 5 As shown, the multimodal image I f and I m , and two inferred deformation fields φ code and φ image The splicing is used as the input of the fusion network. The fusion network combines the original image information to perform fusion reasoning on the outputs of the two sub-networks. The final deformation field is also generated by the velocity field through the VecInt module integral resampling integral.

[0055] Thanks to MIND's sharp feature extraction capabilities and the large receptive field of feature encoding, fusion registration can focus on all global and local information. The optimization of the fusion network also uses MIND as a similarity metric and the L2 norm of the velocity field gradient as a smoothing regularization term. Its loss function is defined as:

[0056]

[0057] It should be noted that since the MIND descriptor is used to drive the network to extract image structural features, and the original image structural features have been encoded into a high-dimensional space, the MIND loss is used to optimize the SN c Therefore, the method of this embodiment trains the three networks in a relatively independent manner, rather than an end-to-end solution.

[0058] To validate the effectiveness of the proposed method, experiments were conducted on a brain dataset and compared with other state-of-the-art iterative and learning-based methods. MUNIT was pre-trained to perform image disentanglement and image-to-image translation. The registration network was implemented in PyTorch and Python and optimized using the Adam optimizer with a learning rate of 1e-4. The scale factors of the regularization term were set to λ and λ, respectively. c =0.01,λ i =0.1,λ f = 0.1. All experiments were implemented on an Ubuntu platform equipped with an i9-10900x CPU and an RTX 3090 GPU.

[0059] Dataset: BraTS 2019 was selected for the experiment. This database comes from the 2019 Multimodal Brain Tumor Segmentation Challenge and provides 335 sets of multimodal brain MRI images. Each set of images includes four modalities: T1, T1Gd, T2, and T2-flair. The size of each image is 240×240×155, and the resolution is 1mm. 3 The middle 20 slices of each volume in the T1 and T2 modality images were selected to construct the dataset, and each image was center-cropped and resampled to a size of 256×256 and a resolution of 1.28 mm. 2 The image is normalized to [-1, 1]. The dataset is randomly divided into validation set, test set and training set in the ratio of 20:20:295.

[0060] Comparison Methods: The proposed method is compared with several advanced methods based on iteration and deep learning. (1) SyN: SyN is a classic mutual information-based iterative registration method in the ANTs toolkit with excellent registration performance. Multimodal images are unified to the same modality through MUNIT, and two registration methods based on image translation are reproduced. In order to avoid the difference in registration performance caused by different image translation methods, MUNIT is uniformly used instead of CycleGAN to achieve image-to-image translation, which can help quantitatively analyze the performance of the two methods based on image translation and disentanglement. (2) trans_MIND: After translating multimodal images into single-modal images, VoxelMorph images are registered using MIND as the similarity metric. (3) trans_Fusion: The cross-modal translated image and the multimodal image are inferred separately through two sub-networks, and the registered image is fused from the two domains of multimodal image and translated single-modal image. (4) Code: Only the code domain subnetwork is used for registration, that is, the deformation field is inferred based on the feature encoding of the image, and the MSE similarity of the feature encoding of the registered image and the fixed image is optimized. (5) Image: Only the image domain subnetwork is used for registration, that is, MIND is used as the multimodal image similarity metric to directly extract features from the multimodal image input to infer the deformation field.

[0061] Table 1 T1-to-T2 and T2-to-T1 test results

[0062]

[0063] In this section, two registration tasks, T1-to-T2 and T2-to-T1, are designed. Taking T1-to-T2 as an example, since all multimodal image pairs in the dataset are aligned, the aligned original T1 image is distorted as the floating image T1 by random FFD. warp The original aligned T1 image is used as the ground truth for the verification result. To verify the performance of the registration algorithm, the RMSE of the registered image and the ground truth is compared to prove the registration accuracy and the deformation field gradient is used to prove the smoothness of the deformation field. In addition, the average running time of each method is also counted.

[0064] Experimental Results and Analysis: All six methods are experimented on two registration tasks, and the quantitative results are shown in Table 1. Figure 6The registered images, deformation fields and differences with the ground truth of this method and its sub-methods are given. All deep learning methods outperform the MI-based iterative registration method SyN. The direct prediction method Image performs best in terms of running time, while the disentanglement method performs poorly due to the complexity of the scheme. The methods trans_MIND and Code, which unify two images into the same domain, are compared with the cross-modal similarity metric method Image. The results show that Image performs best in terms of registration accuracy, followed by Code, and trans_MIND performs worst. The results of the above three methods show that the registration strategy of unifying images into the same domain will cause information loss. However, the disentanglement method bypasses the decoding stage, resulting in better information retention and registration effects.

[0065] like Figure 6 The left column is the input and ground truth images; the three columns on the right are the registered image, the corresponding deformation field, and the difference between the registered image and the ground truth.

[0066] Compared with the Image and Code sub-methods, the method proposed in this embodiment achieves the best registration accuracy and better smoothness than Image. i The deformation field of SN is more sharp, which shows that SN using MIND as the similarity metric i It has a keener insight into the structural information and local details of the image. In contrast, the encoding domain sub-network SN c It shows more attention to the global information of the image, and its deformation field is smoother. c The output of SN provides more global information. i Output a deformation field based on structural information. Based on the above two perspectives, the fusion network can obtain more details from the source image more comprehensively, thereby obtaining better registration results. Regarding the smoothness of the deformation field, we believe that the fusion result is within a reasonable range, while the sub-method focuses too much on a single aspect.

[0067] It is observed that the two fusion methods, trans_Fusion and our proposed method, have better registration performance compared with the single direction method. This proves that the methods working in different fields have different focuses, and the fusion scheme can learn more details from each field. Due to the better information retention and feature description capabilities of the disentangled content code, the performance of this method is better than trans_Fusion. The participation of the original image in the fusion also helps the model extract more key information.

[0068] In summary, the unsupervised multimodal registration method provided in this embodiment establishes two sub-networks based on the multimodal attitude metric method and the disentanglement method, and uses the image pairs output by the sub-network as input to train the fusion network. By using the global and local information of the sub-network, the fusion network can obtain a better fusion effect. The experimental results show that this method can combine the cross-modal similarity metric and the disentanglement method, which is more superior to the most advanced iterative and learning-based methods.

[0069] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0073] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A multimodal medical image registration method based on multi-standard fusion, characterized in that: The following steps are involved: Acquire multimodal images; Based on the multimodal image, the structural features of the image are extracted through an image disentanglement strategy, and the deformation field is estimated in the coding domain according to the structural features; Based on the multimodal images, the deformation field is directly obtained by direct registration using structural feature similarity measurement; Specifically, the coding domain subnetwork SN c First, the input multimodal image I f and I m De-entangle the encoder by corresponding features and Extract the corresponding feature code c f and c m , the registration network is based on c f and c m As input, output is the deformation field φ inferred based on the encoding domain information code , and the deformed image The deformed image is obtained by applying the deformation field to the floating image through STN, and UNet++ is used as the registration network structure of the encoding domain; Using Warped Images and a fixed image I f The feature code c w and c f The MSE between the fixed image and the registered image is used as the similarity measure; the image domain subnetwork SN i Directly with multimodal image I f and I m As input, the inferred deformation field φ image , and the deformed image A differential homeomorphism strategy is constructed to achieve topology preservation and transformation reversibility. The registration network first outputs a velocity field v, which is integrated and resampled to generate a deformation field φ. The L2 norm of the velocity field gradient is used as a smoothing regularization term to optimize the loss functions of the two sub-networks, which are defined as: The λ in the formula c and λ i They are SN c and SN i The scale factor of the smooth regularization term in the image is used; the deformation fields inferred under the above two standards are spliced, and the deformation fields are inferred twice in combination with the characteristics of the source image to obtain the best registration result; Specifically, the multimodal image I f and I m , and two inferred deformation fields φ code and φ image The concatenation is used as the input of the fusion network; The fusion network combines the original image information to perform fusion reasoning on the outputs of the two sub-networks. The strategy is the same as the differential homeomorphism of the two sub-networks. The final deformation field is generated by integrating the velocity field through the VecInt module and resampling the integral.

2. The multimodal medical image registration method based on multi-standard fusion as claimed in claim 1, characterized in that: The structural features of the image are extracted through the image disentanglement strategy. The specific process of estimating the deformation field in the coding domain based on the structural features includes: extracting the corresponding structural feature codes based on the multimodal image through different feature disentanglement encoders, taking the corresponding structural feature codes as input, and using a registration network to output the deformation field and deformed image inferred based on the coding domain information.

3. The multimodal medical image registration method based on multi-standard fusion as claimed in claim 2, characterized in that: The mean square error between the structural feature encoding of the deformed image and the fixed image in the multimodal image is used as the similarity measure between the fixed image and the registered image, and the L2 norm of the velocity field gradient is used as a smoothing regularization term.

4. The multimodal medical image registration method based on multi-standard fusion as claimed in claim 1, characterized in that: Based on the multimodal image, the specific process of directly performing registration to obtain the deformation field using structural feature similarity measurement includes using UNet, taking the multimodal image as input, and inferring the deformation field and the deformed image.

5. The multimodal medical image registration method based on multi-standard fusion as claimed in claim 4, characterized in that: MIND is used as the similarity measure between multimodal images, and the L2 norm of the velocity field gradient is used as a smoothing regularization term.

6. The multimodal medical image registration method based on multi-standard fusion as claimed in claim 1, characterized in that: The fusion network uses MIND as the similarity measure between multimodal images, and adds the L2 norm of the velocity field gradient as a smoothing regularization term.

7. A multimodal medical image registration system based on multi-standard fusion, characterized in that: include: An image acquisition module, configured to acquire multimodal images; The coding domain sub-network module is configured to extract the structural features of the image based on the multimodal image through an image disentanglement strategy, and estimate the deformation field in the coding domain according to the structural features; An image domain sub-network module is configured to directly perform registration based on the multimodal image using a structural feature similarity metric to obtain a deformation field; Specifically, the coding domain subnetwork SN c First, the input multimodal image I f and I m De-entangle the encoder by corresponding features and Extract the corresponding feature code c f and c m , the registration network is based on c f and c m As input, output is the deformation field φ inferred based on the encoding domain information code , and the deformed image The deformed image is obtained by applying the deformation field to the floating image through STN, and UNet++ is used as the registration network structure of the encoding domain; Using Warped Images and a fixed image I f The feature code c w and c f The MSE between the fixed image and the registered image is used as the similarity measure; the image domain subnetwork SN i Directly with multimodal image I f and I m As input, the inferred deformation field φ image , and the deformed image A differential homeomorphism strategy is constructed to achieve topology preservation and transformation reversibility. The registration network first outputs a velocity field v, which is integrated and resampled to generate a deformation field φ. The L2 norm of the velocity field gradient is used as a smoothing regularization term to optimize the loss functions of the two sub-networks, which are defined as: The λ in the formula c and λ i They are SN c and SN i The scale factor of the smoothing regularization term in ; A fusion network module is configured to concatenate the estimated deformation field and the deformation field, perform secondary reasoning on the deformation field in combination with the features of the source image, and obtain the best registration result; Specifically, the multimodal image I f and I m , and two inferred deformation fields φ code and φ image The concatenation is used as the input of the fusion network; The fusion network combines the original image information to perform fusion reasoning on the outputs of the two sub-networks. The strategy is the same as the differential homeomorphism of the two sub-networks. The final deformation field is generated by integrating the velocity field through the VecInt module and resampling the integral.

8. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps in the method according to any one of claims 1 to 6 are completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the method according to any one of claims 1 to 6.

Images