Multi-modal registration method and device based on medical image

Abstract

The invention relates to a multi-modal registration method and device based on medical images, and the method comprises the steps: obtaining a brain registration data set, and dividing the brain registration data set into a training set and a test set; and constructing a registration network based on a UNet network structure. The method comprises the following steps: selecting an input image, inputting the input image into a dual framework, performing bidirectional primary registration on the input image to obtain forward and reverse deformation fields and a primary registration image, and applying the dual two-party exchange deformation field to the primary registration image to perform secondary registration to obtain a secondary registration image. And calling a Laplace operator to extract structural features from the input image, the primary registration image and the secondary registration image, and constraining the registration network through structural similarity measurement. And training an end-to-end unsupervised image registration algorithm through the training set and the test set, and optimizing the algorithm according to loss constraints. And inputting a to-be-detected registration image into the registration network, applying the optimal model weight, outputting a registration result, and ensuring the uniformity and accuracy of the result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and medical imaging technology, and in particular to a multimodal registration method and device based on medical images. Background Art

[0002] Two-dimensional deformable image registration has always been an important research hotspot in the field of medical imaging and is crucial for preoperative planning, intraoperative guidance, disease diagnosis and follow-up. Scene acquisition using different sensors is a common practice in various disciplines, from classic disciplines such as medical imaging and remote sensing to emerging tasks such as autonomous driving. Multimodal sensors allow for the collection of a wide range of physical properties, which in turn results in richer scene representations. For example, in radiation planning, multimodal data (e.g., T1-weighted and T2-weighted images in magnetic resonance imaging (MRI) provide different tissue contrasts, T1 images emphasize the anatomical boundaries of gray and white matter, while T2 images can better highlight the high signal areas of cerebrospinal fluid and lesions) are used for more accurate tumor contours, thereby reducing the risk of damaging healthy tissue during radiotherapy.

[0003] In recent years, deep neural network-based registration techniques have become a powerful standard for large-scale medical image registration. In the training phase, deep registration models use the entire training set as the optimization target to train network parameters instead of using a single image pair. After training, these deep registration models usually take only a few seconds to infer the deformation field of an image pair. VoxelMorph is a basic image registration framework. It was proposed in an article titled An Unsupervised Learning Model for Deformable Medical Image Registration published by MIT in 2018. It was originally aimed at the problem of three-dimensional medical image registration, but it can also be extended to two-dimensional registration scenarios. The core idea is to directly predict the deformation field in an unsupervised learning manner by constructing a fully convolutional neural network (CNN), apply the deformation field to the input image, and output the registered result.

[0004] However, existing image registration methods often ignore the differential homeomorphism characteristics of the deformation field. In medical image registration, the differential homeomorphism characteristics of the deformation field ensure that the geometric structure, differential properties, topological properties and reversibility of the object are maintained during the registration process. The differential homeomorphism transformation can maintain the "smoothness" of the image or anatomical structure, that is, there is no obvious distortion or destruction of the structure during the registration process, thereby ensuring that the registered image can correctly reflect the spatial relationship of the original object. However, ignoring the differential homeomorphism characteristics may lead to discontinuous points or extreme deformations in the deformation field, which will cause the organs or tissues in the image to be stretched, compressed or distorted unnaturally. In brain image registration, if the deformation field does not maintain the differential homeomorphism properties, the boundaries of gray matter and white matter may be overstretched, or even extremely compressed in local areas, making it impossible to accurately align the corresponding areas in the two images. In this way, the shapes of organs and tissues become unnatural, resulting in distorted registration results. The differential homeomorphism transformation ensures the reversibility of the deformation, that is, the deformed image can be restored to its original state through the inverse transformation of the deformation field. In addition, if the deformation field is not diffeomorphic, the transformation may not be reversible. This means that the registered image cannot be restored to its original geometry, or the degree of deformation is too complex to be effectively reverse mapped. This poses a great challenge to subsequent image analysis and cross-modal registration (such as registration of T1 and T2 images), because the structure of the original image cannot be accurately reconstructed, further affecting the accuracy of image analysis.

[0005] In addition, when constraining images, existing registration algorithms usually ignore the important step of extracting structural information, and instead directly use the entire image for constraints. Each part of a medical image has different structural features, and these local differences are particularly evident in complex organs such as the brain and spine. For example, structures such as gray matter, white matter, cerebrospinal fluid, and blood vessels in the brain often need to be accurately registered using local features. When the entire image is used directly for global constraints, the local structural information in the image is ignored, and some small but critical anatomical landmarks or lesion areas may be overlooked. In addition, it may also lead to uneven registration results, that is, some areas have better registration effects, while other areas cannot be accurately aligned. For example, the boundary between white matter and gray matter in the brain has a large contrast difference in T1 and T2 images, and the constraints of the entire image may lead to less accurate registration of these areas, affecting the overall alignment quality of the image. Summary of the invention

[0006] Based on this, it is necessary to provide a multimodal registration method and device based on medical images with high image analysis accuracy and good uniformity of registration results to address the above technical problems.

[0007] The present invention provides a multimodal registration method based on medical images, the method comprising: Acquire a brain registration data set, and divide case images in the brain registration data set into a training set and a test set; Based on the UNet network structure, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection skipping are used alternately in the decoding stage to construct the registration network; Selecting an input image from the brain registration data set and inputting it into the dual architecture, performing a bidirectional primary registration on the input image to obtain a forward and reverse deformation field and a primary registration image, and applying the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; Calling the Laplace operator to extract structural features from the input image, the first registration image, and the second registration image, respectively, and constraining the registration network through a structural similarity metric based on the structural features; The end-to-end unsupervised image registration algorithm is trained by using the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; Using the image to be detected and registered as the input of the registration network, applying the optimal model weight, and outputting the registration result; The brain registration dataset is a BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input image includes a moving image and a fixed image, and the forward and reverse deformation fields are respectively the deformation field from the moving image to the fixed image and the deformation field from the fixed image to the moving image.

[0008] In one embodiment, the step of selecting an input image from the brain registration data set and inputting it into a dual architecture, performing a bidirectional primary registration on the input image to obtain a forward and reverse deformation field and a primary registration image, and applying the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image includes: Calling the basic registration method VoxelMorph to perform a registration from the moving image to the fixed image, and establishing a network architecture for registering from the fixed image to the moving image that is dual to the primary registration image based on the primary registration image, so as to obtain a deformation field for inverse registration; The first registration image is subjected to the deformation field of upper and lower exchange through the duality of differential homeomorphism to obtain the second registration image; The duality of the differential homeomorphism includes push-forward and pull-back. The push-forward is used to describe the local movement direction of the image and the propagation of the image size in the deformation field. The pull-back is used to map the structural features in the image back to the source image to compare the structural features or similarity metrics.

[0009] In one embodiment, calling the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, respectively, and constraining the registration network through a structural similarity measure based on the structural features, includes: The structural similarity between the registered image and the fixed image is calculated based on the structural features, and when the structural similarity reaches a maximum value, a similarity constraint is performed on the registration network. The expression of the similarity constraint is: ; In the formula, is the similarity constraint, is the feature image of the structural feature to be compared, It is the cross-correlation representation of the basic registration method VoxelMorph.

[0010] In one embodiment, the calling of the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, and constraining the registration network through a structural similarity measure based on the structural features, further includes: Acquire the feature image pixel corresponding to the input image and the set of all pixels of the input image, determine a plurality of adjacent pixels around the feature image pixel, and calculate the cross-correlation of the basic registration method VoxelMorph; The cross-correlation expression of the basic registration method VoxelMorph is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, are multiple adjacent pixels around the feature image pixel, and are the average pixel intensities of the structural features to be compared.

[0011] In one embodiment, the calling of the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, and constraining the registration network through a structural similarity measure based on the structural features, further includes: Calculating the smoothing loss of any deformation field of the input image based on the basic registration method VoxelMorph to smooth the deformation field; The smoothing loss expression of the deformation field is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, is the gradient operator, For any deformation field, Represents the norm.

[0012] In one embodiment, the calling of the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, and constraining the registration network through a structural similarity measure based on the structural features, further includes: Calling the Laplace operator to calculate pixels in the input image, the first registered image, and the second registered image by grayscale difference, and determining whether the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other pixels around the central pixel; When the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is increased, and when the grayscale value of the central pixel of the image is lower than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is reduced to extract image structural features.

[0013] In one embodiment, the calling of the Laplace operator extracts structural features from the input image, the first registered image, and the second registered image respectively, and constrains the registration network through a structural similarity measure based on the structural features, and then further includes: The Laplace operator is called to calculate the gradient of the central pixel of the neighborhood in multiple directions, and the multiple gradients calculated are summed to adjust the grayscale of the central pixel of the image according to the gradient calculation result; The convolution kernel is moved row by row on the input image, the value in the convolution kernel and the pixel coincident with the convolution kernel are multiplied and summed, and the result is assigned to the pixel coincident with the center of the convolution kernel to obtain a Laplace result; The Laplace result is an image feature structure as an input of the similarity constraint loss.

[0014] The present invention also provides a multimodal registration device based on medical images, the device comprising: A data preparation module, used for acquiring a brain registration data set, and dividing case images in the brain registration data set into a training set and a test set; The registration network building module is used for the UNet-based network structure, which uses the convolution step size in the encoding layer to halve the spatial dimension to the minimum layer, and alternately uses upsampling, convolution and connection skipping in the decoding stage to construct the registration network; An image registration module is used to select an input image from the brain registration data set and input it into the dual architecture, perform a bidirectional primary registration on the input image to obtain a forward and reverse deformation field and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; A feature extraction module, used for calling the Laplace operator to extract structural features from the input image, the first registration image and the second registration image respectively, and constraining the registration network through a structural similarity metric based on the structural features; A model optimization module, used for training the end-to-end unsupervised image registration algorithm through the training set and the test set, so as to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; An image registration detection module, used to take the registered image to be detected as the input of the registration network, apply the optimal model weight, and output the registration result; The brain registration dataset is a BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input image includes a moving image and a fixed image, and the forward and reverse deformation fields are respectively the deformation field from the moving image to the fixed image and the deformation field from the fixed image to the moving image.

[0015] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the multimodal registration method based on medical images as described in any one of the above.

[0016] The present invention also provides a computer storage medium storing a computer program, wherein when the computer program is executed by a processor, the multimodal registration method based on medical images as described above is implemented.

[0017] The multimodal registration method and device based on medical images obtains a brain registration data set and divides the case images in the brain registration data set into a training set and a test set. Based on the network structure of UNet, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection jump are used alternately in the decoding stage to construct a registration network. After that, an input image is selected from the brain registration data set and input into the dual architecture, and the input image is bidirectionally registered once to obtain the forward and reverse deformation fields and the first registration image, and the dual exchange deformation field is applied to the first registration image for secondary registration to obtain the second registration image. Then, the Laplace operator is called to extract structural features from the input image, the first registration image and the second registration image respectively, and the registration network is constrained by the structural similarity measure based on the structural features. The end-to-end unsupervised image registration algorithm is trained by the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight. Finally, the image to be registered is used as the input of the registration network, and the optimal model weight is applied to output the registration result. This method ensures the differential homeomorphism characteristics of the deformation field through the dual framework and the secondary registration based on the exchange of deformation fields. The push-forward and pull-back operations of the differential homeomorphism duality are used to ensure the internal consistency between the forward and reverse deformation fields in the registration, thereby ensuring the uniformity of the registration results. In addition, the Laplace structure operator is added to extract the structural features of the image that needs to be constrained by structural information, and the similarity constraints are imposed on the structural features, so that more accurate registration results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of the process of the multimodal registration method based on medical images provided by the present invention; Figure 2 A block diagram of the VoxelMorph algorithm of the multimodal registration method based on medical images in a specific embodiment provided by the present invention; Figure 3 A schematic diagram of the overall process of a multimodal registration method based on medical images in a specific embodiment provided by the present invention; Figure 4 A schematic diagram of a registration algorithm flow of a multimodal registration method based on medical images in a specific embodiment provided by the present invention; Figure 5A schematic diagram of the structure of a multimodal registration device based on medical images provided by the present invention; Figure 6 This is a diagram of the internal structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figures 1 to 6 The present invention describes a multimodal registration method and device based on medical images.

[0022] like Figure 1 As shown, in one embodiment, a multimodal registration method based on medical images includes the following steps: Step S110, obtaining a brain registration data set, and dividing the case images in the brain registration data set into a training set and a test set.

[0023] Specifically, the server obtains a pre-configured brain registration dataset, which is a BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging, and divides the case images in the obtained brain registration dataset into corresponding training sets and test sets according to a preset ratio.

[0024] In step S120, based on the network structure of UNet, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection skipping are used alternately in the decoding stage to construct the registration network.

[0025] Specifically, based on the network structure of UNet, the server uses the convolution step size in the encoding layer to halve the spatial dimension to the minimum layer, and alternately uses upsampling, convolution, and connection jumps in the decoding stage to complete the construction of the registration network.

[0026] Step S130, select an input image from the brain registration data set and input it into the dual architecture, perform bidirectional primary registration on the input image, obtain forward and reverse deformation fields and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image.

[0027] The input image includes a moving image and a fixed image, and the forward and reverse deformation fields are respectively a deformation field for registering the moving image to the fixed image and a deformation field for registering the fixed image to the moving image.

[0028] Specifically, the server selects an input image from the brain registration dataset and inputs it into the dual architecture, performs a bidirectional primary registration on the input image to obtain the forward and reverse deformation fields and the primary registration image, and applies the dual exchange deformation field to the primary registration image for secondary registration to obtain the secondary registration image.

[0029] In some embodiments, the multimodal registration method based on medical images provided by the present invention selects an input image from a brain registration data set and inputs it into a dual architecture, performs a bidirectional primary registration on the input image, obtains a forward and reverse deformation field and a primary registration image, and applies the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image, specifically comprising the following steps: Step S131, calling the basic registration method VoxelMorph to perform a registration from the moving image to the fixed image, and based on the primary registration image, establishing a network architecture for registration from the fixed image to the moving image that is dual to the primary registration image, so as to obtain a deformation field for inverse registration.

[0030] Step S132, through the duality of differential homeomorphism, the primary registration image is transformed into a secondary registration image through the deformation field of upper and lower exchanges.

[0031] Among them, the duality of differential homeomorphism includes push-forward and pull-back. Push-forward is used to describe the local movement direction of the image and the propagation of the image size in the deformation field, and pull-back is used to map the structural features in the image back to the source image to compare structural features or similarity metrics.

[0032] Step S140, calling the Laplace operator to extract structural features from the input image, the first registered image and the second registered image respectively, and constraining the registration network through a structural similarity metric based on the structural features.

[0033] Specifically, the server calls the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, and constrains the registration network through a structural similarity metric based on the extracted structural features.

[0034] In some embodiments, the multimodal registration method based on medical images provided by the present invention calls the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image respectively, and constrains the registration network through a structural similarity measure based on the structural features, specifically comprising the following steps: Step S141, the structural similarity between the registered image and the fixed image is calculated based on the structural features, and when the structural similarity reaches the maximum value, the registration network is subjected to similarity constraints. The expression of the similarity constraints is: ; In the formula, is the similarity constraint, is the feature image of the structural feature to be compared, It is the cross-correlation representation of the basic registration method VoxelMorph.

[0035] In some embodiments, the multimodal registration method based on medical images provided by the present invention calls the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, respectively, and constrains the registration network through a structural similarity metric based on the structural features, and specifically includes the following steps: Step S142, obtaining the feature image pixel corresponding to the input image and the entire pixel set of the input image, and determining a plurality of adjacent pixels around the feature image pixel, and calculating the cross-correlation of the basic registration method VoxelMorph.

[0036] The cross-correlation expression of the basic registration method VoxelMorph is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, are multiple adjacent pixels around the feature image pixel, and are the average pixel intensities of the structural features to be compared.

[0037] In some embodiments, the multimodal registration method based on medical images provided by the present invention calls the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, respectively, and constrains the registration network through a structural similarity metric based on the structural features, and specifically includes the following steps: Step S143, calculating the smoothing loss of any deformation field of the input image based on the basic registration method VoxelMorph, so as to smooth the deformation field.

[0038] The smooth loss expression of the deformation field is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, is the gradient operator, For any deformation field, Represents the norm.

[0039] In some embodiments, the multimodal registration method based on medical images provided by the present invention calls the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image, respectively, and constrains the registration network through a structural similarity metric based on the structural features, and specifically includes the following steps: Step S144, calling the Laplace operator to calculate the pixels in the input image, the first registered image and the second registered image through grayscale difference, and determining whether the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other pixels around the central pixel.

[0040] Step S145, when the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is increased, and when the grayscale value of the central pixel of the image is lower than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is reduced to extract image structural features.

[0041] In some embodiments, the multimodal registration method based on medical images provided by the present invention calls the Laplace operator to extract structural features from the input image, the first registered image, and the second registered image respectively, and constrains the registration network through a structural similarity measure based on the structural features, and then further includes the following steps: Step S210, calling the Laplace operator to calculate the gradient of the central pixel of the neighborhood in multiple directions, and summing the multiple gradients obtained to adjust the grayscale of the central pixel of the image according to the gradient operation result.

[0042] Step S220, the convolution kernel is moved row by row on the input image, the value in the convolution kernel and the pixels overlapping with the convolution kernel are multiplied and summed, and the result is assigned to the pixel overlapping with the center of the convolution kernel to obtain a Laplace result.

[0043] Among them, the Laplace result is the image feature structure as the input of the similarity constraint loss.

[0044] Step S150, training the end-to-end unsupervised image registration algorithm through the training set and the test set, so as to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint and obtain the optimal model weight.

[0045] Specifically, the server trains the end-to-end unsupervised image registration algorithm through the previously constructed training set and test to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint, and obtains the optimal model weights when the loss is lowest and the model training is stable.

[0046] Step S160, taking the image to be detected and registered as the input of the registration network, applying the optimal model weight, and outputting the registration result.

[0047] Specifically, the server uses the image to be detected or the test set as the input of the registration network, combines the application of the optimal model weights, and outputs the final registration result.

[0048] The above-mentioned multimodal registration method based on medical images obtains a brain registration dataset and divides the case images in the brain registration dataset into a training set and a test set. Based on the network structure of UNet, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection jump are used alternately in the decoding stage to construct a registration network. After that, the input image is selected from the brain registration dataset and input into the dual architecture, and the input image is bidirectionally registered once to obtain the forward and reverse deformation fields and the first registration image, and the dual exchange deformation field is applied to the first registration image for secondary registration to obtain the second registration image. Then, the Laplace operator is called to extract structural features from the input image, the first registration image and the second registration image respectively, and the registration network is constrained by the structural similarity measure based on the structural features. The end-to-end unsupervised image registration algorithm is trained by the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight. Finally, the image to be registered is used as the input of the registration network, and the optimal model weight is applied to output the registration result. This method ensures the differential homeomorphism characteristics of the deformation field through the dual framework and the secondary registration based on the exchange of deformation fields. The push-forward and pull-back operations of the differential homeomorphism duality are used to ensure the internal consistency between the forward and reverse deformation fields in the registration, thereby ensuring the uniformity of the registration results. In addition, the Laplace structure operator is added to extract the structural features of the image that needs to be constrained by structural information, and the similarity constraints are imposed on the structural features, so that more accurate registration results can be obtained.

[0049] Combination Figures 2 to 4 As shown, in a specific embodiment, the multimodal registration method based on medical images provided by the present invention is implemented in Python programming under the Pytorch deep learning framework. The computer graphics card is configured as 3090 and the operating system is Ubuntu 20.04. The present invention first completes the network framework design for image registration, then sends the training data to the neural network for training, and saves the tuned network weights. Finally, the image registration test of the medical image is performed using the trained network weights, which specifically includes steps 1 to 5: Step 1, data preparation.

[0050] Specifically, the brain registration dataset used in the experiment is the BraTS dataset, which is the two main modalities of magnetic resonance imaging (MRI), T1 and T2. The T1 modality is anatomical imaging, which highlights normal anatomical structures and provides clear tissue contrast and structural information. The T2 modality is pathological imaging, which highlights lesion areas with high water content and provides high-contrast information related to lesions. The total number of cases in the BraTS dataset is 542, of which the number of training set cases is 285, the number of validation set cases is 66, the number of test set cases is 191, and the image size is 256×256. During the training process, the training set is first subjected to certain data augmentation processing, and then it is sent to the neural network for training.

[0051] Step 2: Construct a registration network.

[0052] Specifically, the registration network is a UNet-like network structure that uses convolution steps in the encoding layer to halve the spatial dimension until the minimum layer is reached. The subsequent encoding operation on the coarse representation of the input image is similar to the image pyramid used in traditional image registration work. Upsampling, convolution, and connection skipping are used alternately in the decoding stage. Successive layers of the decoder operate on finer spatial scales, resulting in accurate anatomical alignment.

[0053] Step 3: Design a multimodal image registration algorithm.

[0054] Specifically, the overall algorithm framework is a dual architecture. The input images are moving images and fixed images. First, a bidirectional primary registration is performed to obtain the deformation field and registration image in the forward direction (registration from moving images to fixed images) and the reverse direction (registration from fixed images to moving images). Then, the deformation field is exchanged between the two parties and applied to the primary registration image for secondary registration to obtain the secondary registration image. Finally, the Laplace operator is used to extract structural features from the moving image, fixed image, primary registration image, and secondary registration image, and the structural similarity metric is used for constraints to optimize the registration network and improve system performance.

[0055] In this embodiment, the differential homeomorphism characteristics of the deformation field are guaranteed by the dual framework and the secondary registration based on the exchange of deformation fields. Based on the algorithm VoxelMorph (registration from the moving image M to the fixed image F), a dual network architecture is established (registration from the fixed image F to the moving image M), and the deformation field of the reverse registration is obtained. Then, the registered image is obtained by exchanging the deformation field up and down to obtain the secondary registered image, that is, the internal consistency between the forward and reverse deformation fields in the registration is guaranteed by the push-forward and pull-back operations of the differential homeomorphism duality.

[0056] The duality of diffeomorphism makes it a core tool for studying geometric transformations, because duality can establish a strict one-to-one correspondence between geometric objects. Push forward describes the mapping of tangent vectors from one coordinate system to another through the deformation field, capturing the transformation of geometric shapes. In image registration, push forward can describe how the local movement direction and size of the image propagate in the deformation field. Pull back describes the pulling of covariates (such as gradients and densities) from the target coordinate system to the original coordinate system to ensure the consistency of the transformed information. In image registration, pull back maps the features (structural information) in the target image back to the source image for feature comparison or similarity measurement. In our algorithm, the push forward and pull back process is completed by exchanging the deformation field for secondary registration.

[0057] Since the mode of the registered image remains unchanged but the structure changes, the registered image from the mobile image to the fixed image should theoretically have the same structure as the fixed image, and the registered image from the fixed image to the mobile image should theoretically have the same structure as the mobile image. Therefore, the registered image from the mobile image to the fixed image is further subjected to the deformation field from the fixed image to the mobile image to obtain the secondary registered image. Similarly, the registered image from the fixed image to the mobile image is subjected to the registered image from the mobile image to the fixed image to obtain the secondary registered image. Finally, the primary registered image and the secondary registered image, which should theoretically have the same structural information, are subjected to structural information similarity constraints to complete the closed loop of the push and pull process.

[0058] In this embodiment, the similarity metric inherits the basic registration algorithm VoxelMorph mentioned above, and is represented by cross-correlation CC, which measures the similarity by directly comparing the relationship between pixel intensities. The core goal of image registration that needs to be optimized is to maximize the structural similarity between the registered image and the fixed image, so the similarity constraint expression is: ; In the formula, is the similarity constraint, is the feature image of the structural feature to be compared, It is the cross-correlation representation of the basic registration method VoxelMorph.

[0059] After that, the feature image pixels corresponding to the input image and the entire pixel set of the input image are obtained, and a plurality of adjacent pixels around the feature image pixels are determined, and the cross-correlation of the basic registration method VoxelMorph is calculated.

[0060] The cross-correlation expression of the basic registration method VoxelMorph is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, are multiple adjacent pixels around the feature image pixel, and are the average pixel intensities of the structural features to be compared.

[0061] In addition to the similarity metric loss, the loss_smooth loss is also inherited from the VoxelMorph algorithm and is used to smooth the deformation field so that it does not overlap as much as possible. Its expression is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, is the gradient operator, For any deformation field, Represents the norm.

[0062] In this embodiment, the Laplace structure operator is added to extract the structural features of the image that needs to be constrained by structural information, and to perform similarity constraints on the structural features. The Laplace operator is a second-order differential operator in n-dimensional Euclidean space. It calculates the pixels in the neighborhood by grayscale difference. The basic process is: determine the grayscale value of the central pixel of the image and the grayscale values ​​of other pixels around it. If the grayscale of the central pixel is higher, the grayscale of the central pixel is increased; otherwise, the grayscale of the central pixel is reduced, thereby realizing the image structural feature extraction operation. In the algorithm implementation process, the Laplace operator calculates the gradient of the central pixel in the eight directions of the neighborhood, and then adds the gradients to determine the relationship between the grayscale of the central pixel and the grayscale of other pixels in the neighborhood. Finally, the pixel grayscale is adjusted according to the result of the gradient operation. The differential form of the Laplace operator is:

[0063] In the formula, is the image pixel, for The gray value at , is the gray value of the pixel in the left and right directions, , is the grayscale value of the pixel in the up and down directions, and the remaining four items are the grayscale values ​​of the four points of the pixel. The corresponding convolution kernel is: ; The convolution kernel is moved row by row on the image, and then the values ​​in the convolution kernel are multiplied by the pixels that coincide with it, and the sum is assigned to the pixels that coincide with the center of the convolution kernel. The pixels in the first and last rows and columns of the image that cannot be operated are assigned zero, and the Laplace operation result is obtained. Finally, the Laplace operation result, that is, the image structural feature, is used as the input of loss_sim loss, so that the structural feature can be constrained and a more accurate registration result can be obtained.

[0064] Step 4: Train the neural network.

[0065] Specifically, this algorithm is an end-to-end unsupervised image registration algorithm. No human intervention is required in the intermediate process. The training and verification data sets are directly input for learning. The algorithm will optimize the network through loss constraints. After the loss gradually decreases and stabilizes, the training will end and the optimal weights will be saved.

[0066] Step 5: Multimodal image registration.

[0067] Specifically, the test set or other images to be registered are input into the network, and the optimal weights retained during the training process are applied to conduct experiments to obtain the final registration results and objective indicators.

[0068] Among them, objective indicators include DICE, MI, PSNR and SSIM, and the comparison algorithms used are VoxelMorph, ASNet, Nemar and TMI. The DICE coefficient is an indicator used to measure the degree of overlap between two sets (usually binary images). The higher the value, the higher the overlap and the better the registration effect. MI mutual information is an indicator that measures the correlation between two random variables from the perspective of information theory, and is particularly commonly used in multimodal image registration. It measures the amount of shared information between two variables. The larger the value, the more shared information. PSNR is an indicator of image quality, which is used to compare the difference between the reconstructed image and the original image. It reflects the quality of the image through the signal-to-noise ratio. The higher the value, the better the image quality. SSIM is an indicator that measures the structural similarity of two images, taking into account the similarity of brightness, contrast and structure. The higher the value, the better the image quality. Unlike PSNR, it is more in line with the perceptual characteristics of the human visual system. Table 1 shows the comparison results of objective indicators (DICE, MI, PSNR and SSIM) of this example (Ours) and four comparison algorithms: VoxelMorph, ASNet, Nemar and TMI: Table 1 As shown in the table, this example has good results in all indicators, and the image quality and registration performance are also good.

[0069] The following is a description of a multimodal registration device based on medical images provided by the present invention. The multimodal registration device based on medical images described below and the multimodal registration method based on medical images described above can be referred to each other.

[0070] like Figure 5 As shown, in one embodiment, a multimodal registration device based on medical images includes a data preparation module 510, a registration network construction module 520, an image registration module 530, a feature extraction module 540, a model optimization module 550 and an image registration detection module 560.

[0071] The data preparation module 510 is used to obtain a brain registration data set and divide the case images in the brain registration data set into a training set and a test set.

[0072] The registration network construction module 520 is used for a UNet-based network structure, which uses a convolution step size in the encoding layer to halve the spatial dimension to the minimum layer, and alternately uses upsampling, convolution, and connection skipping in the decoding stage to construct a registration network.

[0073] The image registration module 530 is used to select an input image from the brain registration data set and input it into the dual architecture, perform a bidirectional primary registration on the input image, obtain forward and reverse deformation fields and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image.

[0074] The feature extraction module 540 is used to call the Laplace operator to extract structural features from the input image, the first registration image and the second registration image respectively, and constrain the registration network through a structural similarity metric based on the structural features.

[0075] The model optimization module 550 is used to train the end-to-end unsupervised image registration algorithm through a training set and a test set, so as to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint and obtain the optimal model weight.

[0076] The image registration module 560 is used to use the image to be detected and registered as the input of the registration network, apply the optimal model weights, and output the registration result.

[0077] The brain registration dataset is the BraTS dataset, which includes anatomical imaging and pathological imaging in magnetic resonance imaging. The input images include moving images and fixed images, and the forward and reverse deformation fields are the deformation fields from moving images to fixed images and the deformation fields from fixed images to moving images, respectively.

[0078] In this embodiment, the multimodal registration device based on medical images provided by the present invention, the image registration module 530 is specifically used for: The basic registration method VoxelMorph is called to perform a registration from the moving image to the fixed image, and based on the once-registered image, a network architecture for registration from the fixed image to the moving image is established, which is dual to the once-registered image, to obtain the deformation field of the inverse registration.

[0079] Through the duality of differential homeomorphism, the first registered image is transformed into a second registered image by exchanging the deformation field up and down.

[0080] Among them, the duality of differential homeomorphism includes push-forward and pull-back. Push-forward is used to describe the local movement direction of the image and the propagation of the image size in the deformation field, and pull-back is used to map the structural features in the image back to the source image to compare structural features or similarity metrics.

[0081] In this embodiment, in the multimodal registration device based on medical images provided by the present invention, the feature extraction module 540 is specifically used for: The structural similarity between the registered image and the fixed image is calculated based on the structural features, and when the structural similarity reaches the maximum value, the registration network is constrained by similarity. The expression of the similarity constraint is: ; In the formula, is the similarity constraint, is the feature image of the structural feature to be compared, It is the cross-correlation representation of the basic registration method VoxelMorph.

[0082] In this embodiment, in the multimodal registration device based on medical images provided by the present invention, the feature extraction module 540 is further used for: The feature image pixels corresponding to the input image and the entire pixel set of the input image are obtained, and multiple adjacent pixels around the feature image pixels are determined, and the cross-correlation of the basic registration method VoxelMorph is calculated.

[0083] The cross-correlation expression of the basic registration method VoxelMorph is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, are multiple adjacent pixels around the feature image pixel, and are the average pixel intensities of the structural features to be compared.

[0084] In this embodiment, in the multimodal registration device based on medical images provided by the present invention, the feature extraction module 540 is further used for: Based on the basic registration method VoxelMorph, the smoothing loss of any deformation field of the input image is calculated to smooth the deformation field.

[0085] The smooth loss expression of the deformation field is: ; In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, is the gradient operator, For any deformation field, Represents the norm.

[0086] In this embodiment, in the multimodal registration device based on medical images provided by the present invention, the feature extraction module 540 is further used for: The Laplace operator is called to calculate the pixels in the input image, the first registered image, and the second registered image through grayscale difference, and to determine whether the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other pixels around the central pixel.

[0087] When the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is increased; when the grayscale value of the central pixel of the image is lower than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is reduced to extract image structural features.

[0088] In this embodiment, the multimodal registration device based on medical images provided by the present invention further includes a Laplace constraint module, which is used to: The Laplace operator is called to calculate the gradients of the central pixel in the neighborhood in multiple directions, and the multiple gradients obtained are added together to adjust the grayscale of the central pixel of the image through the gradient operation results.

[0089] The convolution kernel is moved row by row on the input image, the values ​​in the convolution kernel and the pixels coinciding with the convolution kernel are multiplied and summed, and the result is assigned to the pixels coinciding with the center of the convolution kernel to obtain the Laplacian result.

[0090] Among them, the Laplace result is the image feature structure as the input of the similarity constraint loss.

[0091] Figure 6 An example of a physical structure diagram of an electronic device is shown. The electronic device may be a smart terminal, and its internal structure diagram may be as follows: Figure 6As shown. The electronic device includes a processor, an internal memory and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a multimodal registration method based on medical images is implemented, and the method includes: Acquire a brain registration data set, and divide the case images in the brain registration data set into a training set and a test set; Based on the UNet network structure, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection skipping are used alternately in the decoding stage to construct the registration network; Select an input image from the brain registration dataset and input it into the dual architecture, perform a bidirectional primary registration on the input image, obtain forward and reverse deformation fields and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; The Laplace operator is called to extract structural features from the input image, the first registered image, and the second registered image, and the registration network is constrained by the structural similarity metric based on the structural features; The end-to-end unsupervised image registration algorithm is trained through the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; The image to be detected and registered is used as the input of the registration network, and the optimal model weight is applied to output the registration result; Among them, the brain registration dataset is the BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input images include moving images and fixed images, and the forward and reverse deformation fields are the deformation fields from moving images to fixed images and the deformation fields from fixed images to moving images, respectively.

[0092] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present invention, and does not constitute a limitation on the electronic device to which the scheme of the present invention is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0093] On the other hand, the present invention further provides a computer storage medium storing a computer program, which, when executed by a processor, implements a multimodal registration method based on medical images, the method comprising: Acquire a brain registration data set, and divide the case images in the brain registration data set into a training set and a test set; Based on the UNet network structure, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection skipping are used alternately in the decoding stage to construct the registration network; Select an input image from the brain registration dataset and input it into the dual architecture, perform a bidirectional primary registration on the input image, obtain forward and reverse deformation fields and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; The Laplace operator is called to extract structural features from the input image, the first registered image, and the second registered image, and the registration network is constrained by the structural similarity metric based on the structural features; The end-to-end unsupervised image registration algorithm is trained through the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; The image to be detected and registered is used as the input of the registration network, and the optimal model weight is applied to output the registration result; Among them, the brain registration dataset is the BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input images include moving images and fixed images, and the forward and reverse deformation fields are the deformation fields from moving images to fixed images and the deformation fields from fixed images to moving images, respectively.

[0094] In another aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, a multimodal registration method based on medical images is implemented, the method comprising: Acquire a brain registration data set, and divide the case images in the brain registration data set into a training set and a test set; Based on the UNet network structure, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection skipping are used alternately in the decoding stage to construct the registration network; Select an input image from the brain registration dataset and input it into the dual architecture, perform a bidirectional primary registration on the input image, obtain forward and reverse deformation fields and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; The Laplace operator is called to extract structural features from the input image, the first registered image, and the second registered image, and the registration network is constrained by the structural similarity metric based on the structural features; The end-to-end unsupervised image registration algorithm is trained through the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; The image to be detected and registered is used as the input of the registration network, and the optimal model weight is applied to output the registration result; Among them, the brain registration dataset is the BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input images include moving images and fixed images, and the forward and reverse deformation fields are the deformation fields from moving images to fixed images and the deformation fields from fixed images to moving images, respectively.

[0095] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0096] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0097] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multimodal registration method based on medical images, characterized in that: The method comprises: Acquire a brain registration data set, and divide case images in the brain registration data set into a training set and a test set; Based on the UNet network structure, the convolution step size is used in the encoding layer to halve the spatial dimension to the minimum layer, and upsampling, convolution and connection skipping are used alternately in the decoding stage to construct the registration network; Selecting an input image from the brain registration data set and inputting it into the dual architecture, performing a bidirectional primary registration on the input image to obtain a forward and reverse deformation field and a primary registration image, and applying the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; Calling the Laplace operator to extract structural features from the input image, the first registration image, and the second registration image, respectively, and constraining the registration network through a structural similarity metric based on the structural features; The end-to-end unsupervised image registration algorithm is trained by using the training set and the test set to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; Using the image to be detected and registered as the input of the registration network, applying the optimal model weight, and outputting the registration result; The brain registration dataset is a BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input image includes a moving image and a fixed image, and the forward and reverse deformation fields are respectively the deformation field from the moving image to the fixed image and the deformation field from the fixed image to the moving image.

2. The multimodal registration method based on medical images according to claim 1, characterized in that: The input image is selected from the brain registration data set and input into the dual architecture, the input image is bidirectionally registered once to obtain forward and reverse deformation fields and a primary registration image, and the dual exchange deformation field is applied to the primary registration image for secondary registration to obtain a secondary registration image, including: Calling the basic registration method VoxelMorph to perform a registration from the moving image to the fixed image, and establishing a network architecture for registering from the fixed image to the moving image that is dual to the primary registration image based on the primary registration image, so as to obtain a deformation field for inverse registration; The first registration image is subjected to the deformation field of upper and lower exchange through the duality of differential homeomorphism to obtain the second registration image; The duality of the differential homeomorphism includes push-forward and pull-back. The push-forward is used to describe the local movement direction of the image and the propagation of the image size in the deformation field. The pull-back is used to map the structural features in the image back to the source image to compare the structural features or similarity metrics.

3. The multimodal registration method based on medical images according to claim 1, characterized in that: The calling of the Laplace operator to extract structural features from the input image, the first registration image, and the second registration image respectively, and constraining the registration network through a structural similarity metric based on the structural features, includes: The structural similarity between the registered image and the fixed image is calculated based on the structural features, and when the structural similarity reaches a maximum value, a similarity constraint is performed on the registration network. The expression of the similarity constraint is: ； In the formula, is the similarity constraint, is the feature image of the structural feature to be compared, It is the cross-correlation representation of the basic registration method VoxelMorph.

4. The multimodal registration method based on medical images according to claim 3, characterized in that: The calling of the Laplace operator extracts structural features from the input image, the first registration image, and the second registration image respectively, and constrains the registration network through a structural similarity metric based on the structural features, and further includes: Acquire the feature image pixel corresponding to the input image and the set of all pixels of the input image, determine a plurality of adjacent pixels around the feature image pixel, and calculate the cross-correlation of the basic registration method VoxelMorph; The cross-correlation expression of the basic registration method VoxelMorph is: ； In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, are multiple adjacent pixels around the feature image pixel, and are the average pixel intensities of the structural features to be compared.

5. The multimodal registration method based on medical images according to claim 4, characterized in that: The calling of the Laplace operator extracts structural features from the input image, the first registration image, and the second registration image respectively, and constrains the registration network through a structural similarity metric based on the structural features, and further includes: Calculating the smoothing loss of any deformation field of the input image based on the basic registration method VoxelMorph to smooth the deformation field; The smoothing loss expression of the deformation field is: ； In the formula, is the feature image pixel corresponding to the input image, is the set of all pixels of the input image, is the gradient operator, For any deformation field, Represents the norm.

6. The multimodal registration method based on medical images according to claim 5, characterized in that: The calling of the Laplace operator extracts structural features from the input image, the first registration image, and the second registration image respectively, and constrains the registration network through a structural similarity metric based on the structural features, and further includes: Calling the Laplace operator to calculate pixels in the input image, the first registered image, and the second registered image by grayscale difference, and determining whether the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other pixels around the central pixel; When the grayscale value of the central pixel of the image is higher than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is increased, and when the grayscale value of the central pixel of the image is lower than the grayscale values ​​of other surrounding pixels, the grayscale of the central pixel of the image is reduced to extract image structural features.

7. The multimodal registration method based on medical images according to claim 6, characterized in that: The calling of the Laplace operator extracts structural features from the input image, the first registration image and the second registration image respectively, and constrains the registration network through a structural similarity metric based on the structural features, and then further includes: The Laplace operator is called to calculate the gradient of the central pixel of the neighborhood in multiple directions, and the multiple gradients calculated are summed to adjust the grayscale of the central pixel of the image according to the gradient calculation result; The convolution kernel is moved row by row on the input image, the value in the convolution kernel and the pixel coincident with the convolution kernel are multiplied and summed, and the result is assigned to the pixel coincident with the center of the convolution kernel to obtain a Laplace result; The Laplace result is an image feature structure as an input of the similarity constraint loss.

8. A multimodal registration device based on medical images, characterized in that: The device comprises: A data preparation module, used for acquiring a brain registration data set, and dividing case images in the brain registration data set into a training set and a test set; The registration network building module is used for the UNet-based network structure, which uses the convolution step size in the encoding layer to halve the spatial dimension to the minimum layer, and alternately uses upsampling, convolution and connection skipping in the decoding stage to construct the registration network; An image registration module is used to select an input image from the brain registration data set and input it into the dual architecture, perform a bidirectional primary registration on the input image to obtain a forward and reverse deformation field and a primary registration image, and apply the dual exchange deformation field to the primary registration image for secondary registration to obtain a secondary registration image; A feature extraction module, used for calling the Laplace operator to extract structural features from the input image, the first registration image and the second registration image respectively, and constraining the registration network through a structural similarity metric based on the structural features; A model optimization module, used for training the end-to-end unsupervised image registration algorithm through the training set and the test set, so as to optimize the end-to-end unsupervised image registration algorithm according to the loss constraint to obtain the optimal model weight; An image registration detection module, used to take the registered image to be detected as the input of the registration network, apply the optimal model weight, and output the registration result; The brain registration dataset is a BraTS dataset, including anatomical imaging and pathological imaging in magnetic resonance imaging; the input image includes a moving image and a fixed image, and the forward and reverse deformation fields are respectively the deformation field from the moving image to the fixed image and the deformation field from the fixed image to the moving image.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images