Medical image registration method and device, equipment and storage medium

Abstract

The invention relates to a medical image registration method and device, equipment and a storage medium. The method comprises the following steps: acquiring a reference image and a floating image; performing first registration processing on the reference image and the floating image by adopting a preset first neural network model, and determining a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on multiple groups of first training image pairs, and each group of first training image pair comprises a reference training image, a floating training image and a golden standard nonlinear transformation parameter; the relative deformation between the reference training image and the floating training image in a part of the first training image pair meets a first threshold condition; and determining an image registration parameter between the reference image and the floating image according to the first nonlinear transformation parameter. By adopting the method, the brain image with relatively large deformation can be accurately registered, and the registration precision is improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a medical image registration method, apparatus, device and storage medium. Background Art

[0002] As we all know, the brain is vital to the human body. It controls all organs of the body and dominates all life activities of the human body. The spatial correspondence between brain structure and function is crucial to understanding the working mechanism of the brain, and the brain function map is the bridge connecting brain structure and function. By aligning brain images to the standard brain function map space, the spatial correspondence between brain structure and function can be achieved, which is crucial to understanding the pathogenic mechanism of the brain.

[0003] In the related art, when registering a brain image to a standard brain function map space, the brain image and the brain function map are mostly registered by a grayscale-based nonlinear registration algorithm to obtain a spatial transformation relationship between the two. Then, the brain image is transformed using the spatial transformation relationship to register the brain image to the spatial coordinate system where the brain function map is located.

[0004] However, in the actual registration process, the registration accuracy of the above technology is low because there may be large deformation between the brain image and the brain function map. Summary of the invention

[0005] Based on this, it is necessary to provide a medical image registration method, device, equipment and storage medium that can accurately register brain images with large deformations in order to solve the above technical problems.

[0006] In a first aspect, the present application provides a medical image registration method, the method comprising:

[0007] Get reference image and floating image;

[0008] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition;

[0009] Image registration parameters between the reference image and the floating image are determined according to the first nonlinear transformation parameters.

[0010] In one embodiment, the first registration process of the reference image and the floating image using the preset first neural network model to determine the first nonlinear transformation parameter between the reference image and the floating image includes:

[0011] Performing affine transformation processing on the reference image and the floating image to determine linear transformation parameters between the reference image and the floating image;

[0012] A first registration process is performed on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine the first nonlinear transformation parameters between the reference image and the floating image.

[0013] In one embodiment, the first registration process is performed on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine the first nonlinear transformation parameters between the reference image and the floating image, including:

[0014] Performing transformation processing on the floating image according to the linear transformation parameter to determine a first floating image;

[0015] The reference image and the first floating image are input into a first neural network model for a first registration process to determine a first nonlinear transformation parameter.

[0016] In one embodiment, determining the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters includes:

[0017] The image registration parameters are determined according to the linear transformation parameters and the first nonlinear transformation parameters.

[0018] In one embodiment, the determining of the image registration parameter according to the linear transformation parameter and the first nonlinear transformation parameter includes:

[0019] A preset second neural network model is used to perform a second registration process on the reference image and the floating image to determine a second nonlinear transformation parameter between the reference image and the floating image; the second neural network model is obtained by training based on a plurality of groups of second training image pairs, each group of second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs meets a second threshold condition;

[0020] An image registration parameter is determined according to the linear transformation parameter, the first nonlinear transformation parameter and the second nonlinear transformation parameter.

[0021] In one embodiment, the second registration process of the reference image and the floating image using the preset second neural network model to determine the second nonlinear transformation parameter between the reference image and the floating image includes:

[0022] Performing transformation processing on the first floating image according to the first nonlinear transformation parameter to determine a second floating image;

[0023] The reference image and the second floating image are input into the second neural network model for second registration processing to determine the second nonlinear transformation parameters.

[0024] In one embodiment, the first registration process of the reference image and the floating image using the preset first neural network model to determine the first nonlinear transformation parameter between the reference image and the floating image includes:

[0025] Using a preset first neural network model to perform a first registration process on the reference image and the floating image, and determining initial nonlinear transformation parameters between the reference image and the floating image;

[0026] An interpolation process is performed on the initial nonlinear transformation parameters to determine the first nonlinear transformation parameters.

[0027] In one of the embodiments, after acquiring the reference image and the floating image, the method further includes:

[0028] Obtaining an original reference image and an original floating image;

[0029] Preprocessing the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters;

[0030] Correspondingly, after determining the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters, the method further includes:

[0031] The image registration parameters are post-processed according to the pre-processing parameters to determine the target image registration parameters.

[0032] In a second aspect, the present application further provides a medical image registration device, the device comprising:

[0033] An image acquisition module, used for acquiring a reference image and a floating image;

[0034] A first registration module is used to perform a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition;

[0035] The registration parameter determination module is used to determine the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters.

[0036] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0037] Get reference image and floating image;

[0038] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition;

[0039] Image registration parameters between the reference image and the floating image are determined according to the first nonlinear transformation parameters.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0041] Get reference image and floating image;

[0042] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition;

[0043] Image registration parameters between the reference image and the floating image are determined according to the first nonlinear transformation parameters.

[0044] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Get reference image and floating image;

[0046] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition;

[0047] Image registration parameters between the reference image and the floating image are determined according to the first nonlinear transformation parameters.

[0048] The medical image registration method, device, equipment and storage medium are used to register the acquired reference image and floating image by using a first neural network, and determine the image registration parameters between the reference image and the floating image according to the obtained first nonlinear transformation parameters; wherein the first neural network is obtained by training based on a plurality of reference training images, floating training images and first training image pairs consisting of gold standard nonlinear transformation parameters, and the relative deformation between the reference training image and the floating training image in some of the plurality of first training image pairs satisfies the first threshold condition. Since the neural network for image registration is trained based on the training image pairs whose relative deformation between the reference training image and the floating training image satisfies the first threshold condition, the neural network can be used to accurately register images that have deformations between each other and whose deformations meet the threshold condition, and the registration accuracy can be improved; at the same time, since the neural network can also be trained based on other images that may not have deformations between each other, the neural network can also be used to register normal images that do not have deformations between each other, that is, the registration method in this method has a wide range of applications. At the same time, the neural network model can perform registration in one go without the need for multiple iterative registrations, thus shortening the registration time and improving the efficiency of registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is an internal structure diagram of a computer device in one embodiment;

[0050] Figure 2 is a schematic diagram of a process of a medical image registration method in one embodiment;

[0051] Figure 3 is a schematic flow chart of a medical image registration method in another embodiment;

[0052] Figure 4 is a schematic flow chart of a medical image registration method in another embodiment;

[0053] Figure 5is a schematic flow chart of a medical image registration method in another embodiment;

[0054] Figure 6 is a schematic flow chart of a medical image registration method in another embodiment;

[0055] Figure 7 is a flowchart of a medical image registration method in another embodiment;

[0056] Figure 8 FIG. 4 is a structural block diagram of a medical image registration device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] The medical image registration method provided in the embodiment of the present application can be applied to a computer device, which can be a terminal, and its internal structure diagram can be as shown in Figure 1 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a medical image registration method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0059] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0060] In one embodiment, Figure 2 As shown, a medical image registration method is provided, which is applied to Figure 1Taking the computer device in the example as an example, the method may include the following steps:

[0061] S202, obtaining a reference image and a floating image.

[0062] The reference image and the floating image may be two-dimensional images or three-dimensional images, and of course may also be images of other dimensions.

[0063] The reference image and the floating image can be images of the same part of the same object to be tested acquired under different conditions; of course, they can also be images of the same part of different objects to be tested acquired under different conditions. The different conditions here can include different times, different scanning protocols, different scanning periods, etc.

[0064] As for the method of acquiring the reference image and the floating image, it can be an image obtained by scanning and reconstructing the same part of different objects under different conditions; or it can be a pre-stored reference image and floating image obtained from the cloud or server; or it can be other acquisition methods, which are not specifically limited here.

[0065] For example, if both the reference image and the floating image are images of the brain, the reference image may be, for example, a pre-constructed brain function map, and the floating image may be, for example, a brain image of any subject to be tested.

[0066] S204, performing a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image.

[0067] Among them, the architecture and type of the first neural network model can be set according to actual conditions, and are not specifically limited in this embodiment. For example, an FCN (Fully Convolution Network) model can be used, or an improved model based on the FCN model.

[0068] The above-mentioned first neural network model is obtained by training based on multiple groups of first training image pairs, each group of first training image pairs includes reference training images, floating training images and gold standard nonlinear transformation parameters, and the relative deformation between the reference training images and the floating training images in some of the first training image pairs meets the first threshold condition.

[0069] In addition, the two training images in the above training image pair are both training images for the same sample part. For example, the two training images in the training image pair are both training images for the brain.

[0070] When pre-acquiring each group of training image pairs, a portion of the training image pairs with relatively large deformation between the reference training image and the floating training image may be acquired, and a portion of the training image pairs with relatively small deformation between the reference training image and the floating training image may also be acquired. Here, having large deformation means that the relative deformation between the reference training image and the floating training image is large (for example, the deformation of the reference training image is large relative to that of the floating training image), that is, the relative deformation between the two is large, and the relative deformation here can meet the first threshold condition.

[0071] The first threshold condition here may be a threshold condition for distinguishing images with large deformations. For example, the first threshold condition here may include a deformation threshold range or a deformation threshold. If the deformation between the reference training image and the floating training image is within the deformation threshold range or is greater than the deformation threshold, it indicates that the deformation between the reference training image and the floating training image meets the first threshold condition, that is, the deformation between the reference training image and the floating training image is large. For another example, if the deformation between the reference training image and the floating training image is outside the deformation threshold range or is less than the deformation threshold, it indicates that the deformation between the reference training image and the floating training image does not meet the first threshold condition, that is, the deformation between the reference training image and the floating training image may be small.

[0072] In addition, when each group of training image pairs is obtained, the reference training image and the floating training image can also be registered by a related registration algorithm, etc., to determine the nonlinear transformation parameters between the two, which are recorded as the gold standard nonlinear transformation parameters. The related algorithms here can include algorithms such as feature point matching. Taking the training image as a brain image as an example, due to the huge difference in the size of the ventricles of different human brains, the gold standard nonlinear transformation parameters obtained by simply using the traditional registration method are still not accurate enough. Therefore, in this embodiment, the ventricles of the reference training image and the floating training image are extracted, and encoded into the corresponding training image to obtain enhanced ventricle information, and then the LDDMM (Large Displacement Diffeomorphic Metric Mapping) method is used to register the group of reference training images and the floating training image to obtain the registered training image pair and nonlinear transformation parameters, which are the gold standard nonlinear transformation parameters. For example, the nonlinear transformation parameters here can be deformation fields.

[0073] From the above description, it can be seen that the first neural network model in the embodiment of the present application is obtained by training each group of training image pairs and their corresponding gold standard nonlinear transformation parameters, that is, the first neural network is supervised training, that is, the first neural network model is a supervised neural network model, which can make the alignment result of the neural network model controllable.

[0074] After obtaining the above-mentioned groups of training image pairs and their corresponding gold standard nonlinear transformation parameters, the initial first neural network model can be trained using the groups of training image pairs and their corresponding gold standard nonlinear transformation parameters to obtain a trained first neural network model. At the same time, the trained first neural network model in this embodiment does not use differential homeomorphism, which can easily achieve rapid registration of images with large deformations. And in this embodiment, some loss functions can also be used for training, such as the exponential L1 Loss (absolute value loss function), so that the greater the error, the greater the loss, thereby ensuring the accuracy of the supervised training process and improving the accuracy of the trained first neural network model.

[0075] After obtaining the reference image and the floating image, the reference image and the floating image can be directly or pre-processed and input into the trained first neural network model for registration processing to obtain nonlinear transformation parameters between the reference image and the floating image, which are recorded as first nonlinear transformation parameters. Optionally, the first nonlinear transformation parameters may include a deformation field.

[0076] In addition, the first neural network model can output two nonlinear transformation parameters, including forward nonlinear transformation parameters and inverse nonlinear transformation parameters. The forward nonlinear transformation parameters are mainly used to transform the floating image to the coordinate system where the reference image is located, and the inverse nonlinear transformation parameters are mainly used to transform the reference image to the coordinate system where the floating image is located. It should be noted that the first nonlinear transformation parameter in this embodiment is the forward nonlinear transformation parameter mentioned here.

[0077] S206: Determine image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters.

[0078] In this step, after obtaining the first nonlinear transformation parameter between the reference image and the floating image, the first nonlinear transformation parameter can be directly used as the registration parameter between the reference image and the floating image, that is, the first nonlinear transformation parameter is subsequently used to register the floating image to transform the floating image to the coordinate system where the reference image is located.

[0079] Of course, other transformation parameters between the reference image and the floating image may also be calculated, and then the first nonlinear transformation parameter and the other transformation parameters are used together as image registration parameters between the reference image and the floating image.

[0080] It should be noted that the first nonlinear transformation parameter here can be used to subsequently align the reference image and the floating image with a larger deformation, and of course, it can also be used to align the reference image and the floating image with a smaller deformation or no deformation, thereby achieving rapid registration of image pairs with a larger deformation and rapid registration of image pairs with a smaller deformation or no deformation.

[0081] Furthermore, from the above description, it can be seen that the above-mentioned first neural network model can be trained using reference training images and floating training images and corresponding gold standard nonlinear transformation parameters, that is, the first neural network model is a supervised model, and the relative deformation between the reference training image and the floating training image satisfies the first threshold condition, and the first threshold condition is, for example, a threshold condition with a larger deformation. In this way, the first neural network model is supervisedly trained through gold standard nonlinear transformation parameters and training images whose relative deformation satisfies the first threshold condition, so that the trained first neural network model can have a higher prediction accuracy for the nonlinear transformation parameters between images with larger deformations.

[0082] In the above-mentioned medical image registration method, the reference image and the floating image are registered by using a first neural network, and the image registration parameters between the reference image and the floating image are determined according to the obtained first nonlinear transformation parameters; wherein the first neural network is obtained by training based on a plurality of reference training images, a floating training image and a first training image pair consisting of a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the plurality of first training image pairs satisfies the first threshold condition. Since the neural network for image registration is trained based on the training image pairs whose relative deformation between the reference training image and the floating training image satisfies the first threshold condition, the neural network can be used to accurately register images that are deformed and meet the threshold condition, and the registration accuracy can be improved; at the same time, since the neural network can also be trained based on other images that may not be deformed, the neural network can also be used to register normal images that are not deformed, that is, the registration method in the method has a wide range of applications. At the same time, the neural network model can be used to perform registration at one time without multiple iterative registrations, so the registration time can be shortened and the registration efficiency can be improved.

[0083] The above embodiments mentioned that the first nonlinear transformation parameters can be directly obtained through the first neural network model. In actual situations, the first nonlinear transformation parameters may also be obtained by combining the linear transformation parameters between the reference image and the floating image. The following embodiments illustrate the process of obtaining the first nonlinear transformation parameters by combining the linear transformation parameters and the first neural network model.

[0084] In another embodiment, another medical image registration method is provided. Based on the above embodiment, Figure 3 As shown, the above S204 may include the following steps:

[0085] S302, performing affine transformation processing on the reference image and the floating image to determine linear transformation parameters between the reference image and the floating image.

[0086] In this step, the affine transformation here can use a traditional iterative registration method or a deep learning-based registration method, which is mainly used to obtain the linear transformation parameters between the reference image and the floating image.

[0087] In addition, in order to simplify the registration process, a pre-positioning strategy can be used to adjust the reference image or floating image. For example, the Bounding Box alignment method can be used to adjust the translation amount in the reference image or floating image, or the PCA (Principal Components Analysis) correction method can be used to correct some larger rotation amounts in the reference image or floating image.

[0088] That is, after obtaining the reference image and the floating image, the reference image and the floating image can be registered using an algorithm related to affine transformation to obtain linear transformation parameters between the two. Optionally, the linear transformation parameters here can include a transformation matrix, which can include parameters such as translation amount and rotation angle.

[0089] Furthermore, the above affine transformation can also obtain two linear transformation parameters, including forward linear transformation parameters and inverse linear transformation parameters. The forward linear transformation parameters are mainly used to transform the floating image to the coordinate system where the reference image is located, and the inverse linear transformation parameters are mainly used to transform the reference image to the coordinate system where the floating image is located. One of the two linear transformation parameters can be obtained by matrix inversion operation to obtain the other one.

[0090] S304, performing a first registration process on the reference image and the floating image according to the linear transformation parameters and the first neural network model, and determining a first nonlinear transformation parameter between the reference image and the floating image.

[0091] In this step, after obtaining the linear transformation parameters, the first nonlinear transformation parameters between the reference image and the floating image can be obtained by combining with the first neural network model. As an optional embodiment, the method may include: transforming the floating image according to the linear transformation parameters to determine the first floating image; inputting the reference image and the first floating image into the first neural network model to perform a first registration process to determine the first nonlinear transformation parameters.

[0092] Taking the transformation of the floating image to the coordinate system of the reference image as an example, the floating image can be transformed by using linear transformation parameters to transform the floating image to the coordinate system of the reference image, and obtain a linearly registered floating image, which is recorded as the first floating image. After that, the reference image and the first floating image can be input into the trained first neural network model, and the first neural network model performs registration processing on the two images to obtain the first nonlinear transformation parameters between the two.

[0093] After obtaining the linear transformation parameters and the first nonlinear transformation parameters between the reference image and the floating image, the image registration parameters can be optionally determined according to the linear transformation parameters and the first nonlinear transformation parameters. Here, for example, the linear transformation parameters and the first nonlinear transformation parameters can be combined as image registration parameters, or can be combined with some other transformation parameters to serve as image registration parameters.

[0094] In addition, since the originally obtained reference image and floating image may not be images in a standard position, it is necessary to first correct the reference image and floating image to a standard position. Taking the brain image as an example, the reference image and floating image can be scalped to obtain the reference image and floating image after scalping. Here, scalping can be performed using a deep learning segmentation network or based on a traditional segmentation method. Then, a Bounding Box that can contain the scalp boundary can be solved based on the reference image and floating image after scalping. Based on this Bounding Box, the redundant boundaries of the reference image and floating image after scalping can be padded in the up and down, front and back, left and right directions respectively to obtain the padded reference image and the padded floating image, so as to ensure that there is a certain space for the affine transformation and it will not be transformed into the external area of ​​the image. Then, the padded reference image and the padded floating image can be histogram matched and sampled into the fixed size space required by the first neural network model to obtain the reference image and floating image input into the first neural network model for registration.

[0095] In this embodiment, the reference image and the floating image are affine transformed to obtain the linear transformation parameters between the two, and the first nonlinear transformation parameters between the two are obtained by combining the first neural network model. In this way, the first nonlinear transformation parameters are obtained by combining the linear transformation parameters between the reference image and the floating image, and more information is used, which can make the obtained first nonlinear transformation parameters more accurate. In addition, the floating image can be transformed by the linear transformation parameters first and then input into the neural network model for registration to obtain the first nonlinear transformation parameters. The registration method of the image registration parameters divided into two levels here can simplify the complexity of obtaining the image registration parameters and improve the efficiency and accuracy of obtaining the image registration parameters. Further, the image registration parameters are determined by the linear transformation parameters and the first nonlinear transformation parameters. The parameters in the image registration parameters determined in this way are more abundant, so that different image registration requirements can be met. At the same time, the registration can be performed once through the neural network model without multiple iterative registrations, so the registration time can be shortened and the registration efficiency can be improved.

[0096] The above embodiment mentioned that the image registration parameters can be obtained by combining the linear transformation parameters and the first nonlinear transformation parameters with other transformation parameters. The following embodiment will illustrate this process.

[0097] In another embodiment, another medical image registration method is provided. Based on the above embodiment, Figure 4 As shown, the above “determining the image registration parameter according to the linear transformation parameter and the first nonlinear transformation parameter” may include the following steps:

[0098] S402, performing a second registration process on the reference image and the floating image using a preset second neural network model to determine a second nonlinear transformation parameter between the reference image and the floating image.

[0099] In this step, the architecture and type of the second neural network model can also be set according to actual conditions, without specific limitation here. For example, similar to the first neural network model, an FCN (Fully Convolution Network) model can also be used, or an improved model based on the FCN model, etc.

[0100] For the second neural network model in this embodiment, it can be obtained by training based on multiple groups of second training image pairs, each group of second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs satisfies the second threshold condition.

[0101] It can be an unsupervised model or a weakly supervised model. If the second neural network model is an unsupervised model, the training process can be to use reference training images and floating training images, and directly train in an unsupervised manner to obtain a trained unsupervised model.

[0102] In the case where the second neural network model is a weakly supervised model, for example, it can be trained with the assistance of reference training images, floating training images, reference labels corresponding to the reference training images (for example, mask images of tissues or blood vessels segmented from the reference training images), and floating labels corresponding to the floating training images (for example, mask images of tissues or blood vessels segmented from the floating training images), so as to obtain a trained weakly supervised model.

[0103] In addition, in the process of training the second neural network model, more loss functions and additional regularization terms can be used for training, and the regularization terms here can include, for example, reversible terms, Jacobian terms, smoothing terms, etc. Training with more loss functions can ensure that the trained second neural network model is more accurate, and adding regularization terms can avoid the problem of overfitting during the training of the second neural network model.

[0104] For example, for weakly supervised training, reference training images and reference labels as well as floating training images and floating labels can be input into the second neural network model, and a series of low-resolution forward velocity field and reverse velocity field predictions, differential homeomorphism operations, upsampling, and deformation field assembly processes are performed in the second neural network model, and a loss function is designed to achieve training of the second neural network model.

[0105] From the above description, it can be seen that no matter whether the second neural network model is a weakly supervised model or an unsupervised model, the second training image pairs consisting of reference training images and floating training images will be used for training during its training process, and the relative deformation between the reference training images and the floating training images in some of these second training image pairs meets the second threshold condition.

[0106] The second threshold condition here may be a threshold condition for distinguishing images with smaller deformations. For example, the second threshold condition here may include a deformation threshold range or a deformation threshold. If the relative deformation between the reference training image and the floating training image is within the deformation threshold range or is less than the deformation threshold, it indicates that the relative deformation between the reference training image and the floating training image meets the second threshold condition, that is, the deformation between the reference training image and the floating training image is smaller. For another example, if the relative deformation between the reference training image and the floating training image is outside the deformation threshold range or is greater than the deformation threshold, it indicates that the relative deformation between the reference training image and the floating training image does not meet the second threshold condition, that is, the relative deformation between the reference training image and the floating training image may be larger.

[0107] After the second neural network model is obtained through the above training, the second neural network model can be used for registration processing. Optionally, the first floating image can be transformed according to the first nonlinear transformation parameters to determine the second floating image; the reference image and the second floating image can be input into the second neural network model for second registration processing to determine the second nonlinear transformation parameters.

[0108] Taking the transformation of a floating image to the coordinate system of a reference image as an example, the first nonlinear transformation parameters obtained above can be used to perform nonlinear transformation processing on the first floating image that has undergone linear transformation, to obtain a transformed floating image, which is recorded as a second floating image; then, both the reference image and the second floating image can be input into a trained second neural network model for registration processing, and in the second neural network model, the final nonlinear transformation parameters can be obtained by differential homeomorphism, which are recorded as second nonlinear transformation parameters; for example, the second nonlinear transformation parameters here can also include a deformation field.

[0109] In addition, as mentioned above, the second neural network model can output two nonlinear transformation parameters, including forward nonlinear transformation parameters and inverse nonlinear transformation parameters. The forward nonlinear transformation parameters are mainly used to transform the floating image to the coordinate system where the reference image is located, and the inverse nonlinear transformation parameters are mainly used to transform the reference image to the coordinate system where the floating image is located. It should be noted that the second nonlinear transformation parameters in this embodiment are the forward nonlinear transformation parameters mentioned here.

[0110] Furthermore, from the above description, it can be seen that the above-mentioned second neural network model can be an unsupervised model or a weakly supervised model. During training, the second neural network model can be trained using reference training images and floating training images whose relative deformation satisfies the second threshold condition, without adding gold standard nonlinear transformation parameters for training. The second threshold condition is, for example, a threshold condition with a smaller deformation. In this way, the second neural network model trained can better adapt to the prediction of nonlinear transformation parameters between images with smaller deformations, and the training of the second neural network model will also be easier, that is, the training difficulty of the second neural network model can be reduced and the training speed can be improved.

[0111] S404: Determine image registration parameters according to the linear transformation parameters, the first nonlinear transformation parameters, and the second nonlinear transformation parameters.

[0112] In this step, after the linear transformation parameter, the first nonlinear transformation parameter and the second nonlinear transformation parameter are obtained, the three can be combined according to their registration order as image registration parameters.

[0113] When the reference image and the floating image are subsequently registered, the image registration parameters can be retrieved, and at least one of the registration parameters can be used to register the reference image and the floating image. If multiple registration parameters are selected for registration, the reference image and the floating image can be registered according to the registration order of the multiple registration parameters in the above combination process. For example, if the first nonlinear transformation parameter and the second nonlinear transformation parameter are selected for forward transformation, the first nonlinear transformation parameter can be used to register the floating image first to obtain the transformed floating image, and then the second nonlinear transformation parameter can be used to register the transformed floating image to obtain the final transformed floating image.

[0114] In this embodiment, the reference image and the floating image are registered by using a second neural network model to obtain a second nonlinear transformation parameter, and the image registration parameter is obtained by combining the linear transformation parameter and the first nonlinear transformation parameter. In this way, the second nonlinear transformation parameter with a smaller deformation is obtained by another neural network model, so that the image registration parameter finally obtained can be applied to the registration of various deformed and normal images, thereby improving the application scope of image registration. Furthermore, the image registered by using the registration parameter with a larger deformation can be input into the second neural network model for registration, which can reduce the difficulty of the secondary registration and improve the accuracy of the secondary registration.

[0115] In the process of using the first neural network model or the second neural network model to align the reference image and the floating image to obtain corresponding nonlinear transformation parameters, since the accuracy requirements of the nonlinear transformation parameters finally obtained are generally high, directly obtaining high-precision nonlinear transformation parameters will result in a large overhead of the computer device's video memory and a reduction in the model training speed. Based on this, this embodiment proposes a solution, and the following embodiment illustrates the solution.

[0116] In another embodiment, another medical image registration method is provided. Based on the above embodiment, taking the first neural network model as an example, Figure 5 As shown, the above S204 may include:

[0117] S502, using a preset first neural network model to perform a first registration process on the reference image and the floating image, and determining initial nonlinear transformation parameters between the reference image and the floating image.

[0118] In this step, the reference image and the floating image can be directly input into the first neural network model for registration, or the reference image and the first floating image after linear transformation can be input into the first neural network model for registration. In short, the two input images can be registered through the first neural network model to obtain the nonlinear transformation parameters between the reference image and the floating image, which are recorded as the initial nonlinear transformation parameters.

[0119] The initial nonlinear transformation parameters are generally low-resolution nonlinear transformation parameters, which can reduce the overhead of the computer device's video memory. At the same time, using lower-resolution nonlinear transformation parameters to train the first neural network model can improve the model training speed.

[0120] S504: Perform interpolation processing on the initial nonlinear transformation parameters to determine first nonlinear transformation parameters.

[0121] In this step, since the nonlinear transformation parameters that are ultimately required are generally nonlinear transformation parameters with higher resolution, the nonlinear transformation parameters with lower resolution can be interpolated to convert the nonlinear transformation parameters with lower resolution into nonlinear transformation parameters with higher resolution to obtain the first nonlinear transformation parameters. In other words, the resolution of the first nonlinear transformation parameters is higher than the resolution of the initial nonlinear transformation parameters, so that the registration accuracy of the subsequent image registration using the first nonlinear transformation parameters can be guaranteed.

[0122] The above is a process for obtaining the first nonlinear transformation parameters for the first neural network model. The process for obtaining the second nonlinear transformation parameters for the second neural network model can also refer to the process for obtaining the first nonlinear transformation parameters for the first neural network model. For example, the initial nonlinear transformation parameters with a lower resolution output by the second neural network model can also be interpolated to obtain second nonlinear transformation parameters with a higher resolution.

[0123] In this embodiment, the nonlinear transformation parameters with higher resolution are obtained by interpolating the nonlinear transformation parameters with lower resolution output by the neural network model. This can reduce the video memory overhead of the computer device and improve the training speed of the neural network model.

[0124] Furthermore, in the above-mentioned process of registering the reference image and the floating image, due to the limitation of the computing power of the computer equipment and other reasons, some processing may be performed on the reference image and the floating image, which may also cause the image registration parameters finally obtained to be not the real image registration parameters, affecting the final registration accuracy. The following embodiment describes a way to solve this problem.

[0125] In another embodiment, another medical image registration method is provided. Based on the above embodiment, Figure 6 As shown, the above S202 may include:

[0126] S602, acquiring an original reference image and an original floating image.

[0127] In this step, the original reference image and the original floating image may be originally obtained images, for example, images directly obtained after scanning and reconstructing the same part of different objects to be tested.

[0128] S604, preprocessing the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters.

[0129] In this step, both the original reference image and the original moving image may be preprocessed before registration, and the preprocessing may include, for example, Crop and Resample. When the original reference image and the original floating image are preprocessed, the parameters in the preprocessing process may be recorded accordingly as preprocessing parameters, such as the translation amount of the original reference image and the original floating image, the scaling ratio of the scaling, etc.

[0130] After the original reference image and the original floating image are preprocessed respectively, the preprocessed images can be obtained, which are respectively recorded as the reference image and the floating image. That is to say, the reference image and the floating image can both be preprocessed images.

[0131] Correspondingly, after determining the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameter in S206, the method may further include:

[0132] S606: Post-process the image registration parameters according to the pre-processing parameters to determine the target image registration parameters.

[0133] In this step, in order to ensure the accuracy of subsequent image registration, the above preprocessing operation needs to be compensated to the above obtained image registration parameters to obtain the real image registration parameters corresponding to the original reference image and the original floating image. Then in this step, the above preprocessing parameters can be used to preprocess the obtained image registration parameters, such as reverse translation, reverse scaling, etc., to obtain the final target image registration parameters.

[0134] For example, preprocessing parameters can be used to post-process the linear transformation parameters of the affine transformation, the supervised first nonlinear transformation parameters, and the unsupervised second nonlinear transformation parameters, and these post-processed transformation parameters are used to transform the floating image in turn, and the final image registration effect is better and the image quality is higher.

[0135] In this embodiment, the image registration is post-processed by preprocessing the reference image and the floating image to obtain the preprocessing parameters to obtain the final image registration parameters, so that the final image registration parameters can match the original reference image and the original floating image, that is, more realistic and more in line with the actual situation.

[0136] The following is a detailed embodiment to illustrate the technical solution of the present application. Based on the above embodiment, the image registration parameters including matrix and deformation field are taken as an example. Figure 7 The alignment framework diagram shown in the figure, the method may include the following steps:

[0137] S1, obtaining the original reference image and the original floating image;

[0138] S2, preprocessing the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters;

[0139] S3, performing affine transformation processing on the reference image and the floating image to determine linear transformation parameters between the reference image and the floating image;

[0140] S4, transforming the floating image according to the linear transformation parameter to determine a first floating image;

[0141] S5, inputting the reference image and the first floating image into a first neural network model for a first registration process, and determining an initial first nonlinear transformation parameter; the first neural network model is obtained by training based on a plurality of training image pairs, each training image pair includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some training image pairs meets a first threshold condition;

[0142] S6, performing interpolation processing on the initial first nonlinear transformation parameter to determine the first nonlinear transformation parameter;

[0143] S7, transforming the first floating image according to the first nonlinear transformation parameter to determine a second floating image;

[0144] S8, inputting the reference image and the second floating image into the second neural network model for second registration processing, and determining initial second nonlinear transformation parameters;

[0145] S9, performing interpolation processing on the initial second nonlinear transformation parameters to determine the second nonlinear transformation parameters;

[0146] S10, determining image registration parameters according to the linear transformation parameters, the first nonlinear transformation parameters, and the second nonlinear transformation parameters;

[0147] S11, post-processing the image registration parameters according to the pre-processing parameters to determine the target image registration parameters.

[0148] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a medical image registration device for implementing the medical image registration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more medical image registration device embodiments provided below can refer to the limitations of the medical image registration method above, and will not be repeated here.

[0150] In one embodiment, Figure 8 As shown, a medical image registration device is provided, comprising: an image acquisition module, a first registration module and a registration parameter determination module, wherein:

[0151] An image acquisition module, used for acquiring a reference image and a floating image;

[0152] A first registration module is used to perform a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition;

[0153] The registration parameter determination module is used to determine the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters.

[0154] In another embodiment, another medical image registration device is provided. Based on the above embodiment, the above first registration module may include:

[0155] A linear transformation unit, used for performing affine transformation processing on the reference image and the floating image, and determining a linear transformation parameter between the reference image and the floating image;

[0156] The first registration unit is used to perform a first registration process on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine the first nonlinear transformation parameters between the reference image and the floating image.

[0157] Optionally, the first registration unit may include:

[0158] A transformation subunit, used for performing transformation processing on the floating image according to the linear transformation parameter to determine a first floating image;

[0159] The registration subunit is used to input the reference image and the first floating image into the first neural network model to perform a first registration process and determine the first nonlinear transformation parameters.

[0160] Optionally, the registration parameter determination module is specifically used to determine the image registration parameter according to the linear transformation parameter and the first nonlinear transformation parameter.

[0161] In another embodiment, another medical image registration device is provided. Based on the above embodiment, the registration parameter determination module may include:

[0162] A second registration unit is used to perform a second registration process on the reference image and the floating image using a preset second neural network model to determine a second nonlinear transformation parameter between the reference image and the floating image; the second neural network model is obtained by training based on a plurality of groups of second training image pairs, each group of second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs meets a second threshold condition;

[0163] The registration parameter determination unit is used to determine the image registration parameter according to the linear transformation parameter, the first nonlinear transformation parameter and the second nonlinear transformation parameter.

[0164] Optionally, the second registration unit is specifically used to transform the first floating image according to the first nonlinear transformation parameters to determine the second floating image; input the reference image and the second floating image into the second neural network model to perform a second registration process to determine the second nonlinear transformation parameters.

[0165] In another embodiment, another medical image registration device is provided. Based on the above embodiment, the above first registration module may include:

[0166] An initial registration unit, used to perform a first registration process on the reference image and the floating image using a preset first neural network model, and determine initial nonlinear transformation parameters between the reference image and the floating image;

[0167] The target registration unit is used to perform interpolation processing on the initial nonlinear transformation parameters to determine the first nonlinear transformation parameters.

[0168] In another embodiment, another medical image registration device is provided. Based on the above embodiment, the image acquisition module may include:

[0169] An acquisition unit, used for acquiring an original reference image and an original floating image;

[0170] A preprocessing unit, used for preprocessing the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters;

[0171] Correspondingly, after the registration parameter determination module determines the image registration parameter between the reference image and the floating image according to the first nonlinear transformation parameter, the apparatus may further include:

[0172] The post-processing module is used to post-process the image registration parameters according to the pre-processing parameters to determine the target image registration parameters.

[0173] Each module in the above-mentioned medical image registration device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0174] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0175] Acquire a reference image and a floating image; perform a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on multiple groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition; determine the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameter.

[0176] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0177] Affine transformation is performed on the reference image and the floating image to determine the linear transformation parameters between the reference image and the floating image; first registration is performed on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine the first nonlinear transformation parameters between the reference image and the floating image.

[0178] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0179] The floating image is transformed according to the linear transformation parameters to determine the first floating image; the reference image and the first floating image are input into the first neural network model to perform the first registration process to determine the first nonlinear transformation parameters.

[0180] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0181] The image registration parameters are determined according to the linear transformation parameters and the first nonlinear transformation parameters.

[0182] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0183] A preset second neural network model is used to perform a second registration process on the reference image and the floating image to determine the second nonlinear transformation parameters between the reference image and the floating image; the above-mentioned second neural network model is obtained by training based on multiple groups of second training image pairs, each group of second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs meets the second threshold condition; according to the linear transformation parameters, the first nonlinear transformation parameters and the second nonlinear transformation parameters, the image registration parameters are determined.

[0184] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0185] The first floating image is transformed according to the first nonlinear transformation parameters to determine the second floating image; the reference image and the second floating image are input into the second neural network model for second registration processing to determine the second nonlinear transformation parameters.

[0186] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0187] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine initial nonlinear transformation parameters between the reference image and the floating image; and an interpolation process is performed on the initial nonlinear transformation parameters to determine first nonlinear transformation parameters.

[0188] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0189] Acquire an original reference image and an original floating image; preprocess the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters; post-process the image registration parameters according to the preprocessing parameters to determine the target image registration parameters.

[0190] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0191] Acquire a reference image and a floating image; perform a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on multiple groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition; determine the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameter.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] Affine transformation is performed on the reference image and the floating image to determine the linear transformation parameters between the reference image and the floating image; first registration is performed on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine the first nonlinear transformation parameters between the reference image and the floating image.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] The floating image is transformed according to the linear transformation parameters to determine the first floating image; the reference image and the first floating image are input into the first neural network model to perform the first registration process to determine the first nonlinear transformation parameters.

[0196] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0197] The image registration parameters are determined according to the linear transformation parameters and the first nonlinear transformation parameters.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0199] A preset second neural network model is used to perform a second registration process on the reference image and the floating image to determine the second nonlinear transformation parameters between the reference image and the floating image; the above-mentioned second neural network model is obtained by training based on multiple groups of second training image pairs, each group of second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs meets the second threshold condition; according to the linear transformation parameters, the first nonlinear transformation parameters and the second nonlinear transformation parameters, the image registration parameters are determined.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] The first floating image is transformed according to the first nonlinear transformation parameters to determine the second floating image; the reference image and the second floating image are input into the second neural network model for second registration processing to determine the second nonlinear transformation parameters.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0203] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine initial nonlinear transformation parameters between the reference image and the floating image; and an interpolation process is performed on the initial nonlinear transformation parameters to determine first nonlinear transformation parameters.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0205] Acquire an original reference image and an original floating image; preprocess the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters; post-process the image registration parameters according to the preprocessing parameters to determine the target image registration parameters.

[0206] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0207] Acquire a reference image and a floating image; perform a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on multiple groups of first training image pairs, each group of first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition; determine the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameter.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0209] Affine transformation is performed on the reference image and the floating image to determine the linear transformation parameters between the reference image and the floating image; first registration is performed on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine the first nonlinear transformation parameters between the reference image and the floating image.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0211] The floating image is transformed according to the linear transformation parameters to determine the first floating image; the reference image and the first floating image are input into the first neural network model to perform the first registration process to determine the first nonlinear transformation parameters.

[0212] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0213] The image registration parameters are determined according to the linear transformation parameters and the first nonlinear transformation parameters.

[0214] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0215] A preset second neural network model is used to perform a second registration process on the reference image and the floating image to determine the second nonlinear transformation parameters between the reference image and the floating image; the above-mentioned second neural network model is obtained by training based on multiple groups of second training image pairs, each group of second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs meets the second threshold condition; according to the linear transformation parameters, the first nonlinear transformation parameters and the second nonlinear transformation parameters, the image registration parameters are determined.

[0216] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0217] The first floating image is transformed according to the first nonlinear transformation parameters to determine the second floating image; the reference image and the second floating image are input into the second neural network model for second registration processing to determine the second nonlinear transformation parameters.

[0218] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0219] A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine initial nonlinear transformation parameters between the reference image and the floating image; and an interpolation process is performed on the initial nonlinear transformation parameters to determine first nonlinear transformation parameters.

[0220] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0221] Acquire an original reference image and an original floating image; preprocess the original reference image and the original floating image respectively to obtain a reference image, a floating image and preprocessing parameters; post-process the image registration parameters according to the preprocessing parameters to determine the target image registration parameters.

[0222] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all data fully authorized by all parties.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0224] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A medical image registration method, characterized in that: The method comprises: Get reference image and floating image; A preset first neural network model is used to perform a first registration process on the reference image and the floating image to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of groups of first training image pairs, each group of the first training image pairs includes a reference training image, a floating training image and a gold standard nonlinear transformation parameter, and the relative deformation between the reference training image and the floating training image in some of the first training image pairs meets a first threshold condition; Image registration parameters between the reference image and the floating image are determined according to the first nonlinear transformation parameters.

2. The method according to claim 1, characterized in that The step of performing a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image includes: Performing affine transformation processing on the reference image and the floating image to determine linear transformation parameters between the reference image and the floating image; A first registration process is performed on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image.

3. The method according to claim 2, characterized in that The step of performing a first registration process on the reference image and the floating image according to the linear transformation parameters and the first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image comprises: Performing transformation processing on the floating image according to the linear transformation parameters to determine a first floating image; The reference image and the first floating image are input into the first neural network model to perform a first registration process, and the first nonlinear transformation parameter is determined.

4. The method according to claim 3, characterized in that Determining the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters includes: The image registration parameters are determined according to the linear transformation parameters and the first nonlinear transformation parameters.

5. The method according to claim 4, characterized in that The step of determining the image registration parameter according to the linear transformation parameter and the first nonlinear transformation parameter comprises: A preset second neural network model is used to perform a second registration process on the reference image and the floating image to determine a second nonlinear transformation parameter between the reference image and the floating image; the second neural network model is obtained by training based on a plurality of groups of second training image pairs, each group of the second training image pairs includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the second training image pairs meets a second threshold condition; The image registration parameters are determined according to the linear transformation parameters, the first nonlinear transformation parameters, and the second nonlinear transformation parameters.

6. The method according to claim 5, characterized in that The step of performing a second registration process on the reference image and the floating image using a preset second neural network model to determine a second nonlinear transformation parameter between the reference image and the floating image includes: Performing transformation processing on the first floating image according to the first nonlinear transformation parameter to determine a second floating image; The reference image and the second floating image are input into the second neural network model for second registration processing to determine the second nonlinear transformation parameters.

7. The method according to claim 1, characterized in that The step of performing a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image includes: Using a preset first neural network model to perform a first registration process on the reference image and the floating image, and determining initial nonlinear transformation parameters between the reference image and the floating image; An interpolation process is performed on the initial nonlinear transformation parameters to determine the first nonlinear transformation parameters.

8. The method according to claim 1, characterized in that The obtaining of the reference image and the floating image comprises: Obtaining an original reference image and an original floating image; Preprocessing the original reference image and the original floating image respectively to obtain the reference image, the floating image and preprocessing parameters; Correspondingly, after determining the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters, the method further includes: The image registration parameters are post-processed according to the pre-processing parameters to determine target image registration parameters.

9. A medical image registration device, characterized in that: The device comprises: An image acquisition module, used for acquiring a reference image and a floating image; A first registration module is used to perform a first registration process on the reference image and the floating image using a preset first neural network model to determine a first nonlinear transformation parameter between the reference image and the floating image; the first neural network model is obtained by training based on a plurality of training image pairs, each of which includes a reference training image and a floating training image, and the relative deformation between the reference training image and the floating training image in some of the training image pairs meets a first threshold condition; The registration parameter determination module is used to determine the image registration parameters between the reference image and the floating image according to the first nonlinear transformation parameters.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images