Image registration method, device, electronic device and storage medium

Abstract

The embodiments of the present invention relate to an image registration method, device, electronic device and storage medium, and relate to the field of image processing technology. The method acquires a moving image to be registered and a fixed image to be registered; inputs the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of a region of interest; applies the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest; and registers the deformed image of the region of interest with the fixed image to be registered to obtain a registered image; that is, the embodiment of the present invention inputs the moving image to be registered and the fixed image into a model specifically used for region of interest registration for registration, thereby reducing the interference of non-regions of interest in its registration process and improving the image registration effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image registration method, device, electronic equipment and storage medium. Background Art

[0002] Image registration refers to the process of aligning a moving image with a fixed image through rigid, affine or nonlinear transformation, and plays an important role in medical clinical diagnosis.

[0003] Current image registration methods are usually based on deep learning, which obtains the deformation field by learning the correspondence between image pairs. However, they are insensitive to the regions of interest and non-regions of interest in the image, resulting in poor image registration results. Summary of the invention

[0004] The present invention provides an image registration method, device, electronic equipment and storage medium to solve the technical problem of poor image registration effect.

[0005] In a first aspect, the present invention provides an image registration method, comprising: acquiring a moving image to be registered and a fixed image to be registered; inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a region of interest deformation field; wherein the first registration model is trained based on a first region of interest image corresponding to the moving image used for training and a second region of interest image corresponding to the fixed image used for training; applying the region of interest deformation field to the moving image to be registered to obtain a deformed region of interest image; and registering the deformed region of interest image and the fixed image to be registered to obtain a registered image.

[0006] As an optional embodiment of the present invention, the step of inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest comprises: inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a first-level deformation field of the region of interest; obtaining a second-level deformation field of the region of interest according to the moving image to be registered and the first-level deformed image of the region of interest, wherein the first-level deformed image of the region of interest is obtained by applying the first-level deformation field of the region of interest to the moving image to be registered; cascading the first-level deformation field of the region of interest and the second-level deformation field of the region of interest to obtain a multi-level cascaded deformation field of the region of interest; applying the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest comprises: applying the multi-level cascaded deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest.

[0007] As an optional embodiment of the present invention, after obtaining the second-level region of interest deformation field according to the first-level region of interest deformation image and the moving image to be registered, it also includes: repeatedly performing the following steps until i satisfies a first preset value: obtaining the i-th level region of interest deformation field according to the moving image to be registered and the i-1-th level region of interest deformation image, wherein the i-1-th level region of interest deformation image is obtained by applying the i-1-th level region of interest deformation field to the i-2-th level region of interest deformation image, and i is greater than or equal to 3; cascading the first-level region of interest deformation field and the second-level region of interest deformation field to obtain a multi-level cascaded region of interest deformation field, includes: cascading the first-level region of interest deformation field, the second-level region of interest deformation field, and the i-th level region of interest deformation field in sequence to obtain a multi-level cascaded region of interest deformation field.

[0008] As an optional embodiment of the present invention, after applying the deformation field of the region of interest to the moving image to be registered to obtain the deformed image of the region of interest, it also includes: inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the non-interest region, wherein the second registration model is trained based on a first non-interest region image corresponding to the moving image used for training and a second non-interest region image corresponding to the fixed image used for training; adjusting the deformed image of the region of interest according to the deformation field of the non-interest region to obtain a final deformed image; and registering the deformed image of the region of interest and the fixed image to be registered to obtain a registered image, including: registering the final deformed image with the fixed image to be registered to obtain a registered image.

[0009] As an optional embodiment of the present invention, the step of inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the region of non-interest comprises: inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a first-level deformation field of the region of non-interest; obtaining a second-level deformation field of the region of non-interest according to the first-level deformed image and the fixed image to be registered, wherein the first-level deformed image is obtained by applying the first-level deformation field of the region of non-interest to the deformed image of the region of interest; cascading the first-level deformation field of the region of non-interest and the second-level deformation field of the region of non-interest to obtain a multi-level cascaded deformation field of the region of non-interest; and adjusting the deformed image of the region of interest according to the deformation field of the region of non-interest to obtain a final deformed image comprises: adjusting the deformed image of the region of interest according to the multi-level cascaded deformation field of the region of non-interest to obtain a final deformed image.

[0010] As an optional embodiment of the present invention, after obtaining the second-level non-interest area deformation field according to the first-level deformed image and the fixed image to be registered, it also includes: repeating the following steps until j satisfies a second preset value: obtaining the j-th level non-interest area deformation field according to the j-1-th level deformed image and the fixed image to be registered, wherein the j-1-th level deformed image is obtained by applying the j-1-th level non-interest area deformation field to the j-2-th level deformed image, and j is greater than or equal to 3; cascading the first-level non-interest area deformation field and the second-level non-interest area deformation field to obtain a multi-level cascaded non-interest area deformation field, includes: cascading the first-level non-interest area deformation field, the second-level non-interest area deformation field, and the j-th level non-interest area deformation field in sequence to obtain a multi-level cascaded non-interest area deformation field.

[0011] As an optional embodiment of the present invention, before inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model, it also includes: repeating the following steps until the pre-trained first registration model is obtained when the value of the first preset loss function is minimized: inputting the first region of interest image corresponding to the moving image for training and the second region of interest image corresponding to the fixed image for training into the first registration model to be trained to obtain a deformation field of the region of interest to be trained; applying the deformation field of the region of interest to be trained to the first region of interest image to obtain a deformed first region of interest image; wherein the first preset loss function includes an image similarity loss function between the deformed first region of interest image and the second region of interest image, and a smoothness regularization function of the deformation field of the region of interest to be trained.

[0012] As an optional embodiment of the present invention, before inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model, it also includes: repeating the following steps until the pre-trained second registration model is obtained when the second preset loss function takes a minimum value: inputting the first non-interest region image corresponding to the moving image used for training and the second non-interest region image corresponding to the fixed image used for training into the second registration model to be trained to obtain the non-interest region deformation field to be trained; applying the non-interest region deformation field to be trained to the first non-interest region image to obtain the deformed first non-interest region image; wherein the second preset loss function includes an image similarity loss function between the deformed first non-interest region image and the second non-interest region image, and a smoothness regularization function of the non-interest region deformation field to be trained.

[0013] As an optional embodiment of the present invention, the step of repeatedly performing the following steps until the pre-trained first registration model is obtained when the first preset loss function takes a minimum value also includes: acquiring a moving image for training and a fixed image for training; inputting the moving image for training and the fixed image for training into a pre-trained segmentation model to obtain a first segmented image corresponding to the moving image for training and a second segmented image corresponding to the fixed image for training; obtaining the first area of ​​interest image based on the moving image and the first segmented image, and obtaining the second area of ​​interest image based on the fixed image and the second segmented image; and / or obtaining a first non-interest area image based on the moving image and the first area of ​​interest image, and obtaining a second non-interest area image based on the fixed image and the second area of ​​interest image.

[0014] As an optional embodiment of the present invention, before inputting the moving image for training and the fixed image for training into a pre-trained segmentation model, it also includes: training the segmentation model to be trained until the value of the third preset loss function is minimized, thereby obtaining the pre-trained segmentation model; wherein the first preset loss function is a negative log-likelihood function between the segmentation label and the predicted segmentation label.

[0015] As an optional embodiment of the present invention, the first registration model is a registration convolutional neural network model Reg-CNN; and / or, the second registration model is a registration distance neural network model Reg-CNN; and / or, the segmentation model is a segmentation convolutional neural network Seg-CNN model.

[0016] In a second aspect, the present invention provides an image registration device, comprising: an acquisition module, used to acquire a moving image to be registered and a fixed image to be registered; an input module, used to input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest; wherein the first registration model is trained based on a first region of interest image corresponding to the moving image used for training and a second region of interest image corresponding to the fixed image used for training; a deformation module, used to apply the region of interest deformation field to the moving image to be registered to obtain a deformed region of interest image; and a registration module, used to register the deformed region of interest image and the fixed image to be registered to obtain a registered image.

[0017] In a third aspect, the present invention provides an electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the steps of the image registration method described in any one of the first aspects when executing the program stored in the memory.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the image registration method as described in any one of the first aspects are implemented.

[0019] The image registration method, device, electronic device and storage medium provided by the embodiments of the present invention obtain a moving image to be registered and a fixed image to be registered; input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest; wherein the first registration model is trained based on a first region of interest image corresponding to the moving image for training and a second region of interest image corresponding to the fixed image for training; the region of interest deformation field is applied to the moving image to be registered to obtain a deformed region of interest image; the deformed region of interest image and the fixed image to be registered are registered to obtain a registered image; that is, the embodiment of the present invention inputs the moving image to be registered and the fixed image into a model specifically used for region of interest registration for registration, thereby reducing the interference of non-regions of interest in its registration process and improving the image registration effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A schematic diagram of a flow chart of an image registration method provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the structure of a Reg-CNN model provided in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of a flow chart of another image registration method provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of a flow chart of another image registration method provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a dynamic cascade image registration method provided by an embodiment of the present invention;

[0027] Figure 6A schematic diagram of a process of training a registration model provided by an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of the structure of a Seg-CNN model provided in an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of a training process of a dual-channel network registration model provided in an embodiment of the present invention;

[0030] Fig. 9 A schematic diagram of the structure of an image registration device provided by an embodiment of the present invention;

[0031] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Image registration refers to the process of aligning a moving image (or motion image) with a fixed image through rigid, affine or nonlinear transformation. Deformable medical image registration is to establish a nonlinear correspondence between a pair of medical images and estimate the appropriate nonlinear transformation to achieve the purpose of registering a pair of medical images. Medical personnel perform visual analysis of the region of interest in a unified anatomical space. This task is a key image processing step in clinical diagnosis and plays an important role in many clinical applications.

[0034] Traditional medical image registration is an iterative optimization process. However, the iterative method is extremely slow and cannot achieve real-time registration in clinical operations. Therefore, a medical image registration method based on deep learning has been proposed. Currently, there are three main types of medical image registration methods based on deep learning: (1) medical image registration methods based on unsupervised learning; (2) image registration methods based on label constraints; and (3) medical image registration methods based on weak supervision.

[0035] However, the above methods only obtain the deformation field by learning the correspondence between image pairs, so they do not have the perception ability to identify the region of interest and the region of no interest (or so-called task-indifferent region). The task-indifferent region has a significant and non-corresponding structure between the moving image and the fixed image. When the registration model without region-aware ability performs image registration, the region of interest has to compromise the task-indifferent region, which leads to incorrect registration of the region of interest and poor image registration effect.

[0036] In view of the above technical problems, the technical concept of the present invention is to directly input the image to be registered into a model specifically used for region of interest registration for registration, thereby reducing the interference of the task-irrelevant region on the region of interest and reducing the distortion of the region of interest.

[0037] In addition, the existing technology also has the following technical problems: For example, the one-time registration model is currently mandatory. When encountering complex deformations, especially when there is a large deformation between the fixed image and the moving image, the registration effect between the image pairs will be poor. For example, the image registration method based on unsupervised supervision uses a small number of labels and a fine similarity metric to learn the deformation field of the region of interest between a pair of images. Since the segmentation label lacks texture information to constrain the corresponding behavior, it only focuses on the alignment of the edge of the region of interest and ignores the texture of the region of interest, resulting in distortion of the region of interest. Therefore, this deformation process will deform the texture of the region of interest, making the deformed image unconvincing.

[0038] Figure 1 A flowchart of an image registration method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the image registration method includes:

[0039] Step S101: Acquire a moving image to be registered and a fixed image to be registered.

[0040] Specifically, the execution subject of this embodiment is an image registration device, or a server or electronic device on which the image registration device is deployed. In this step, the image registration device obtains a moving image M to be registered and a fixed image F to be registered.

[0041] Step S102: input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest.

[0042] The first registration model is obtained through training based on a first region of interest image corresponding to a moving image used for training and a second region of interest image corresponding to a fixed image used for training.

[0043] Optionally, the first registration model may be a registration convolutional neural network (Registration Convolutional Neural Network, Reg-CNN for short) model. Figure 2 A structural diagram of a Reg-CNN model provided in an embodiment of the present invention is shown in FIG. Figure 2 As shown in the figure, the Reg-CNN model inputs the region of interest image during the training phase and outputs the region of interest deformation field. It follows the traditional U-Net structure and has a five-level hierarchical encoder-decoder structure. In the encoder, each resolution stage has a 3×3×3 convolution layer, and the feature map between each stage is downsampled by pooling operations. In the decoder, the first two stages have a 3×3×3 convolution, and the last two stages have two 3×3×3 convolutions. Each convolution is followed by a GN function and a ReLU function for detailed estimation. Finally, a 1×1×1 convolution is used to estimate the deformation of each voxel in the three channels in the x, y, and z directions.

[0044] That is to say, the first Reg-CNN model is specifically used to output the deformation field of the region of interest of the image pair. In this step, after the moving image M to be registered and the fixed image F to be registered are input into the first Reg-CNN model, the corresponding deformation field Φ of the region of interest can be obtained.

[0045] Step S103: applying the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest.

[0046] Specifically, the deformation field of the region of interest Φ uses a spatial transform network (STN) to perform spatial transformation processing on the moving image M to be registered, and obtains the deformed image W of the region of interest R .

[0047] Step S104: registering the deformed image of the region of interest and the fixed image to be registered to obtain a registered image.

[0048] Specifically, the region of interest deformed image W R The registration process is performed with the fixed image F to be registered to obtain the registered image. The typical deformable image registration formula is shown in formula (1).

[0049]

[0050] in, represents the optimized deformation field, represents the image similarity loss function, and the composite operation represents the spatial transformation of the moving image according to Φ.

[0051] The image registration method provided by the embodiment of the present invention obtains a moving image to be registered and a fixed image to be registered; inputs the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest; wherein the first registration model is trained based on a first region of interest image corresponding to the moving image for training and a second region of interest image corresponding to the fixed image for training; applies the deformation field of the region of interest to the moving image to be registered to obtain a deformed region of interest image; and registers the deformed region of interest image and the fixed image to be registered to obtain a registered image; that is, the embodiment of the present invention reduces the interference of non-interest regions of the image on its registration process by inputting the image pair to be registered into a model specifically used for region of interest registration for registration, thereby improving the image registration effect.

[0052] Based on the above embodiments, Figure 3 A flowchart of another image registration method provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the image registration method includes:

[0053] Step S201: Acquire a moving image to be registered and a fixed image to be registered.

[0054] Step S202: input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a first-level region of interest deformation field.

[0055] Step S203: obtaining a second-level region of interest deformation field according to the moving image to be registered and the first-level region of interest deformation image.

[0056] The first-level region of interest deformed image is obtained by applying the first-level region of interest deformation field to the moving image to be registered.

[0057] Step S204: cascade the first-level region of interest deformation field and the second-level region of interest deformation field to obtain a multi-level cascaded region of interest deformation field.

[0058] Step S205: Apply the multi-stage cascaded region of interest deformation field to the moving image to be registered to obtain a region of interest deformed image.

[0059] Step S206: registering the deformed image of the region of interest and the fixed image to be registered to obtain a registered image.

[0060] The implementation methods of step S201 and step S206 in this embodiment are similar to the implementation methods of step S101 and step S104 in the aforementioned embodiments, and will not be repeated here.

[0061] The difference from the above-mentioned embodiment is that, considering that the one-time direct registration method adopted in the prior art brings a burden to accurate registration when dealing with complex deformations, especially when there is a large deformation between the fixed image and the moving image, it will lead to a technical problem of poor registration effect between the image pairs. Therefore, in order to improve the registration effect of the image pair under complex deformation, in this embodiment, the moving image to be registered and the fixed image to be registered are input into a pre-trained first registration model to obtain a first-level region of interest deformation field; a second-level region of interest deformation field is obtained according to the moving image to be registered and the first-level region of interest deformation image, wherein the first-level region of interest deformation image is obtained by applying the first-level region of interest deformation field to the moving image to be registered; the first-level region of interest deformation field and the second-level region of interest deformation field are cascaded to obtain a multi-level cascaded region of interest deformation field; the multi-level cascaded region of interest deformation field is applied to the moving image to be registered to obtain a region of interest deformation image.

[0062] Specifically, the moving image M to be registered and the fixed image F to be registered are input into the first registration model to obtain the first-level region of interest deformation field As shown in formula (2).

[0063]

[0064] in, represents the deformation field learned by the convolutional neural network.

[0065] Then, the deformation field of the first-level region of interest Acting on the moving image M to be registered, the first-level region of interest deformed image can be obtained As shown in formula (3).

[0066]

[0067] The composite operation represents a spatial transformation of the moving image according to the deformation field.

[0068] Then, the moving image M to be registered and the first-level region of interest deformed image The second-level region of interest deformation field can be obtained As shown in formula (4); the deformation field of the second-level interest region Acting on the first level region of interest to deform the image The second level region of interest deformed image can be obtained As shown in formula (5).

[0069]

[0070]

[0071] Then, the deformation field of the first level region of interest is Second level region of interest deformation field Cascade to obtain the deformation field Φ of the region of interest of the two-stage cascade roi (F,M), as shown in formula (6).

[0072]

[0073] Then, the two-stage cascaded deformation field of the region of interest is applied to the moving image M to be registered to obtain the deformed image W of the region of interest R , as shown in formula (7).

[0074] WR=Φ roi (F,M)M(7)

[0075] Finally, the region of interest deformed image W R It is registered with the fixed image F to be registered to obtain a registered image.

[0076] As an optional embodiment, after step S203, it also includes: repeatedly performing the following steps until i satisfies a first preset value: obtaining an i-th level region of interest deformation field according to the moving image to be registered and the i-1-th level region of interest deformation image, wherein the i-1-th level region of interest deformation image is obtained by applying the i-1-th level region of interest deformation field to the i-2-th level region of interest deformation image, and i is greater than or equal to 3; then the step S204 includes: cascading the first-level region of interest deformation field, the second-level region of interest deformation field, and the i-th level region of interest deformation field in sequence to obtain a multi-level cascaded region of interest deformation field.

[0077] Specifically, the third-level region of interest deformation field, the fourth-level region of interest deformation field, ... the i-th level region of interest deformation field may be continuously obtained until the n-th level region of interest deformation field is obtained, where n is a first preset value.

[0078] Similar to formula (4) and formula (5), the deformation field of the n-th level region of interest can be further derived: As shown in formula (8), the deformed image of the n-th level interest region is As shown in formula (9).

[0079]

[0080]

[0081] Then, the first-level region of interest deformation field, the second-level region of interest deformation field, the third-level region of interest deformation field, and the n-th level region of interest deformation field are sequentially cascaded to obtain a multi-level cascaded region of interest deformation field, as shown in formula (10).

[0082]

[0083] The multi-level cascaded region of interest deformation field shown in formula (10) is applied to the moving image to be registered to obtain a region of interest deformation image. Finally, the region of interest deformation image and the fixed image to be registered are registered to obtain a registered image.

[0084] The image registration method provided by the embodiment of the present invention obtains a first-level region of interest deformation field by inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model; obtains a second-level region of interest deformation field according to the moving image to be registered and the first-level region of interest deformation image, wherein the first-level region of interest deformation image is obtained by acting the first-level region of interest deformation field on the moving image to be registered; further, obtains an i-th level region of interest deformation field according to the moving image to be registered and the i-1th level region of interest deformation image, wherein the i-1th level region of interest deformation image is acted on the i-1th level region of interest deformation field. A 2-level region of interest deformed image is obtained, where i is greater than or equal to 3; the first-level region of interest deformation field, the second-level region of interest deformation field, and the i-th level region of interest deformation field are cascaded in sequence to obtain a multi-level cascaded region of interest deformation field; the first-level region of interest deformation field and the second-level region of interest deformation field are cascaded to obtain a multi-level cascaded region of interest deformation field; the multi-level cascaded region of interest deformation field is applied to the moving image to be registered to obtain a region of interest deformed image; that is, the embodiment of the present invention adopts a dynamic cascade strategy, that is, by using two or more levels of cascaded region of interest deformation fields to gradually register the region of interest, global alignment is achieved, and the accuracy of image registration during complex deformation is improved.

[0085] Based on the above embodiments, Figure 4 A flowchart of another image registration method provided by an embodiment of the present invention is shown as follows: Figure 4 As shown, the image registration method includes:

[0086] Step S301: Acquire a moving image to be registered and a fixed image to be registered.

[0087] Step S302: input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest.

[0088] The first registration model is obtained through training based on a first region of interest image corresponding to a moving image used for training and a second region of interest image corresponding to a fixed image used for training.

[0089] Step S303: Apply the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest.

[0090] Step S304: input the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the non-region of interest.

[0091] The second registration model is obtained through training based on a first non-interest region image corresponding to a moving image used for training and a second non-interest region image corresponding to a fixed image used for training.

[0092] Step S305: adjusting the deformed image of the region of interest according to the deformation field of the non-region of interest to obtain a final deformed image.

[0093] Step S306: register the final deformed image and the fixed image to be registered to obtain a registered image.

[0094] The implementation methods of step S301 to step S303 in this embodiment are similar to the implementation methods of step S101 to step S103 in the above embodiment, and will not be repeated here.

[0095] The difference from the above-mentioned embodiment lies in that, considering that the potential region of interest of the image is lost due to inaccurate division of the region of interest image used to train the first registration model, in order to further explore the potential region of interest images of the non-interest region of the image and further improve the image registration effect, in the present embodiment, the deformed image of the region of interest and the fixed image to be registered are input into a pre-trained second registration model to obtain a deformation field of the non-interest region, wherein the second registration model is trained based on a first non-interest region image corresponding to the moving image used for training and a second non-interest region image corresponding to the fixed image used for training; the deformed image of the region of interest is adjusted according to the deformation field of the non-interest region to obtain a final deformed image; the final deformed image and the fixed image to be registered are registered to obtain a registered image.

[0096] Optionally, the second registration model is a registration convolutional neural network model Reg-CNN, which has the same structure as Figure 2 The difference is that the input of the second Reg-CNN model in the training phase is the non-interest region image, and the output is the non-interest region deformation field. In other words, the second Reg-CNN is specifically used to predict the non-interest region deformation field of the image pair to be registered.

[0097] In this embodiment, after the deformation field of the region of interest obtained by the first Reg-CNN model or the two-level or multi-level cascaded deformation field of the region of interest is applied to the moving image to be registered to obtain the deformed image of the region of interest, the deformed image of the region of interest and the fixed image to be registered can be input into the second Reg-CNN model to predict the corresponding deformation field of the non-region of interest, and then the predicted deformation field of the non-region of interest is applied to the deformed image of the region of interest, that is, further fine-tuning of the deformation image of the region of interest is achieved to obtain the final deformed image, and finally the final deformed image and the fixed image to be registered are registered to obtain the registered image.

[0098] As an optional embodiment, the step S304 includes: inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a first-level non-interest region deformation field; obtaining a second-level non-interest region deformation field based on the first-level deformed image and the fixed image to be registered, wherein the first-level deformed image is obtained by applying the first-level non-interest region deformation field to the deformed image of the region of interest; cascading the first-level non-interest region deformation field and the second-level non-interest region deformation field to obtain a multi-level cascaded non-interest region deformation field; then the step S305 includes: adjusting the deformed image of the region of interest according to the multi-level cascaded non-interest region deformation field to obtain a final deformed image.

[0099] Furthermore, after obtaining the second-level non-interest area deformation field according to the first-level deformed image and the fixed image to be registered, it also includes: repeating the following steps until j satisfies a second preset value: obtaining the j-th level non-interest area deformation field according to the j-1-th level deformed image and the fixed image to be registered, wherein the j-1-th level deformed image is obtained by applying the j-1-th level non-interest area deformation field to the j-2-th level deformed image, and j is greater than or equal to 3; cascading the first-level non-interest area deformation field and the second-level non-interest area deformation field to obtain a multi-level cascaded non-interest area deformation field, includes: cascading the first-level non-interest area deformation field, the second-level non-interest area deformation field, and the j-th level non-interest area deformation field in sequence to obtain a multi-level cascaded non-interest area deformation field.

[0100] Specifically, similar to the deformation field of the region of interest cascaded in two or more stages, the deformation field of the non-region of interest can also be cascaded in two or more stages. R The fixed image F to be registered is input into the second Reg-CNN model to obtain the first-level non-interest region deformation field As shown in formula (11).

[0101]

[0102] Then, the first-level non-interest region deformation field Act on the fixed image F to be registered to obtain the first-level deformed image As shown in formula (12).

[0103]

[0104] By analogy, the third level non-interest region deformation field, the fourth level non-interest region deformation field, ... the jth level interest region deformation field can be obtained, until the nth level non-interest region deformation field is obtained, where n is the second preset value, and the first preset value and the second preset value can be the same or different. As shown in formula (13), the n-th level deformed image As shown in formula (14).

[0105]

[0106]

[0107] Therefore, the first-level non-interest region deformation field, the second-level non-interest region deformation field, the third-level non-interest region deformation field, and the n-th-level non-interest region deformation field are cascaded in sequence, and the obtained multi-level cascaded non-interest region deformation field is shown in formula (15).

[0108]

[0109] Then, the multi-level cascaded deformation field of the non-interest region is applied to the deformed image of the interest region to obtain the final deformed image W as shown in formula (16).

[0110] W=Φ nroi (W R , F)W R (16)

[0111] Finally, the final deformed image W is registered with the fixed image F to obtain a registered image.

[0112] The image registration method provided by the embodiment of the present invention obtains a first-level non-interest region deformation field by inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model; obtains a second-level non-interest region deformation field according to the first-level deformed image and the fixed image to be registered, wherein the first-level deformed image is obtained by applying the first-level non-interest region deformation field to the deformed image of the region of interest; further, the following steps are repeatedly performed until j meets a second preset value: obtains a j-th level non-interest region deformation field according to the j-1-th level deformed image and the fixed image to be registered, wherein the j-1-th level deformed image is obtained by applying the j-1-th level deformed image to the deformed image of the region of interest A non-interest region deformation field is applied to the j-2th level deformed image, where j is greater than or equal to 3, and the first level non-interest region deformation field, the second level non-interest region deformation field, and the jth level non-interest region deformation field are sequentially cascaded to obtain a multi-level cascaded non-interest region deformation field; the interest region deformed image is adjusted according to the multi-level cascaded non-interest region deformation field to obtain a final deformed image; the final deformed image and the fixed image to be registered are registered to obtain a registered image; that is, the present invention implements a stepwise fine-tuning of the interest region deformed image through a multi-level cascaded non-interest region deformation field, thereby mining potential interest region information in the image non-interest region, and further improving the image registration effect.

[0113] In order to further understand this embodiment, Figure 5 A schematic diagram of a dynamic cascade image registration method provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, taking brain MRI as an example. The moving image and the fixed image are input into the first Reg-CNN model corresponding to the region of interest. In the first level region of interest deformation field, Then the deformation field of the first level region of interest Combined with the moving image, the first level deformed region of interest image is obtained And the first level deformed region of interest image and the first-level region of interest deformation field Perform convolution to obtain the second-level region of interest deformation field Second level region of interest deformation field and the first level deformed region of interest image Perform a composite operation to obtain the second-level deformed region of interest image And so on, until the n-th level deformed region of interest image is obtained. Then the nth level deformed region of interest image The second registration model corresponding to the fixed image input to the non-interest area In the first level non-interest region deformation field, Region of interest image of the nth level deformation After the first level deformation field of non-interest area Second level deformation field of non-interest area The third level deformation field of non-interest area ...the deformation field of the nth level non-interest region The final target image W is obtained by gradual fine-tuning, and the final target image W is registered with the fixed image to obtain a registered image.

[0114] Among them, after the moving image to be registered is deformed by the multi-level cascaded deformation field of the region of interest, the deformation image of the region of interest can be obtained. The similarity measure between the fixed image and the image is as follows. If the similarity is large, the registered image can be directly obtained. If the similarity is low, the deformation field of the non-interest area can be cascaded to obtain the registered image. Fine-tune to obtain the final target image W, and obtain the similarity measure between the final target image and the fixed image.

[0115] In summary, the embodiment of the present invention first obtains the region of interest W of the moving image by dynamically adjusting multiple cascaded region of interest deformation fields. R Then use the deformation field function Φ of the task-indifferent region nroi (W R ,F) for W R Fine-tuning is performed to obtain the final deformed image W, and finally the deformed image W is registered with the fixed image, thereby completing the entire medical image registration. The embodiment of the present invention realizes an image registration process from coarse to fine.

[0116] On the basis of the above embodiment, before step S102, steps S401-S403 are further included to implement training of the first registration model, and optionally, steps S404-S406 are further included to implement training of the second registration model. Figure 6 A flowchart of a registration model training process provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the following steps are included:

[0117] Step S401: input a first region of interest image corresponding to a moving image for training and a second region of interest image corresponding to a fixed image for training into a first registration model to be trained, to obtain a deformation field of the region of interest to be trained.

[0118] Step S402: Apply the to-be-trained region of interest deformation field to the first region of interest image to obtain a deformed first region of interest image.

[0119] Step S403, repeating steps S401 and S402 until the first preset loss function is minimized to obtain the pre-trained first registration model.

[0120] The first preset loss function includes an image similarity loss function between the deformed first region of interest image and the second region of interest image, and a smoothness regularization function of the region of interest deformation field to be trained.

[0121] Specifically, first, the first region of interest image R corresponding to the moving image used for training M The second region of interest image R corresponding to the fixed image used for training F Input into the first Reg-CNN model to be trained to obtain the deformation field of the region of interest to be trained, which is R M and R F The convolution operation result; then, the deformation field of the region of interest to be trained is processed by the spatial transformation operation R m , obtain the deformed first region of interest image; then calculate whether the first preset loss function obtains the minimum value, if so, end the training of the first Reg-CNN model, if not, continue to iterate and execute steps S401 and S402.

[0122] Among them, the first preset loss function includes the image similarity loss function between the deformed first region of interest image and the second region of interest image, and the smoothness regularization function of the region of interest deformation field to be trained, as shown in formula (17).

[0123]

[0124] Among them, this embodiment uses Local Normalized Cross-Correlation (LNCC) as the similarity metric To evaluate the similarity of two image alignments, LNCC is shown in formula (18).

[0125]

[0126] Among them, F and W represent two input images, x i represents the local area centered on x, and Respectively represent F(x i ) and W(x i ) in x i The local mean of the location.

[0127] The larger the LNCC value, the better the image registration effect. In this embodiment, the image similarity loss function between two images is expressed as: Therefore, minimizing This helps W approach F, but this may result in a non-smooth deformation field Φ. To alleviate this problem, this embodiment uses diffusion regularization of the spatial gradient to obtain a smooth Φ, as shown in formula (19).

[0128]

[0129] The spatial gradient is approximated by the difference between adjacent voxels. and Combined, the first preset loss function shown in formula (17) is obtained. In addition, a is a weight parameter, which is set according to the experience of technicians in this field and is not limited by the present invention.

[0130] Step S404: input the first non-interest region image corresponding to the moving image for training and the second non-interest region image corresponding to the fixed image for training into the second registration model to be trained to obtain the deformation field of the non-interest region to be trained.

[0131] Step S405: Apply the non-interest region deformation field to be trained to the first non-interest region image to obtain a deformed first non-interest region image.

[0132] Step S406, repeating steps S404 and S405 until the second preset loss function takes a minimum value to obtain the pre-trained second registration model.

[0133] Specifically, first, the first non-interest region image nR corresponding to the moving image used for training M The second region of interest image nR corresponding to the fixed image used for training F Input it into the second Reg-CNN model to be trained to obtain the deformation field of the non-interest region to be trained, and the deformation field of the non-interest region to be trained is nR M and nR F The convolution operation result; then, the deformation field of the non-interest area to be trained is processed by the spatial transformation operation nR M , obtain the deformed first non-region of interest image; then calculate whether the second preset loss function obtains the minimum value, if so, end the training of the second Reg-CNN model, if not, continue to iterate and execute steps S404 and S405.

[0134] Among them, the second preset loss function is similar to the first preset loss function, including the image similarity loss function between the deformed first non-interest area image and the second non-interest area image, and the smoothness regularization function of the non-interest area deformation field to be trained, as shown in formula (20).

[0135]

[0136] Wherein, γ is a weight parameter, which is set according to the experience of those skilled in the art and is not limited in the present invention. In summary, in order to avoid mutual interference between the region of interest and the region of no concern to the task during the registration process, this embodiment trains the first registration model and the second registration model respectively, and optimizes the first registration model and the second registration model using different preset loss functions respectively.

[0137] As an optional embodiment, before step S401, steps S407-S410 may be further included. Figure 6 shown.

[0138] Step S407: Acquire a moving image for training and a fixed image for training.

[0139] Step S408: input the moving image for training and the fixed image for training into a pre-trained segmentation model to obtain a first segmented image corresponding to the moving image for training and a second segmented image corresponding to the fixed image for training.

[0140] Step S409: obtaining the first region of interest image according to the moving image and the first segmented image, and obtaining the second region of interest image according to the fixed image and the second segmented image.

[0141] Step S410: obtaining a first non-interest region image according to the moving image and the first interest region image, and obtaining a second non-interest region image according to the fixed image and the second interest region image.

[0142] Specifically, first obtain the moving image and the fixed image for training; then input the moving image and the fixed image into the segmentation model respectively to obtain the first segmented image s′ corresponding to the moving image M The second segmented image s′ corresponding to the fixed image F ; Then the first segmented image s′ M Multiply the moving image by the moving image to obtain the moving image of the region of interest, that is, the first region of interest image R M ; The second segmented image S′ F Multiply the fixed image by the fixed image to obtain the fixed image of the region of interest, that is, the second region of interest image R F , then R M and R F The first Reg-CNN model to be trained is trained as training data. Further, the moving image can be subtracted from the first region of interest image R M Obtain the non-interest region of the moving image, that is, the first non-interest region image nR M , fixed image F minus the second region of interest image R FObtain the non-interest region of the fixed image, that is, the second non-interest region image nR F , then nB M and nR F The second Reg-CNN model to be trained is trained using the training data.

[0143] Optionally, the segmentation model is a segmentation convolutional neural network (Segmentation Convolutional Neural Network, Seg-CNN for short) model. Figure 7 A schematic diagram of the structure of a Seg-CNN model provided in an embodiment of the present invention is shown in FIG. Figure 7 As shown in Figure 1, Seg-CNN segments the labels of the regions of interest in moving and fixed images. It is similar to the traditional U-Net structure and consists of a four-layer hierarchical encoder-decoder with skip connections. For each layer in the encoder, two consecutive convolutional layers are used, followed by a set of normalization and a rectified linear unit, which contains a 3×3×3 convolutional layer with a step size of 1, and then a 3×3×3 convolutional layer is added behind it with a step size of 2 to downsample the feature map until it reaches the lowest layer. At each layer of the decoder, the feature maps of the encoder are connected through a skip connection and the deconvolution layer is upsampled using a 3×3×3 convolution with a step size of 1. Connections in the same resolution stage add detailed information to the decoder, and each resolution stage of the decoder outputs a deep supervised segmentation through a 1×1×1 convolution.

[0144] It should be noted that compared with the prior art in which the segmentation label lacks texture information to constrain the corresponding behavior, which only focuses on the alignment of the edge of the region of interest and ignores the texture of the region of interest, resulting in the problem of distortion of the region of interest, the Seg-CNN model used in this embodiment can retain the texture information of the region of interest.

[0145] As an optional embodiment, before step S408, the step also includes: training the segmentation model to be trained until the value of the first preset loss function is minimized to obtain the segmentation model; wherein the first preset loss function is a negative log-likelihood function between the segmentation label and the predicted segmentation image.

[0146] Specifically, the segmentation model is pre-trained, that is, the segmentation model to be trained needs to be trained before step S408. In this embodiment, the cross entropy loss function is used to calculate the segmentation label S of the sample. l,n and the predicted segmented image s′ i,n The minimum value of the negative log-likelihood function between is used to optimize Seg-CNN, as shown in formula (21):

[0147]

[0148] Where N is the number of semantic channels of Seg-CNN and L is the number of voxels in each semantic channel.

[0149] In summary, the embodiment of the present invention provides a network registration model based on dual channels (a first registration model corresponding to the region of interest and a second registration model corresponding to the non-region of interest) during the training phase of the registration model. Figure 8 A schematic diagram of a training process of a dual-channel network registration model provided in an embodiment of the present invention, taking brain MRI as an example, referring to Figure 8 As shown in the figure, firstly, a moving image M and a fixed image F are obtained; then the moving image M and the fixed image F are respectively input into the segmentation convolutional neural network Seg-CNN to obtain the segmentation image s′ of the moving image. M and the segmented image s′ of the fixed image F ; Then, s′ M and s′ F Multiply by M and F respectively to obtain the moving image R of the area of ​​interest M and a fixed image R F At the same time, the moving image M and the fixed image F are respectively subtracted from the corresponding regions of interest to obtain the non-interest region nR M and nR F ; Then, the regions of interest and non-regions of interest of the moving image and the fixed image with preserved texture are respectively put into the corresponding Reg-CNN to obtain the deformation field; The deformation field uses the spatial transformation network STN to deform the regions of interest and non-regions of interest of the moving image to obtain the region of interest W of the deformed image R and the non-interest region W nR ; Then optimize according to the preset loss function of the region of interest and the region of no interest respectively. That is, the dual-channel network registration model of this embodiment enables the regional perception ability of the model to identify the region of interest and the region of no concern in the image, and retain the texture information of the region of interest and the region of no concern in the image, and use the region of interest with texture information and the region of no concern in the task to learn the deformation field respectively, so as to eliminate the mutual interference between the region of interest and the region of no concern in the task, and then eliminate the misalignment of the region of interest and reduce the distortion of the region of interest.

[0150] Fig. 9 A schematic diagram of the structure of an image registration device provided by an embodiment of the present invention is shown in FIG. Fig. 9 As shown, the image registration device comprises:

[0151] The acquisition module 10 is used to acquire the moving image to be registered and the fixed image to be registered; the deformation field module 20 is used to input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain the deformation field of the region of interest; wherein the first registration model is obtained by training based on the first region of interest image corresponding to the moving image used for training and the second region of interest image corresponding to the fixed image used for training; the deformation module 30 is used to apply the deformation field of the region of interest to the moving image to be registered to obtain a deformed region of interest image; the registration module 40 is used to register the deformed region of interest image and the fixed image to be registered to obtain a registered image.

[0152] As an optional embodiment of the present invention, the deformation field module 20 is specifically used to: input the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a first-level region of interest deformation field; obtain a second-level region of interest deformation field according to the moving image to be registered and the first-level region of interest deformed image, wherein the first-level region of interest deformed image is obtained by applying the first-level region of interest deformation field to the moving image to be registered; cascade the first-level region of interest deformation field and the second-level region of interest deformation field to obtain a multi-level cascaded region of interest deformation field; the deformation module 30 is specifically used to: apply the multi-level cascaded region of interest deformation field to the moving image to be registered to obtain a region of interest deformed image.

[0153] As an optional embodiment of the present invention, the deformation field module 20 is also used to: repeatedly perform the following steps until i satisfies a first preset value: obtain an i-th level region of interest deformation field according to the moving image to be registered and the i-1-th level region of interest deformation image, wherein the i-1-th level region of interest deformation image is obtained by applying the i-1-th level region of interest deformation field to the i-2-th level region of interest deformation image, and i is greater than or equal to 3; cascade the first-level region of interest deformation field, the second-level region of interest deformation field, and the i-th level region of interest deformation field in sequence to obtain a multi-level cascaded region of interest deformation field.

[0154] As an optional embodiment of the present invention, the deformation field module 20 is further used to: input the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the non-interest region, wherein the second registration model is obtained by training based on a first non-interest region image corresponding to the moving image used for training and a second non-interest region image corresponding to the fixed image used for training; the deformation module 30 is used to adjust the deformed image of the region of interest according to the deformation field of the non-interest region to obtain a final deformed image; the registration module 40 is specifically used to: register the final deformed image with the fixed image to be registered to obtain a registered image.

[0155] As an optional embodiment of the present invention, the deformation field module 20 is specifically used to: input the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a first-level non-interest region deformation field; obtain a second-level non-interest region deformation field based on the first-level deformed image and the fixed image to be registered, wherein the first-level deformed image is obtained by applying the first-level non-interest region deformation field to the deformed image of the region of interest; cascade the first-level non-interest region deformation field and the second-level non-interest region deformation field to obtain a multi-level cascaded non-interest region deformation field; the deformation module 30 is specifically used to: adjust the deformed image of the region of interest according to the multi-level cascaded non-interest region deformation field to obtain a final deformed image.

[0156] As an optional embodiment of the present invention, the deformation field module 20 is also used to: repeatedly perform the following steps until j satisfies a second preset value: obtain a j-th level non-interest area deformation field based on the j-1-th level deformed image and the fixed image to be registered, wherein the j-1-th level deformed image is obtained by applying the j-1-th level non-interest area deformation field to the j-2-th level deformed image, and j is greater than or equal to 3; cascade the first-level non-interest area deformation field, the second-level non-interest area deformation field, and the j-th level non-interest area deformation field in sequence to obtain a multi-level cascaded non-interest area deformation field.

[0157] As an optional embodiment of the present invention, the device also includes a training module 50, and the training module 50 includes a registration model training module 501, wherein the registration model training module 501 is used to: repeatedly perform the following steps until the pre-trained first registration model is obtained when the first preset loss function takes a minimum value: input the first region of interest image corresponding to the moving image used for training and the second region of interest image corresponding to the fixed image used for training into the first registration model to be trained to obtain the region of interest deformation field to be trained; apply the region of interest deformation field to the first region of interest image to be trained to obtain the deformed first region of interest image; wherein the first preset loss function includes an image similarity loss function between the deformed first region of interest image and the second region of interest image, and a smoothness regularization function of the region of interest deformation field to be trained.

[0158] As an optional embodiment of the present invention, the registration model training module 501 is also used to: repeatedly perform the following steps until the pre-trained second registration model is obtained when the second preset loss function takes a minimum value: input the first non-interest area image corresponding to the moving image used for training and the second non-interest area image corresponding to the fixed image used for training into the second registration model to be trained to obtain the non-interest area deformation field to be trained; apply the non-interest area deformation field to be trained to the first non-interest area image to obtain the deformed first non-interest area image; wherein the second preset loss function includes an image similarity loss function between the deformed first non-interest area image and the second non-interest area image, and a smoothness regularization function of the non-interest area deformation field to be trained.

[0159] As an optional embodiment of the present invention, the training module 50 also includes a region perception module 502, and the region perception module 502 is used to: obtain a moving image for training and a fixed image for training; input the moving image for training and the fixed image for training into a pre-trained segmentation model to obtain a first segmented image corresponding to the moving image for training and a second segmented image corresponding to the fixed image for training; obtain the first region of interest image based on the moving image and the first segmented image, and obtain the second region of interest image based on the fixed image and the second segmented image; and / or obtain a first non-interest region image based on the moving image and the first region of interest image, and obtain a second non-interest region image based on the fixed image and the second region of interest image.

[0160] As an optional embodiment of the present invention, the region perception module 502 is also used to: train the segmentation model to be trained until the third preset loss function takes a minimum value to obtain the pre-trained segmentation model; wherein the first preset loss function is a negative log-likelihood function between the segmentation label and the predicted segmentation label.

[0161] As an optional embodiment of the present invention, the first registration model is a registration convolutional neural network model Reg-CNN; and / or, the second registration model is a registration distance neural network model Reg-CNN; and / or, the segmentation model is a segmentation convolutional neural network Seg-CNN model.

[0162] The implementation principle and technical effect of the image registration device provided in the embodiment of the present invention are similar to those of the above embodiment and will not be described in detail here.

[0163] like Fig.10As shown, an embodiment of the present invention provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0164] Memory 113, used for storing computer programs;

[0165] In one embodiment of the present invention, the processor 111 is used to implement the steps of the image registration method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 113.

[0166] The implementation principle and technical effect of the electronic device provided by the embodiment of the present invention are similar to those of the above embodiment and will not be described in detail here.

[0167] The above-mentioned memory 113 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. The memory 113 has a storage space for program codes for executing any method steps in the above-mentioned method. For example, the storage space for program codes may include various program codes for implementing the various steps in the above method respectively. These program codes can be read from or written into one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are usually portable or fixed storage units. The storage unit may have a storage segment or storage space arranged similarly to the memory 113 in the above-mentioned electronic device. The program code can be compressed, for example, in an appropriate form. Generally, the storage unit includes a program for executing the method steps according to an embodiment of the present disclosure, that is, a code that can be read by a processor such as 111, which, when run by an electronic device, causes the electronic device to execute the various steps in the method described above.

[0168] The embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned image registration method are implemented.

[0169] The computer-readable storage medium may be included in the device / apparatus described in the above embodiment; or it may exist independently without being assembled into the device / apparatus. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0170] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0171] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0172] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An image registration method, characterized in that: include: Acquire a moving image to be registered and a fixed image to be registered; Inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest; wherein the first registration model is obtained by training based on a first region of interest image corresponding to the moving image used for training and a second region of interest image corresponding to the fixed image used for training; Applying the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest; Registering the deformed image of the region of interest with the fixed image to be registered to obtain a registered image; Wherein, after applying the deformation field of the region of interest to the moving image to be registered to obtain the deformed image of the region of interest, the method further includes: Inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the non-interest region, wherein the second registration model is trained based on a first non-interest region image corresponding to the moving image used for training and a second non-interest region image corresponding to the fixed image used for training; Adjusting the deformed image of the region of interest according to the deformation field of the non-region of interest to obtain a final deformed image; The registering the deformed image of the region of interest and the fixed image to be registered to obtain a registered image includes: The final deformed image is registered with the fixed image to be registered to obtain a registered image.

2. The method according to claim 1, characterized in that The step of inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest includes: Inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a first-level region of interest deformation field; Obtaining a second-level region-of-interest deformation field according to the moving image to be registered and the first-level region-of-interest deformation image, wherein the first-level region-of-interest deformation image is obtained by applying the first-level region-of-interest deformation field to the moving image to be registered; Cascading the first-level region of interest deformation field and the second-level region of interest deformation field to obtain a multi-level cascaded region of interest deformation field; The step of applying the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest includes: The multi-stage cascaded region of interest deformation field is applied to the moving image to be registered to obtain a region of interest deformed image.

3. The method according to claim 2, characterized in that After obtaining the deformation field of the second-level region of interest according to the moving image to be registered and the deformed image of the first-level region of interest, the method further includes: Repeat the following steps until i satisfies a first preset value: obtaining an i-th level region of interest deformation field according to the moving image to be registered and the i-1th level region of interest deformation image, wherein the i-1th level region of interest deformation image is obtained by applying the i-1th level region of interest deformation field to the i-2th level region of interest deformation image, and i is greater than or equal to 3; The step of cascading the first-level region of interest deformation field and the second-level region of interest deformation field to obtain a multi-level cascaded region of interest deformation field includes: The first-level region of interest deformation field, the second-level region of interest deformation field, and the i-th level region of interest deformation field are sequentially cascaded to obtain a multi-level cascaded region of interest deformation field.

4. The method according to claim 1, characterized in that The step of inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the non-region of interest includes: Inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a first-level deformation field of the non-region of interest; Obtaining a second-level non-interest region deformation field according to the first-level deformed image and the fixed image to be registered, wherein the first-level deformed image is obtained by applying the first-level non-interest region deformation field to the interest region deformed image; Cascading the first-level non-interest region deformation field and the second-level non-interest region deformation field to obtain a multi-level cascaded non-interest region deformation field; The step of adjusting the deformed image of the region of interest according to the deformation field of the non-region of interest to obtain a final deformed image includes: The deformed image of the region of interest is adjusted according to the multi-stage cascaded deformation field of the non-region of interest to obtain a final deformed image.

5. The method according to claim 4, characterized in that After obtaining the second-level deformation field of the non-interest region according to the first-level deformed image and the fixed image to be registered, the method further includes: Repeat the following steps until j satisfies a second preset value: obtaining a j-th level non-interest region deformation field according to the j-1-th level deformed image and the fixed image to be registered, wherein the j-1-th level deformed image is obtained by applying the j-1-th level non-interest region deformation field to the j-2-th level deformed image, and j is greater than or equal to 3; The step of cascading the first-level non-interest region deformation field and the second-level non-interest region deformation field to obtain a multi-level cascaded non-interest region deformation field includes: The first-level non-interest region deformation field, the second-level non-interest region deformation field, and the j-th level non-interest region deformation field are sequentially cascaded to obtain a multi-level cascaded non-interest region deformation field.

6. The method according to claim 1, characterized in that Before inputting the moving image to be registered and the fixed image to be registered into the pre-trained first registration model, the method further includes: Repeat the following steps until the pre-trained first registration model is obtained when the value of the first preset loss function is minimized: input the first region of interest image corresponding to the moving image for training and the second region of interest image corresponding to the fixed image for training into the first registration model to be trained to obtain the region of interest deformation field to be trained; apply the region of interest deformation field to the first region of interest image to obtain the deformed first region of interest image; The first preset loss function includes an image similarity loss function between the deformed first region of interest image and the second region of interest image, and a smoothness regularization function of the region of interest deformation field to be trained.

7. The method according to claim 6, characterized in that Before inputting the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model, the method further includes: Repeat the following steps until the second preset loss function takes a minimum value to obtain the pre-trained second registration model: input the first non-interest region image corresponding to the moving image used for training and the second non-interest region image corresponding to the fixed image used for training into the second registration model to be trained to obtain the non-interest region deformation field to be trained; apply the non-interest region deformation field to the first non-interest region image to obtain the deformed first non-interest region image; The second preset loss function includes an image similarity loss function between the deformed first non-interest region image and the second non-interest region image, and a smoothness regularization function of the non-interest region deformation field to be trained.

8. The method according to claim 7, characterized in that The step of repeatedly performing the following steps until the pre-trained first registration model is obtained when the first preset loss function takes a minimum value further includes: Acquire a moving image for training and a fixed image for training; Inputting the moving image for training and the fixed image for training into a pre-trained segmentation model to obtain a first segmented image corresponding to the moving image for training and a second segmented image corresponding to the fixed image for training; Obtain the first region of interest image according to the moving image and the first segmented image, and obtain the second region of interest image according to the fixed image and the second segmented image; And / or, a first non-interest area image is obtained according to the moving image and the first interest area image, and a second non-interest area image is obtained according to the fixed image and the second interest area image.

9. The method according to claim 8, characterized in that Before inputting the moving image for training and the fixed image for training into the pre-trained segmentation model, the method further includes: Training the segmentation model to be trained until the third preset loss function takes a minimum value, thereby obtaining the pre-trained segmentation model; The first preset loss function is a negative log-likelihood function between the segmentation label and the predicted segmentation label.

10. The method according to claim 9, characterized in that The first registration model is a registration convolutional neural network model Reg-CNN; And / or, the second registration model is a registration convolutional neural network model Reg-CNN; And / or, the segmentation model is a segmentation convolutional neural network Seg-CNN model.

11. An image registration device, characterized in that: include: An acquisition module, used for acquiring a moving image to be registered and a fixed image to be registered; A deformation field module, used for inputting the moving image to be registered and the fixed image to be registered into a pre-trained first registration model to obtain a deformation field of the region of interest; wherein the first registration model is obtained by training based on a first region of interest image corresponding to the moving image used for training and a second region of interest image corresponding to the fixed image used for training; A deformation module, used for applying the deformation field of the region of interest to the moving image to be registered to obtain a deformed image of the region of interest; A registration module, used for registering the deformed image of the region of interest with the fixed image to be registered to obtain a registered image; The deformation field module is further used to: input the deformed image of the region of interest and the fixed image to be registered into a pre-trained second registration model to obtain a deformation field of the non-interest region, wherein the second registration model is obtained by training based on a first non-interest region image corresponding to the moving image used for training and a second non-interest region image corresponding to the fixed image used for training; The deformation module is used to adjust the deformed image of the interest region according to the deformation field of the non-interest region to obtain a final deformed image; The registration module is specifically used to register the final deformed image with the fixed image to be registered to obtain a registered image.

12. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is used to implement the steps of the image registration method described in any one of claims 1 to 10 when executing a program stored in a memory.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image registration method according to any one of claims 1 to 10 are implemented.

Images