Cross-modal 3D medical image registration method for prostate biopsy puncture

Abstract

The invention discloses a cross-modal 3D medical image registration method for prostate biopsy puncture. The method comprises the following steps: preprocessing acquired 3D image data, wherein the 3D image data comprises two modal images, namely a magnetic resonance imaging image and a rectal ultrasonic imaging image; for the 3D image data, utilizing a trained first network to segment the prostate region of interest of the two modal images, and obtaining the two modal images after segmentation and positioning; for the two modal images after segmentation and positioning, performing global structure alignment on the moving image through a non-rigid registration algorithm to obtain a pre-registered image pair; and taking the pre-registered image pair as input, and utilizing a trained second network as a fine registration network to predict a deformation field so as to obtain an image registration result. According to the invention, the accuracy of cross-modal medical image registration is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and more particularly to a cross-modality 3D medical image registration method for prostate biopsy puncture. BACKGROUND

[0002] In the diagnosis and treatment process of prostate tumors, medical images play an irreplaceable function. Among them, magnetic resonance imaging (MRI) and transrectal ultrasound (TRUS) imaging are common imaging techniques for prostate cancer diagnosis. MRI has excellent soft tissue resolution and multi-parameter imaging capability, which can clearly show the anatomical position and invasion range of tumors. TRUS is simple to operate, real-time imaging and low cost.

[0003] By fusing or jointly analyzing MRI and TRUS images, the advantages of MRI and TRUS can be fully utilized to improve the accuracy of prostate tumor detection, which has become an important direction of current research. However, the existing joint analysis method of MRI and TRUS images still has the problem of low accuracy of multi-modal image registration. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art, and to provide a cross-modality 3D medical image registration method for prostate biopsy puncture. The method comprises the following steps:

[0005] The collected 3D image data is preprocessed, and the 3D image data includes two modal images of magnetic resonance imaging and transrectal ultrasound imaging;

[0006] For the 3D image data, the first trained network is used to segment the prostate region of interest of the two modal images, and the segmented and positioned two modal images are obtained;

[0007] For the segmented and positioned two modal images, a non-rigid registration algorithm is used to align the global structure of the moving image, and a pre-registered image pair is obtained;

[0008] The pre-registered image pair is taken as input, and the second trained network is used as a fine registration network to predict the deformation field, and then the image registration result is obtained.

[0009] Compared with the prior art, the advantage of the present application is that a two-stage cross-modality medical image registration model based on Region of Interest (ROI) segmentation is proposed, which is specially used to solve the registration problem based on 3D MRI and 3D TRUS images involved in biopsy puncture of prostate-related diseases. The present application is aimed at the biopsy puncture operation of prostate-related diseases, and realizes the morphological structure alignment and fine deformation modeling of the prostate region in the MRI / TRUS images through the deep learning registration model of the cross-modality medical image sequence, which can better utilize the advantages of the traditional registration toolkit and the deep learning model, and effectively solve the multi-modality image registration problem.

[0010] Other features and advantages of the present application will become apparent from the following detailed description of illustrative embodiments thereof, which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0012] Figure 1 is a flow chart of a cross-modality 3D medical image registration method for prostate biopsy puncture according to an embodiment of the present application;

[0013] Figure 2 is a process schematic diagram of a cross-modality 3D medical image registration method for prostate biopsy puncture according to an embodiment of the present application;

[0014] Figure 3 is a structural schematic diagram of a fine registration network according to an embodiment of the present application;

[0015] Figure 4 is a segmentation result schematic diagram of an MRI image according to an embodiment of the present application;

[0016] Figure 5 is a segmentation result schematic diagram of a TRUS image according to an embodiment of the present application;

[0017] Figure 6 is a two-stage image registration result schematic diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated.

[0019] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the application or its application or uses.

[0020] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, such techniques, methods, and devices can be viewed as part of the specification.

[0021] In all of the compositions and methods described herein, any of the specific values stated can be replaced by alternative values of the same dimension.

[0022] It should be noted that like reference numerals and letters refer to like items throughout the several views of the drawings and that different items sharing the same reference numeral or letter should not be limited in scope to the common usage thereof.

[0023] The application proposes a cross-modal 3D medical image registration model for prostate biopsy puncture, which completes the registration task of prostate 3D MRI and 3D TRUS images through the combination of a prostate region of interest (ROI) segmentation network and a two-stage registration method. In the ROI segmentation part, the V-Net architecture network is used to realize accurate prostate anatomical region positioning; the two-stage registration utilizes the advantages of the non-rigid registration method SyN in the traditional registration toolkit and the deep learning model SMRNet, realizes the morphological structure alignment and fine deformation modeling of the prostate region in the MRI / TRUS image, and finally completes the registration task. The application efficiently integrates the imaging advantages of two modal images, improves the success rate of prostate tumor puncture surgery and the accuracy of disease diagnosis.

[0024] Overall, the application mainly involves the stages of ROI segmentation, pre-registration and fine registration. The ROI segmentation can use the V-Net architecture network to segment the ROI from the original 3D image, realize accurate prostate anatomical region positioning; the pre-registration stage can use the non-rigid registration algorithm SyN (Symmetric Normalization) in the ANTs toolkit to perform initial alignment of different modal images at the global structure level. The fine registration stage introduces the SMRNet (Salient Modality Registration Network) model based on convolutional neural network to perform local fine registration on the pre-registered prostate ROI.

[0025] Reference is made to Figure 1As shown, the provided cross-modality 3D medical image registration method for prostate biopsy puncture generally includes the following steps: S110, preprocessing the collected 3D image data, which includes two modal images of magnetic resonance imaging images and rectal ultrasound imaging images; S120, for the 3D image data, segmenting the prostate region of interest of the two modal images using a trained first network, obtaining the segmented and positioned two modal images; S130, for the segmented and positioned two modal images, performing global structure alignment on the moving images by a non-rigid registration algorithm, obtaining a pre-registered image pair; S140, taking the pre-registered image pair as input, using a trained second network as a fine registration network, predicting a deformation field, and then obtaining an image registration result.

[0026] Specifically, in combination with Figure 1 and Figure 2 As shown, the provided cross-modality 3D medical image registration method for prostate biopsy puncture includes the following steps:

[0027] S1, obtaining a 3D MRI and a 3D TRUS labeled data set of a prostate tumor patient and preprocessing to obtain a sample data set.

[0028] For example, the preprocessing of the data set includes the following steps:

[0029] S11, reading the pictures in the data set;

[0030] S12, uniformly format the 3D image data of the two modalities using center padding, for example, to (128, 128, 128, 1). Fill the label data to (128, 128, 128, 6) format, and adjust the image parameters such as voxel size, voxel spacing, and direction to ensure consistency.

[0031] S2, segment the prostate region of interest (ROI) of the preprocessed 3D image data of the two modalities using two pre-trained networks respectively, complete the positioning of the prostate region, and reduce the interference of irrelevant background parts.

[0032] The two networks can use the same type or different types of models, such as convolutional neural networks, recurrent neural networks, etc. In one embodiment, both networks use V-Net architecture, and the two pre-trained V-Net architecture networks are used for ROI segmentation, which includes the following steps:

[0033] S21, constructing a V-Net architecture network

[0034] For example, V-Net adopts a symmetrical encoder-decoder design, in which the encoder part is used to extract deep-level features in the input image step by step, and the decoder part is responsible for recovering the spatial resolution layer by layer to ensure that the output result has the same dimension as the input image, and combines the multi-level residual connection and jump connection mechanism to effectively fuse the low-level spatial details and high-level semantic information.

[0035] S22, training the V-Net architecture network

[0036] For example, supervised learning is performed by using the prostate region marked with labels to train the network to predict the mask of the prostate region of interest (ROI) in the image. In order to deal with the class imbalance problem commonly existing in medical images, V-Net introduces the Dice coefficient based on voxel-level overlap as the main loss function, so as to directly optimize the region consistency in the training process.

[0037] S23, evaluating and testing the trained V-Net architecture network

[0038] For example, the segmentation effect is evaluated and tested mainly by using DSC, Jaccard and other indicators, and finally the ROI segmentation result with high accuracy is obtained. In addition, due to the universality of the V-Net architecture network itself, two V-Net architecture networks with the same parameter settings can be used to train and segment two kinds of modal images respectively, and both of them obtain relatively accurate segmentation results.

[0039] The preprocessed image can obtain the segmentation mask (mask) of the prostate region of interest (ROI) through the pre-trained V-Net architecture network.

[0040] S3, pre-registration based on ROI segmentation.

[0041] On the basis of ROI segmentation, the MRI image is taken as the moving image and the TRUS image is taken as the fixed image, and the non-rigid registration algorithm SyN (Symmetric Normalization) in the traditional registration tool kit ANT is applied to complete the initial alignment at the global structure level, so as to realize the pre-registration.

[0042] S4, further fine registration of the pre-registered image.

[0043] For example, the SMRNet model based on the convolutional neural network is constructed as a fine registration network to further fine deformation on the pre-registered image. Specifically, the SMRNet first uses the design of the double-flow encoder and decoder to effectively extract specific features of each modality; meanwhile, the cross-modal spatial attention mechanism (CMSA) and the skip connection are added to effectively fuse the inter-modal feature information and the high-low layer feature information; finally, the fine registration deformation field is predicted, and the deformation field is applied to the moving image, so as to complete the fine registration.

[0044] In one embodiment, the fine registration specifically includes the following steps:

[0045] S41, constructing the SMRNet model as a fine registration network

[0046] For example, the pre-registered image pair is taken as input, the double-flow encoder and decoder design is used to extract the features of the two modal images layer by layer, and the cross-modal spatial attention mechanism (CMSA) is added after each layer of feature extraction to fuse the inter-modal feature information of the layer; the skip connection in the encoding-decoding architecture design fuses the high-level features and low-level features. In this way, in the decoding process of gradually restoring the resolution, the fine registration deformation field is predicted, the deformation field is applied to the pre-registered moving image through the STN (spatial transformation network) module, the registered image is obtained, and the structure of the fine registration network is as shown in Figure 3 .

[0047] S42, training the SMRNet model

[0048] For example, the 3D medical image fine registration network based on the SRMNet architecture is trained by using weakly supervised learning. The loss function mainly includes: salient region matching loss (SRML), weighted multi-class Dice loss, and regularization loss of deformation field. Through this design, on the one hand, the network is guided to focus on the prostate region of interest (ROI), and on the other hand, the structural information of the prostate is supplemented, and finally the non-real deformation of the deformation field is inhibited.

[0049] S43, evaluating and testing the SMRNet model

[0050] In the evaluation stage, the Dice coefficient is mainly used to evaluate the registration effect of multiple different class parts in the ROI, so that the model finally converges. In addition to the DSC index to measure the overall registration effect in the test, the TRE, stdJac, and negJac indexes are also added to measure the alignment deviation of the anatomical points and the abnormality of the deformation field.

[0051] To understand the present application, the construction, training and verification process of the cross-modal 3D medical image registration model for prostate biopsy puncture provided below is further completely described. Specifically, it includes the following steps:

[0052] Step 1. Preprocessing of 3D image datasets of two modalities

[0053] For example, the image parameters such as the size, voxel spacing, voxel size of 3D MRI and 3D TRUS image datasets are unified.

[0054] Step 2. ROI segmentation

[0055] The prostate region of interest (ROI) of two modalities is segmented using the trained V-Net architecture network, which completes the accurate positioning of the prostate ROI. The segmentation effect can be referred to Figure 4 、 Figure 5 . Figure 4 is the MRI image segmentation result, where the first column from left to right is the original MRI image, the second column is the predicted ROI mask, the third column is the predicted ROI, and the fourth column is the true ROI. Figure 5 is the TRUS image segmentation result, where the first column from left to right is the original TRUS image, the second column is the predicted ROI mask, the third column is the predicted ROI, and the fourth column is the true ROI.

[0056] Step 3. Perform pre-registration

[0057] After the segmentation and positioning of the two modalities described above, the non-rigid registration algorithm SyN in the traditional registration tool kit ANT is called to perform global structure alignment on the moving image, and the pre-registered image pair is obtained.

[0058] Step 4. Construct a fine registration network.

[0059] The fine registration network SMRNet takes the pre-registered image pair as input to predict the deformation field. For example, a dual-stream encoder-decoder design is used to extract features of two modalities image layer by layer. Each stream contains four down-sampling levels, each level consists of a convolution block, a CMSA (cross-modal spatial attention mechanism) module and a max-pooling layer. The convolution block includes a 3x3x3 convolution layer and a LeakyReLU activation layer. During feature extraction, the encoder first extracts initial features through C1=16 channels, and then doubles the number of channels at the second level while halving the spatial resolution of the feature map at each level. The CMSA module contains a ReLU activation function to introduce non-linear characteristics, and a Sigmoid activation function to normalize the attention value.

[0060] In one embodiment, the definition of the loss function involved in training the SMRNet model is represented as:

[0061]

[0062] where MI(X, Y) represents the mutual information between random variables X and Y, which measures the amount of shared information of intensity distribution between images; denotes the prostate ROI in the MR image; denotes the prostate ROI in the TRUS image. This loss function is based on the intensity and structural similarity within the prostate region of interest (ROI), and calculates the mutual information (MI) to guide the network to focus on the region of interest and reduce the interference of irrelevant anatomic information outside the prostate.

[0063] Since the mutual information mainly captures the similarity of intensity distribution between images, its direct guidance for structure alignment is limited. Therefore, a weighted multi-class Dice loss is introduced to supplement the structural information, which is defined as follows:

[0064]

[0065] where K is the total number of structure classes, such as prostate gland, lesions, etc.; P k is the structure mask of the kth class in the MR image; G k is the structure mask of the kth class in the TRUS image; w k represents the weight of the kth class, reflecting its importance in the registration process; |P k | and |G k | respectively represent the number of voxels of the kth structure mask; |P k ∩G k | represents the number of intersection voxels of the kth mask in MR and TRUS.

[0066] Finally, in order to suppress non-real deformation, a regularization loss of deformation field is also introduced, which is defined as follows:

[0067]

[0068] where φ represents the deformation field; v ∈ Ω represents that the voxel v belongs to the image volume Ω, i.e. the definition domain of the entire three-dimensional image; denotes the gradient of φ at voxel v; represents the square of the L2 norm of the gradient, which measures the smoothness of the point.

[0069] Combining the above three types of loss functions, the overall loss function is defined as follows:

[0070]

[0071] In addition to the three types of loss functions, λ is defined as the weight coefficient of the deformation field regularization term, which is used to adjust the influence degree of the regularization term on the total loss.

[0072] Step 5, model training and experimental verification

[0073] Taking the selected public dataset μ-RegPro as an example, the dataset is divided into a training set, a validation set and a test set, each containing 58, 7 and 8 pairs of registered MR-TRUS image pairs respectively. The initial learning rate is set to 0.0001; the total number of training rounds is 250; the batch size is 1; and the regularization term weight a is 0.4. In order to enhance the perception ability of the model in the key structure area, the prostate area is particularly emphasized in the loss calculation, and the weights of the three types of losses are dynamically adjusted to strengthen the foreground learning. In addition, a spatial transformer module (STN) is added to the training process to deform the label interpolation, which is used to maintain the coherence of the anatomical structure.

[0074] Step 6, analyze the model effect by statistical registration results

[0075] The DSC, TRE, stdJac and negJac indicators in the statistical registration results reflect the overall registration effect and the error of local anatomical point alignment of the model. The indicator results are shown in Table 1; then the registered image is visualized to intuitively show the registration results. The two-stage image registration results are shown in FIG. 1, wherein the first column from left to right is the original TRUS image (fixed), the second column is the original MRI image (moving), the third column is the MRI image after pre-registration, and the fourth column is the final MRI image after pre-registration and fine registration. Figure 6

[0076] Table 1: statistical indicators of registration results

[0077]

[0078] In each indicator of Table 1, (↑) indicates that a larger indicator value is better, and (↓) indicates that a smaller indicator value is better.

[0079] ​Correspondingly, the application also provides a cross-modality 3D medical image registration system for prostate biopsy, for implementing one or more aspects of the above method. For example, the system comprises: a preprocessing unit: for preprocessing the collected 3D image data, the 3D image data comprising two modal images of magnetic resonance imaging images and rectal ultrasound imaging images; an image segmentation unit: for segmenting the prostate region of interest of the two modal images by using a trained first network for the 3D image data, to obtain segmented two modal images; a pre-registration unit: for performing global structure alignment on the segmented two modal images by using a non-rigid registration algorithm, to obtain a pre-registered image pair; and a fine registration unit: for taking the pre-registered image pair as input, and using a trained second network as a fine registration network to predict a deformation field, and then obtaining an image registration result. The functions of each unit in the system can be implemented by using a general-purpose processor, a special-purpose processor, an FPGA, or a combination of software.

[0080] The application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0081] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a magneto-optical or other optical medium, and / or any suitable combination of the foregoing. Computer readable storage media is not a transitory signal per se, but is a computer or machine readable storage medium having properties associated with'storage' of information, such as an electrical, optical, or other signals that are capable of being stored.

[0082] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0083] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computing / processing device, partly on the user's computing / processing device, as a stand-alone software package, partly on the user's computing / processing device and partly on a remote computing / processing device or entirely on the remote computing / processing device or server. In the latter scenario, the remote computing / processing device can be connected to the user's computing / processing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing / processing device, for example, through the Internet using an Internet Service Provider. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0084] The computer readable program instructions can also be loaded onto a computing / processing device, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computing / processing device, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computing / processing device, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0085] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have instructions stored thereon or therein which may

[0086] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0087] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0088] Having described various embodiments of the application, it is to be understood that the above description is meant to be illustrative only and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art, without departing from the scope and spirit of the described embodiments. The selection of terms to be used in the description is intended to best explain the principles of the embodiments, the practical application, or technical improvement over the prior art, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the application is defined by the claims appended hereto.

Claims

1. A cross-modality 3D medical image registration method for prostate biopsy puncture, comprising the following steps: preprocessing the collected 3D image data, the 3D image data including two modality images of magnetic resonance imaging images and rectal ultrasound imaging images; for the 3D image data, segmenting the prostate region of interest of the two modalities of images by using a trained first network, to obtain the segmented and positioned two modality images; for the segmented and positioned two modality images, performing global structure alignment on the moving image by using a non-rigid registration algorithm, to obtain a pre-registered image pair; taking the pre-registered image pair as input, and using a trained second network as a fine registration network to predict a deformation field, and then obtaining an image registration result.

2. The method of claim 1, wherein, The first network is constructed based on a V-Net architecture, adopts a symmetrical encoder-decoder, and combines a multi-level residual connection and a jump connection mechanism, the encoder is used to extract deep-level features in the input image step by step, and the decoder is used to restore the spatial resolution layer by layer, so that the output result maintains the same dimension as the input image.

3. The method of claim 2, wherein, The first network uses the prostate region marked by a label for supervised learning to predict the mask of the prostate region of interest in the input image.

4. The method of claim 1, wherein, For the segmented and positioned two modality images, performing global structure alignment on the moving image by using a non-rigid registration algorithm includes: taking the magnetic resonance imaging image as the moving image, taking the rectal ultrasound imaging image as the fixed image, and using a non-rigid registration algorithm in a registration toolkit to perform global structure alignment on the moving image.

5. The method of claim 1, wherein, The preprocessing of the collected 3D image data includes: for the two modalities of images, unifying the parameters of the size, voxel spacing, and voxel size of the image data.

6. The method of claim 1, wherein, In the process of training the first network, a Dice coefficient based on voxel-level overlap is used to measure the loss value.

7. The method of claim 1, wherein, The second network is constructed using SMRNet, uses a double-flow encoder and decoder, each flow contains multiple down-sampling levels, each level contains a convolution block, a cross-modality spatial attention mechanism module, and a maximum pooling layer, in the feature extraction process, the encoder first extracts initial features through multiple channels, then the number of channels doubles at the second level, and the spatial resolution of each level feature map is halved, the cross-modality spatial attention mechanism module includes a ReLU activation function and a Sigmoid activation function.

8. The method of claim 1, wherein, The overall loss function employed to train the second network is set to: is set to: Wherein: wherein, is the saliency region matching loss, is the weighted multi-class Dice loss, is the regularization loss of the deformation field, and λ is the weight coefficient of the deformation field regularization term, denotes the prostate region of interest in the magnetic resonance imaging image, denotes the prostate region of interest in the rectal ultrasound imaging image, K is the total number of structure classes, P k is the structure mask of the k-th class in the magnetic resonance imaging image; G k is the structure mask of the k-th class in the rectal ultrasound imaging image; w k represents the weight of the k-th class, |P k | and |G k | respectively represent the number of voxels of the k-th structure mask, |P k ∩G k | represents the number of intersection voxels of the k-th mask in the magnetic resonance imaging image and the rectal ultrasound imaging image, φ represents the deformation field; v e Ω represents that the voxel v belongs to the image volume Ω, denotes the gradient of φ at the voxel v; represents the square of the L2 norm of the gradient, and MI represents mutual information.

9. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to realize the steps of the method according to any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, having stored on the memory a computer program capable of running on the processor, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 8.

Images