A method of cross-modality registration of CT images and MRI images

Abstract

The application belongs to the technical field of medical image processing, and particularly relates to a CT image and MRI image cross-modality registration method, which comprises the following steps: firstly, CT images and MRI images are coarsely registered, and then based on a coarse registration structure cost volume, features are matched, continuous down-sampling is performed in a fine registration network to extract features of multiple scales, decoding is performed in combination with cost volume matching features under each scale and joint features based on saliency map enhancement, a residual deformation field is finally obtained, the deformation field of the coarse registration network is corrected by using the residual deformation field to obtain a composite deformation field; in a spatial transformation network, the MRI image is spatially transformed based on the composite deformation field to obtain a fine registration MRI image which is spatially aligned with the CT image. The application can effectively improve the CT-MRI cross-modality registration accuracy, and maintain a low deformation folding rate and good deformation stability.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a cross-modal registration method for CT images and MRI images. Background Technology

[0002] In existing technologies, medical image registration is an important research direction in the field of medical image processing. Its goal is to establish spatial correspondences between medical images from different times, individuals, devices, or modalities. For cross-modal medical image registration between computed tomography (CT) and magnetic resonance imaging (MRI), CT images mainly reflect tissue density and bone structure information, while MRI images emphasize soft tissue contrast information. There are significant differences in grayscale distribution, tissue contrast, and local texture expression between the two modalities. Therefore, cross-modal non-rigid registration remains quite challenging.

[0003] Existing technologies have been used to study medical image registration from various aspects, including cross-modal structural descriptors, CNN unsupervised registration, Transformer long-distance relation modeling, coarse-to-fine pyramid registration, cross-modal attention, contrast-independent training, and local correlation modeling. These methods provide an important foundation for CT-MRI cross-modal registration, but they still have the following shortcomings related to this invention:

[0004] 1. Existing MIND-related methods mostly use MIND as a similarity measure or structural descriptor, and rarely utilize the MIND loss gradient after coarse registration to generate task-related multi-scale saliency maps.

[0005] 2. Existing deep learning registration networks typically rely on ordinary CNNs, Transformers, or attention features, which are easily affected by CT-MRI modal appearance differences, background response, and local mismatches.

[0006] 3. Although existing attention or correlation methods can enhance the feature interaction between image pairs, their attention area may not directly correspond to the current registration error sensitive area, and they do not explicitly distinguish between reliable positive sample areas, difficult mismatch areas and unreliable background areas.

[0007] 4. Complete 3D local matching search or complex attention calculation may result in high computational and memory overhead in 3D medical images.

[0008] Therefore, existing technologies still need a method suitable for cross-modal non-rigid registration in CT-MRI that can automatically locate key structural regions based on coarse registration using cross-modal structural errors, fuse CNN features and MIND structural features to construct lightweight local matching relationships, and enhance the cross-modal feature discrimination capability through saliency and MIND confidence constraints, thereby improving the matching accuracy of key anatomical structures and the stability of deformation fields in the fine registration stage. Summary of the Invention

[0009] To address the problems in existing technologies where significant differences in grayscale distribution, tissue contrast, and local structural expression between CT and MRI lead to inaccurate matching of key anatomical structures, unstable estimation of local deformation, and susceptibility of common feature representation to interference from modal appearance and background response, this invention proposes a cross-modal registration method for CT and MRI images. This method corrects the deformation field of the coarse registration network and uses the corrected deformation field to transform the MRI image. Specifically, it includes the following steps:

[0010] 101. Acquire the CT image to be registered. and MRI images And perform preprocessing;

[0011] 102. In the coarse registration network, the deformation field output by the trained coarse registration network is used to analyze MRI images. The transformation is performed to obtain the coarsely registered MRI image;

[0012] 102. Input the CT image and MRI image to be registered into the trained coarse registration network. The coarse registration network predicts the coarse deformation field and uses the coarse deformation field to perform spatial transformation on the MRI image to obtain the coarsely registered MRI image.

[0013] 103. The fine registration network utilizes separate CT images Continuous downsampling was performed on the coarsely registered MRI images to extract features at multiple scales;

[0014] 104. After the features of the deepest layer of the fine-fit registration network are concatenated, a convolution operation is performed through the bottleneck layer to obtain the bottleneck layer output. The subsequent cascaded decoding units upsample the features of the corresponding scale, the cost volume matching features, and the output of the previous decoding unit or the bottleneck layer to the current scale and then concatenate them. The decoding features of the current scale are obtained through the decoding convolution module.

[0015] 105. Map the output of the decoding module to a residual deformation field, and use the residual deformation field to correct the deformation field of the coarse registration network to obtain a composite deformation field.

[0016] 106. In spatial transformation networks, MRI images are processed based on composite deformation fields. Perform spatial transformation to obtain the CT image Spatially aligned, finely registered MRI images.

[0017] This invention generates a multi-scale gradient saliency map using the MIND structural error information after coarse registration, guiding the fine registration network to focus on key structural regions sensitive to cross-modal registration errors. Simultaneously, it combines CNN-MIND joint matching features, sparse local cost volumes, and saliency-MIND confidence triple constraints to improve the cross-modal local structural matching capability of CT-MRI. Furthermore, it maintains the smoothness and stability of the deformation field by combining residual deformation prediction with the deformation field. Experimental results show that this invention achieves registration results of Dice (0.7525±0.0452), HD95 (2.3107±0.6718), ASSD (0.7324±0.1979), and NJD (0.000148±0.000162) on the CT-MRI test set, indicating that this method can improve structural overlap accuracy while maintaining a low deformation folding rate and good deformation stability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the network structure used in the cross-modal registration method for CT and MRI images according to the present invention.

[0019] Figure 2 This is a schematic diagram of the patch-level triplet constraint construction used in this invention;

[0020] Figure 3 This is a comparison chart of the results of this invention with other registration methods on the Dice index;

[0021] Figure 4 This is a comparison chart of the ASSD index between the present invention and other registration methods;

[0022] Figure 5 This is a comparison chart of the HD95 index between the present invention and other registration methods. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] This invention proposes a cross-modal registration method for CT and MRI images. It corrects the deformation field of the coarse registration network and uses the corrected deformation field to transform the MRI image. Specifically, it includes the following steps:

[0025] 101. Acquire the CT image to be registered. and MRI images And perform preprocessing;

[0026] 102. In the coarse registration network, the deformation field output by the trained coarse registration network is used to analyze MRI images. The transformation is performed to obtain the coarsely registered MRI image;

[0027] 102. Input the CT image and MRI image to be registered into the trained coarse registration network. The coarse registration network predicts the coarse deformation field and uses the coarse deformation field to perform spatial transformation on the MRI image to obtain the coarsely registered MRI image.

[0028] 103. The fine registration network utilizes separate CT images Continuous downsampling was performed on the coarsely registered MRI images to extract features at multiple scales;

[0029] 104. After the features of the deepest layer of the fine-fit registration network are concatenated, a convolution operation is performed through the bottleneck layer to obtain the bottleneck layer output. The subsequent cascaded decoding units upsample the features of the corresponding scale, the cost volume matching features, and the output of the previous decoding unit or the bottleneck layer to the current scale and then concatenate them. The decoding features of the current scale are obtained through the decoding convolution module.

[0030] 105. Map the output of the decoding module to a residual deformation field, and use the residual deformation field to correct the deformation field of the coarse registration network to obtain a composite deformation field.

[0031] 106. In spatial transformation networks, MRI images are processed based on composite deformation fields. Perform spatial transformation to obtain the CT image Spatially aligned, finely registered MRI images.

[0032] In a specific implementation of this invention, the coarse registration network and the fine registration network are encoder-decoder architectures with the same number of structural layers. For example, a learning-based deformable registration network based on a three-dimensional U-Net encoder-decoder structure can be used. The coarse registration network is used to predict the initial coarse deformation field based on the input CT and MRI images. The fine registration network is used to predict the residual deformation field based on the joint matching features of the CT image and the coarsely registered MRI image, as well as the sparse local cost volume features. Finally, the initial coarse deformation field and the residual deformation field are combined to perform spatial transformation on the MRI image to obtain the final registered image. This invention does not limit the specific structure of the coarse registration network, as long as it can output a coarse deformation field for initial spatial alignment.

[0033] like Figure 1This embodiment provides a schematic diagram of a saliency-guided fine registration network structure. The diagram illustrates the overall process from CT image, coarsely registered MRI image, pre-generated saliency map in the coarse registration stage, saliency feature enhancement, joint feature construction, decoder, multi-scale sparse local cost volume construction, and residual deformation field solution. In this embodiment, the CT image is used as the fixed image, denoted as... Using MRI images as floating images, denoted as A two-stage registration framework, from coarse to fine, is adopted. First, the initial deformation field is obtained through a coarse registration network. Then, the residual deformation field is predicted through a fine registration network guided by gradient significance. Finally, the finely registered deformation field was obtained. .

[0034] In this embodiment, the features at each scale include CT image features, MRI image features, and cost volume matching features. The process of obtaining cost volume matching features includes:

[0035] 201. The CT image and the coarsely registered MRI image are respectively input into two image encoders in the fine registration network. Each image encoder extracts basic features at multiple scales through continuous convolution and downsampling operations. At the same time, MIND structural descriptors are calculated based on the CT image and the coarsely registered MRI image, and the MIND structural descriptors are adjusted to the same spatial size and channel dimension as the basic features at each scale through pooling and convolution mapping operations to obtain the MIND structural features at the corresponding scale. Then, saliency maps are used to enhance the basic features and MIND structural features at each scale, and the joint feature fusion convolution module is used to obtain the joint matching features of the CT image and the joint matching features of the MRI image, respectively.

[0036] 202. The dimension reduction matching convolution module is used to perform channel dimension reduction on the joint matching features of CT images and the joint matching features of MRI images, and then normalization is performed to obtain the dimension reduction matching features of CT images and the dimension reduction matching features of MRI images.

[0037] 203. Using the dimensionality reduction matching features of CT images as a reference, obtain the similarity between each pixel of the dimensionality reduction matching features of CT images and the corresponding pixel of the dimensionality reduction matching features of MRI images and their candidate displacement positions, and obtain sparse local cost volume features.

[0038] 204. The sparse local cost volume features are compressed using the cost volume compression convolution module to obtain cost volume matching features.

[0039] In this embodiment, the joint matching features of CT images are represented as follows:

[0040] ;

[0041] The joint matching features of MRI images are represented as follows:

[0042] ;

[0043] in, This represents the joint matching features corresponding to the CT image during the l-th downsampling process in the continuous downsampling process; Indicates the first A convolutional module for processing CT images at various scales. This represents the convolution module used to process MRI images at the l-th scale; Indicates a splicing operation; This represents the basic features of the CT image obtained by the l-th downsampling in the continuous downsampling process; Indicates and MIND structural features in scale-consistent CT images; express Enhanced features; express Enhanced features; This represents the joint matching feature corresponding to the MRI image at the l-th downsampling stage in the continuous downsampling process; This represents the basic features of the MRI image obtained by the l-th downsampling in the continuous downsampling process; Indicates and MIND structural features in scale-consistent MRI images; express Enhanced features; express Enhanced features.

[0044] This embodiment obtains and Then, channel dimensionality reduction is performed on both to reduce the computational cost of subsequent correlation calculations:

[0045]

[0046]

[0047] in, for Dimensionality reduction characteristics; for Dimensionality reduction characteristics; Representing three dimensions Convolution operation.

[0048] Subsequently, channel normalization is performed on the dimensionality-reduced matching features:

[0049]

[0050]

[0051] in, for Normalization characteristics; for Normalization characteristics; Represents the L2 norm, This represents a very small constant that prevents division by zero.

[0052] In this embodiment, the first The set of sparse candidate displacements for a layer is defined as follows:

[0053]

[0054] in, This indicates that the l-th layer is located at The set of candidate displacements for pixels. This represents the search radius of the l-th layer, which is an integer greater than or equal to 0; Indicates the axial offset step size, if 0 indicates that the candidate displacement set only includes those located at... Pixels, only when An axial offset step size only exists when the value is greater than 0. Indicates the current pixel position. , , These represent the positive and negative candidate offsets along the three-dimensional coordinate axes, respectively. The meaning of this formula is that the sparse local cost volume does not enumerate the complete three-dimensional cubic window, but only retains the center point and a finite number of candidate displacements along the three coordinate axes, thereby reducing the computational load and memory overhead in three-dimensional medical image registration.

[0055] The number of candidate displacements in the l-th layer is:

[0056]

[0057] in, This represents the number of candidate displacements. The formula means that the candidate set consists of a central position and candidate positions along the positive and negative directions of the three coordinate axes; therefore, the number of candidate displacements is... .

[0058] Using the spatial location of the CT image as a reference, feature correlations are calculated within the neighboring candidate locations of the MRI image to obtain a sparse local cost volume:

[0059]

[0060] in, This represents the k-th pixel in the candidate set corresponding to the p-th pixel of the CT image and the p-th pixel of the MRI image in the l-th layer features. The sparse local cost volume between pixels; the sparse local cost volume of a feature map is composed of the sparse local cost volumes of all its pixels. The sparse local cost volume of each pixel is composed of the sparse local cost volumes between each pixel in the candidate set corresponding to the pixel position in the CT image and the pixel position in the MRI image.

[0061] This embodiment uses a saliency map to enhance the features, resulting in the corresponding enhanced features, including:

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] in, This indicates element-wise multiplication; Indicates the significance enhancement coefficient; This represents the saliency map corresponding to the CT image at the l-th downsampling stage during continuous downsampling. This represents the MRI image at the lth downsampling stage during continuous downsampling. The corresponding saliency plot.

[0067] As an optional embodiment, the process of obtaining the saliency image includes:

[0068] CT images MIND structural features and coarsely registered MRI images The L1 norm of the difference between the structural features of MIND is used as the cross-modal structural error of MIND in the coarse registration stage, and is expressed as:

[0069]

[0070] in, This represents the computational operation of the MIND structure descriptor; This indicates the MIND structural error under coarse registration results; Represents the norm distance;

[0071] The gradients of the MIND cross-modal structural error at each scale of the CT and MRI images during the coarse registration stage are calculated to obtain the gradient responses at the corresponding scales, i.e.:

[0072]

[0073]

[0074] in, This represents the intermediate features of the CT image at the l-th scale in the coarse registration network. The corresponding gradient response, This indicates the partial derivative; This represents the intermediate features of the MRI image at the l-th scale in the coarse registration network. The corresponding gradient response;

[0075] The gradient responses at each scale were processed by absolute value processing, channel aggregation, and normalization to obtain gradient significance maps for CT images and MRI images, respectively.

[0076] Gradient saliency maps of each MRI size are transformed to CT image space using deformation fields;

[0077] Perform a stop gradient propagation operation on the gradient saliency map of the CT image at the current scale and the gradient saliency map of the MRI image converted to the CT image space to obtain the corresponding saliency images.

[0078] As an optional implementation, the loss function of the coarse registration network during training in this embodiment is expressed as follows:

[0079] ;

[0080] in, The loss function of the coarse registration network is represented. This represents the Dice structure overlap loss during the coarse registration stage; This represents the MIND structural similarity loss during the coarse registration stage; This represents the coarse deformation field smoothing regularization loss; , , These represent the weights of the corresponding loss terms.

[0081] Figure 2 The process of constructing the saliency-confidence map, generating positive and negative sample weights, extracting weighted patch descriptors, screening difficult negative samples, and calculating triplet constraints is presented. In the fine-registration network training process of this invention, a patch-level triplet loss term is used, which is expressed as:

[0082] ;

[0083] in, This represents the triplet loss of the g-th patch in the l-th layer; This indicates finding the maximum value; This represents the similarity of the most difficult negative sample in the g-th patch of the l-th layer; This represents the similarity between the CT positive sample descriptor and the MRI positive sample descriptor in the g-th patch of the l-th layer; This indicates the triplet interval.

[0084] In this invention, the most difficult negative sample similarity of the g-th patch in the l-th layer. Represented as:

[0085]

[0086] The similarity between CT positive sample descriptors and MRI positive sample descriptors in the g-th patch of layer l is represented as follows:

[0087]

[0088] in, This represents the similarity calculation function; Represents the set of negative samples. Represents the set of negative samples Any negative sample in the sample; This represents the CT positive sample descriptor in the g-th patch of layer l; This represents the MRI positive sample descriptor in the g-th patch of the l-th layer.

[0089] Specifically, the process of obtaining positive and negative sample descriptors includes:

[0090] Combine the matching features of the CT images at the l-th scale Joint matching features with MRI images aligned with residual deformation fields The area is divided into multiple non-overlapping 3D local patches using the same spatial grid. Let the spatial region corresponding to the g-th patch be denoted as . ;

[0091] CT saliency map Aligned MRI saliency map Element-wise multiplication is performed, and the results are normalized to obtain the cross-modal common salient regions. , represented as:

[0092]

[0093] The above formula means that only regions in both CT and MRI images that are sensitive to MIND structural errors will receive a high response in the common saliency map;

[0094] The local MIND error after coarse registration is converted into a confidence map. The confidence map is then transformed to the same size as the cross-modal common salient region to obtain the MIND confidence map at the l-th scale, as follows:

[0095]

[0096] in, Indicates spatial location Local MIND error at the location, Indicates the location of CT images The MIND structure description at that location. Indicates the pixel location of the MRI image after coarse registration. The formula represents the difference between coarsely registered CT and MRI images in the local structural description space.

[0097] Convert local MIND errors into confidence levels:

[0098]

[0099] in, Indicates pixel position MIND confidence level at the location. The temperature coefficient that controls the range of confidence level variation. The confidence level represents the exponential function. The smaller the local MIND error, the more consistent the local structures of CT and MRI are after coarse registration, and the higher the confidence level.

[0100] The confidence map at the original resolution is adjusted to a scale consistent with the features of layer l in order to construct pointwise weights. The MIND confidence map is then downsampled to the scale of the features of layer l.

[0101]

[0102] in, This represents the MIND confidence plot for the l-th layer. This indicates a downsampling operation. This represents the spatial size of the common saliency map of the l-th layer.

[0103] The MIND confidence map at the l-th scale of each patch is multiplied sequentially by the cross-modal common salient region, and then normalized to obtain the reliable positive sample weights at the l-th scale. Based on the weight and the joint matching features of each pixel in the current patch of the CT image, a positive sample descriptor for the CT image is constructed; based on the weight and the joint matching features of each pixel in the current patch of the MRI image, a positive sample descriptor for the MRI image is constructed.

[0104] The cross-modal common salient region at the l-th scale of each patch and Element-wise multiplication is performed, followed by normalization, to obtain the weights of the first type of negative samples. Based on this weight and the joint matching features of each pixel in the current patch of the MRI image, a first-class negative sample descriptor is constructed.

[0105] use The joint matching features are weighted to obtain the background or unreliable region response. Then, the background or unreliable region response is normalized to obtain the second type of negative sample weight. Based on the weight and the joint matching features of each pixel in the current patch of the MRI image, the second type of negative sample descriptor is constructed.

[0106] Construct a third type of negative sample descriptor for the current patch based on the positive sample descriptors of MRI images from other patches.

[0107] In this embodiment, the positive sample descriptor of the CT image is represented as:

[0108] ;

[0109] The positive sample descriptor for an MRI image is represented as follows:

[0110] ;

[0111] The first-class negative sample descriptor of an MRI image is represented as:

[0112] ;

[0113] The second type of negative sample descriptor for MRI images is represented as:

[0114] ;

[0115] The third type of negative sample descriptor for MRI images is represented as follows:

[0116] ;

[0117] in, Represents the l-th scale. Positive sample descriptors of CT images for each patch; This represents the positive sample descriptor of the MRI image at the l-th scale and the g-th patch; This represents the set of pixels in the current g-th patch at the l-th scale; This represents the weight of a reliable positive sample for pixel p at the l-th scale; This represents the joint matching feature of pixel p in the CT image at the l-th scale in the fine registration network; To prevent the minimum value where the denominator is 0; This represents the first-class negative sample descriptor of the MRI image of the current g-th patch at the l-th scale; This represents the weight of the first type of negative sample for pixel p at the l-th scale; This represents the joint matching feature of pixel p in the MRI image at the l-th scale in the fine registration network; This represents the second-type negative sample descriptor of the MRI image of the current g-th patch at the l-th scale; This represents the weight of the second type of negative sample for pixel p at the l-th scale; This represents the third-class negative sample descriptor of the MRI image of the current g-th patch at the l-th scale; This represents the set of pixels from all patches except the current one.

[0118] The fine registration network of this invention adopts a 4-layer structure (i.e., l∈{1,2,3,4}). The features of the deepest layer are concatenated and then convolved through a bottleneck layer to obtain the bottleneck layer output, which is represented as:

[0119]

[0120] The subsequent decoding unit uses a step-by-step upsampling and fusion process, which can be represented as follows:

[0121]

[0122]

[0123]

[0124] Final output residual deformation field:

[0125]

[0126] in, This represents the residual deformation field predicted by the fine registration network. This indicates the output convolutional layer.

[0127] The final deformation field is obtained through deformation field composite:

[0128]

[0129] in, This represents the final fine-fit deformation field. This indicates a composite operation of the deformation field. Represents a coarse deformation field. This represents the residual deformation field.

[0130] The final registered image is represented as follows:

[0131]

[0132] in, This represents the final, finely registered MRI image. Represents a spatial transformation network.

[0133] Figure 3 This figure shows the comparison results of the method of this invention with other registration methods in terms of the Dice index. The Dice index is used to evaluate the degree of structural overlap between the registered CT label and the transformed MRI label. The larger the value, the higher the structural overlap and the better the registration effect. In the figure, the horizontal bars represent the mean Dice value of different methods, and the error bars represent the standard deviation. The method of this invention is represented by Ours. As can be seen from the figure, the Dice index of this method is the highest.

[0134] Figure 4 This figure shows the comparison results of the method of this invention with other registration methods in terms of ASSD index. ASSD stands for Average Symmetric Surface Distance, which is used to evaluate the average distance error between registered structural surfaces. The smaller the value, the closer the structural surfaces are and the better the consistency of the registration boundary. The horizontal bars in the figure represent the mean ASSD of different methods, and the error bars represent the standard deviation. The method of this invention is represented by Ours. Compared with other registration methods, the figure shows that the ASSD of this method is better than most of the comparison methods, indicating that this method can achieve better structural surface registration consistency.

[0135] Figure 5 This figure compares the HD95 index of the method of this invention with other registration methods. HD95 represents 95% Hausdorff Distance, used to evaluate the distance error between structural boundaries after registration. The smaller the value, the smaller the boundary matching error and the higher the registration accuracy. The horizontal bars in the figure represent the mean HD95 of different methods, and the error bars represent the standard deviation. The method of this invention is represented by Ours and compared with other registration methods. As can be seen from the figure, the HD95 of this method is the smallest.

[0136] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A cross-modal registration method for CT images and MRI images, characterized in that, The deformation field of the coarse registration network is corrected, and the MRI image is transformed using the corrected deformation field. This process includes the following steps:

101. Acquire the CT images to be registered. and MRI images And perform preprocessing; 102. Input the CT image and MRI image to be registered into the trained coarse registration network. The coarse registration network predicts the coarse deformation field and uses the coarse deformation field to perform spatial transformation on the MRI image to obtain the coarsely registered MRI image.

103. The fine registration network utilizes separate CT images The MRI image after coarse registration is continuously downsampled to extract features at multiple scales; 104. The features of the deepest layer of the fine registration network are concatenated and then convolved through the bottleneck layer to obtain the bottleneck layer output; the subsequent cascaded decoding units upsample the features of the corresponding scale, the cost volume matching features, and the output of the previous decoding unit or the bottleneck layer to the current scale and then concatenate them, and obtain the decoding features of the current scale through the decoding convolution module.

105. Map the output of the decoding module to a residual deformation field, and use the residual deformation field to correct the deformation field of the coarse registration network to obtain a composite deformation field.

106. In spatial transformation networks, MRI images are processed based on composite deformation fields. Perform spatial transformation to obtain the CT image Spatially aligned, finely registered MRI images.

2. The cross-modal registration method for CT images and MRI images according to claim 1, characterized in that, Features at each scale include CT image features, MRI image features, and cost volume matching features. The process of obtaining cost volume matching features includes:

201. The CT image and the coarsely registered MRI image are respectively input into two image encoders in the fine registration network. Each image encoder extracts basic features at multiple scales through continuous convolution and downsampling operations. At the same time, MIND structural descriptors are calculated based on the CT image and the coarsely registered MRI image, and the MIND structural descriptors are adjusted to the same spatial size and channel dimension as the basic features at each scale through pooling and convolution mapping operations to obtain the MIND structural features at the corresponding scale. Then, saliency maps are used to enhance the basic features and MIND structural features at each scale, and the joint feature fusion convolution module is used to obtain the joint matching features of the CT image and the joint matching features of the MRI image, respectively.

202. The dimension reduction matching convolution module is used to perform channel dimension reduction on the joint matching features of CT images and the joint matching features of MRI images, and then normalization is performed to obtain the dimension reduction matching features of CT images and the dimension reduction matching features of MRI images.

203. Using the dimensionality reduction matching features of CT images as a reference, obtain the similarity between each pixel of the dimensionality reduction matching features of CT images and the corresponding pixel of the dimensionality reduction matching features of MRI images and their candidate displacement positions, and obtain sparse local cost volume features.

204. The sparse local cost volume features are compressed using the cost volume compression convolution module to obtain cost volume matching features.

3. The cross-modal registration method for CT images and MRI images according to claim 2, characterized in that, The joint matching features of CT images are represented as follows: ； The joint matching features of MRI images are represented as follows: ； in, This represents the joint matching features corresponding to the CT image during the l-th downsampling process in the continuous downsampling process; This represents the convolution module used to process CT images at the l-th scale. This represents the convolution module used to process MRI images at the l-th scale; Indicates a splicing operation; This represents the basic features of the CT image obtained by the l-th downsampling in the continuous downsampling process; Indicates and MIND structural features in scale-consistent CT images; express Enhanced features; express Enhanced features; This represents the joint matching feature corresponding to the MRI image at the l-th downsampling stage in the continuous downsampling process; This represents the basic features of the MRI image obtained by the l-th downsampling in the continuous downsampling process; Indicates and MIND structural features in scale-consistent MRI images; express Enhanced features; express Enhanced features.

4. The cross-modal registration method for CT images and MRI images according to claim 3, characterized in that, The features are enhanced using saliency maps, resulting in the corresponding enhanced features, including: ； ； ； ； in, This indicates element-wise multiplication; Indicates the significance enhancement coefficient; This represents the saliency map corresponding to the CT image at the l-th downsampling stage during continuous downsampling. This represents the MRI image at the lth downsampling stage during continuous downsampling. The corresponding saliency plot.

5. The cross-modal registration method for CT images and MRI images according to claim 4, characterized in that, The process of obtaining a saliency image includes: CT images MIND structural features and coarsely registered MRI images The L1 norm of the difference between the structural features of MIND is used as the cross-modal structural error of MIND in the coarse registration stage. The gradient of the MIND cross-modal structural error is calculated with respect to the features of CT and MRI images at various scales during the coarse registration stage, and the gradient response at the corresponding scale is obtained. The gradient responses at each scale were processed by absolute value processing, channel aggregation, and normalization to obtain gradient significance maps for CT images and MRI images, respectively. Using the coarse deformation field predicted by the coarse registration network, the gradient saliency maps of MRI images at various scales are transformed to the CT image space; Perform a stop gradient propagation operation on the gradient saliency map of the CT image at the current scale and the gradient saliency map of the MRI image converted to the CT image space to obtain the corresponding saliency images.

6. The cross-modal registration method for CT images and MRI images according to claim 1, characterized in that, The loss function for training a coarse registration network is expressed as: ； in, The loss function of the coarse registration network is represented. This represents the Dice structure overlap loss during the coarse registration stage; This represents the MIND structural similarity loss during the coarse registration stage; This represents the coarse deformation field smoothing regularization loss; , , These represent the weights of the corresponding loss terms.

7. The cross-modal registration method for CT images and MRI images according to claim 5, characterized in that, During the training of the fine-registration network, a patch-level triplet loss term is used, which is expressed as: ； in, This represents the triplet loss of the g-th patch in the l-th layer; This indicates finding the maximum value; This represents the similarity of the most difficult negative sample in the g-th patch of the l-th layer; This represents the similarity between the CT positive sample descriptor and the MRI positive sample descriptor in the g-th patch of the l-th layer; This indicates the triplet interval.

8. The cross-modal registration method for CT images and MRI images according to claim 7, characterized in that, The most difficult negative sample similarity of the g-th patch in layer l Represented as: ； No. Layer The similarity between CT positive sample descriptors and MRI positive sample descriptors in each patch is represented as follows: ； in, This represents the similarity calculation function; Represents the set of negative samples. Represents the set of negative samples Any negative sample in the sample; This represents the CT positive sample descriptor in the g-th patch of layer l; This represents the MRI positive sample descriptor in the g-th patch of the l-th layer.

9. A cross-modal registration method for CT images and MRI images according to claim 8, characterized in that, The process of obtaining positive and negative sample descriptors includes: Combine the matching features of the CT images at the l-th scale Joint matching features with MRI images aligned with residual deformation fields The area is divided into multiple non-overlapping 3D local patches using the same spatial grid. Let the spatial region corresponding to the g-th patch be denoted as . ; CT saliency map Aligned MRI saliency map Element-wise multiplication is performed, and the results are normalized to obtain the cross-modal common salient regions. ; The local MIND error after coarse registration is converted into a confidence map. The confidence map is then transformed to the same size as the cross-modal common salient region to obtain the MIND confidence map at the l-th scale; The MIND confidence map at the l-th scale of each patch is multiplied sequentially by the cross-modal common salient region, and then normalized to obtain the reliable positive sample weights at the l-th scale. Based on the weight and the joint matching features of each pixel in the current patch of the CT image, a positive sample descriptor for the CT image is constructed; based on the weight and the joint matching features of each pixel in the current patch of the MRI image, a positive sample descriptor for the MRI image is constructed. The cross-modal common salient region at the l-th scale of each patch and Element-wise multiplication is performed, followed by normalization, to obtain the weights of the first type of negative samples. Based on this weight and the joint matching features of each pixel in the current patch of the MRI image, a first-class negative sample descriptor is constructed. use The joint matching features are weighted to obtain the background or unreliable region response. Then, the background or unreliable region response is normalized to obtain the second type of negative sample weight. Based on the weight and the joint matching features of each pixel in the current patch of the MRI image, the second type of negative sample descriptor is constructed. Construct a third type of negative sample descriptor for the current patch based on the positive sample descriptors of MRI images from other patches.

10. A cross-modal registration method for CT images and MRI images according to claim 9, characterized in that, The positive sample descriptor for a CT image is represented as follows: ； The positive sample descriptor for an MRI image is represented as follows: ； The first-class negative sample descriptor of an MRI image is represented as: ； The second type of negative sample descriptor for MRI images is represented as: ； The third type of negative sample descriptor for MRI images is represented as follows: ； in, This represents the positive sample descriptor of the CT image in the g-th patch at the l-th scale; This represents the positive sample descriptor of the MRI image for the g patches at the l-th scale; This represents the set of pixels in the current g-th patch at the l-th scale; This represents the weight of a reliable positive sample for pixel p at the l-th scale; This represents the joint matching feature of pixel p in the CT image at the l-th scale in the fine registration network; To prevent the minimum value where the denominator is 0; This represents the first-class negative sample descriptor of the MRI image of the current g-th patch at the l-th scale; This represents the weight of the first type of negative sample for pixel p at the l-th scale; This represents the joint matching feature of pixel p in the MRI image at the l-th scale in the fine registration network; This represents the second-type negative sample descriptor of the MRI image of the current g-th patch at the l-th scale; This represents the weight of the second type of negative sample for pixel p at the l-th scale; This represents the third-class negative sample descriptor of the MRI image of the current g-th patch at the l-th scale; This represents the set of pixels from all patches except the current g-th patch.

Images