Multi-scale implicit-explicit hybrid medical image registration method and system

Abstract

The invention provides a multi-scale implicit-explicit mixed medical image registration method and system. The method comprises the following steps: acquiring CT images of a moving image and a fixed image; processing the CT image by using the trained registration network model, performing down-sampling on a three-dimensional coordinate grid to different set proportions of an original resolution, and then predicting deformation of different scales by using a multi-layer perceptron network model to generate a first deformation field and a second deformation field; down-sampling the fixed image and the moving image to a third set proportion of the original resolution, predicting third scale deformation by using the CNN network model, and generating a third deformation field; and fusing the three deformation fields to obtain a final deformation field, and generating a final deformation image through a spatial transformation network. The method is simple in registration process, and has good practical application efficiency and expandability.

Description

Technical Field

[0001] This invention belongs to the field of image registration technology, specifically relating to a multi-scale implicit-explicit hybrid medical image registration method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of medical imaging, Deformable Image Registration (DIR) technology integrates images acquired under different conditions into a unified reference frame through advanced spatial mapping algorithms, which is crucial for accurate diagnosis, treatment planning, and monitoring in the medical process.

[0004] Compared to other organs, the lungs' unique high deformability and complex motion patterns place higher demands on image registration technology: the strong deformability of the lungs requires that registration take into account a variety of deformation scenarios from large to small areas, which poses a challenge to the flexibility and adaptability of the algorithm; while the complex motion patterns place higher demands on the accuracy of the registration results, which may lead to problems such as inconsistent deformation or excessive deformation.

[0005] Traditional image registration algorithms typically involve three steps: (1) selecting a transformation model, (2) measuring the difference between the fixed image and the deformed image, and (3) updating the model parameters through optimization methods, and iteratively repeating steps (2) and (3) until the convergence condition is met. However, traditional methods are often time-consuming and inefficient.

[0006] Supervised learning methods use real deformation fields to train neural networks to predict the deformation between the target image and the source image. However, these deformation fields are usually generated by traditional methods, which limits the upper limit of the accuracy of supervised methods. Unsupervised learning methods rely on large datasets to train models. Although they are faster than traditional methods after training, they often sacrifice accuracy, and the lack of suitable datasets remains a key bottleneck.

[0007] Implicit Neural Representation (INR) technology uses the 3D coordinates of the source image as input and outputs the corresponding displacement vector to continuously model image deformation. It naturally captures complex nonlinear deformations, avoids information loss in traditional discretization methods, and generates a smooth and continuous deformation field, thus aligning the source and target images more accurately. However, existing INR-based registration methods still have significant shortcomings: First, these methods are not fully end-to-end and highly dependent on mask accuracy, limiting their robustness in practical applications; second, existing methods lack specific designs for large deformation registration, making it difficult to handle extreme deformation scenarios; third, how to effectively preserve and process details in medical image registration using INR remains a core challenge, especially in high-resolution data, where detail loss may negatively impact registration accuracy.

[0008] In summary, existing methods typically require large-scale datasets for training, and data scarcity remains a key bottleneck. Furthermore, they cannot simultaneously model large deformations and preserve fine-grained structural details. The registration process is also quite complex and computationally expensive, which limits the efficiency and scalability of practical applications. Summary of the Invention

[0009] To address the aforementioned problems, this invention proposes a multi-scale implicit-explicit hybrid medical image registration method and system. The registration process of this invention is simple and has good practical application efficiency and scalability.

[0010] According to some embodiments, the present invention adopts the following technical solution: A multi-scale implicit-explicit hybrid medical image registration method includes the following steps: Acquire CT images of both moving and stationary images; The trained registration network model is used to process CT images, specifically including: After downsampling the three-dimensional coordinate grid to different set ratios of the original resolution, the deformation at different scales is predicted using a multilayer perceptron network model, generating the first and second deformation fields. After downsampling the fixed image and the moving image to a third predetermined ratio of the original resolution, the third-scale deformation is predicted using a CNN network model to generate the third deformation field. The three deformation fields are fused to obtain the final deformation field, and the final deformation image is generated through a spatial transformation network.

[0011] As an alternative implementation, the process of downsampling the three-dimensional coordinate grid to different predetermined ratios of the original resolution, and then using a multilayer perceptron network model to predict deformations at different scales to generate the first and second deformation fields includes: After downsampling the three-dimensional coordinate grid to a first set ratio of the original resolution, the first-scale deformation is predicted using a multilayer perceptron network model to generate the first deformation field. After downsampling the three-dimensional coordinate grid to a second predetermined scale of the original resolution, another multilayer perceptron network model is used to predict the second-scale deformation and generate the second deformation field.

[0012] As an alternative implementation, the first scale is larger than the second scale, the second scale is larger than the third scale, the first set ratio is smaller than the second set ratio, and the second set ratio is smaller than the third set ratio.

[0013] As an alternative implementation, the two multilayer perceptron network models have the same structure, each containing four hidden layers with 512 hidden units per layer.

[0014] As an alternative implementation, the process of downsampling the three-dimensional coordinate grid to different predetermined ratios of the original resolution, and then using a multilayer perceptron network model to predict deformations at different scales to generate the first and second deformation fields includes: With a three-dimensional coordinate grid Location point in As input, output the displacement vector corresponding to that point. ,Right now:

[0015] By moving the original position of each point in image M Adding it to its displacement vector yields the new position after deformation. ,Right now: Using a sine function as the activation function, based on a coordinate grid... Generated neural displacement field Represented as:

[0016]

[0017] in, This represents the output of the i-th layer of the network. Let be the weight matrix of the i-th layer. For bias vectors, The input is a spatial coordinate vector.

[0018] As an alternative implementation, the process of downsampling the fixed image and the moving image to a third predetermined ratio of the original resolution and then using a CNN network model to predict the third-scale deformation to generate the third deformation field includes: downsampling the fixed image and the moving image to 1 / 2 of the original resolution and then inputting them into a CNN network model to predict fine-scale deformation to generate the third deformation field.

[0019] As an alternative implementation, the moving and fixed images are input into the CNN network model at half the original resolution. After three downsampling layers: initial feature extraction is performed on the image using convolutional kernels, feature enhancement processing is performed by applying instance normalization and Leaky ReLU activation function, and then the feature map size is reduced to half the original size by trilinear interpolation. After two such feature extraction and size reduction, a feature map sequence is obtained.

[0020] As a further defined implementation, in the upsampling path of the CNN network model, the tensor sequence in the upsampling path is concatenated one by one with the low-dimensional feature map sequence in the same channel dimension. The concatenated feature map is processed by a convolution kernel, and the feature map size is upsampled to twice the original size through trilinear interpolation. Finally, the feature map is mapped channel by channel to the deformation field in the time dimension.

[0021] As an alternative implementation, during the training and optimization process of the registration network model, the final loss function is optimized by weighting the similarity loss function, the smoothness loss function, and the Jacobian determinant loss function.

[0022] A multi-scale implicit-explicit hybrid medical image registration system, comprising: The image acquisition module is configured to acquire CT images of both moving and stationary images; The image registration module is configured to process CT images using a trained registration network model, specifically including: The multi-scale registration module is configured to downsample the three-dimensional coordinate grid to different set ratios of the original resolution, and then use a multilayer perceptron network model to predict the deformation at different scales to generate the first and second deformation fields. The fine-scale registration module is configured to downsample the fixed image and the moving image to a third set ratio of the original resolution, and then use a CNN network model to predict the third-scale deformation to generate the third deformation field. The fusion module is configured to fuse three deformation fields to obtain the final deformation field, and then generate the final deformation image through a spatial transformation network.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention innovatively proposes a registration method based on multi-scale neural fields. Through the collaborative design of implicit neural representation (INR), multilayer perceptron (MLP), and CNN network, it successfully solves the problems of data scarcity and large lung deformation.

[0024] The multi-scale strategy of this invention enables coarse-to-fine deformation field learning, ensuring the efficiency and stability of the registration process. The entire registration process is completed end-to-end without intermediate processing steps. The target image and source image are directly input for registration without the need to generate a mask, effectively alleviating dataset limitations, significantly simplifying the registration process and reducing data preprocessing complexity and time costs.

[0025] This invention utilizes INR to predict large-scale deformations at low-resolution levels to capture global features, while employing explicit CNNs to fine-tune details at high-resolution levels, compensating for the shortcomings of INR in detail processing. By fully leveraging the complementary advantages of CNNs and INRs, the model achieves a significant improvement in registration performance, ensuring accuracy in large deformations while enhancing the precision of detail processing.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0028] Figure 1 This is a schematic diagram of the registration process in one embodiment; Figure 2 This is a visualization comparison of marker points at times T00 and T50 in a DIR-Lab dataset example of one embodiment, where (A) is the distribution of marker points at time T00; (B) is the distribution of marker points at time T50; and (C) is a comparison of the superposition of marker points at T00 and T50 within the T00 spatial frame. Figure 3 This is a multi-resolution lung CT cross-sectional image of case T00 and T50 in one embodiment; Figure 4 This is a scatter plot of different ablation models in one embodiment; Figure 5 This is a cross-sectional intensity difference map of slice 70 from Case 8 in the DIR-Lab dataset, as one embodiment. Figure 6 This is a visualization of the deformation field of case 55 from the DIR-Lab dataset, one of the embodiments. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] Where there is no conflict, the embodiments and features described in this application may be combined with each other.

[0033] As described in the background section, existing registration methods still have significant shortcomings: First, these methods do not fully achieve end-to-end integration and are highly dependent on mask accuracy, which restricts their robustness in practical applications; second, existing methods lack specific designs for large deformation registration and are difficult to handle extreme deformation scenarios; third, how to effectively preserve and process details in medical image registration using INR remains a core challenge, especially in high-resolution data, where detail loss may negatively impact registration accuracy.

[0034] To address the aforementioned problems, this invention provides a multi-scale implicit-explicit hybrid medical image registration method, comprising the following steps: Acquire CT images of both moving and stationary images; The trained registration network model is used to process CT images, specifically including: After downsampling the three-dimensional coordinate grid to different set ratios of the original resolution, the deformation at different scales is predicted using a multilayer perceptron network model, generating the first and second deformation fields. After downsampling the fixed image and the moving image to a third predetermined ratio of the original resolution, the third-scale deformation is predicted using a CNN network model to generate the third deformation field. The three deformation fields are fused to obtain the final deformation field, and the final deformation image is generated through a spatial transformation network.

[0035] Specifically, the registration network model of this invention refers to the Multi-Scale Neural Field Registration Network (MS-HIENet), which adopts a maskless end-to-end registration framework and fuses explicit CNN and implicit INR through a multi-scale strategy. This method not only significantly simplifies the data preprocessing process but also greatly improves registration accuracy. Experimental results show that MS-HIENet performs excellently in terms of target registration error and deformation field smoothness, and is particularly adept at efficiently handling large deformations, providing an efficient and accurate solution for lung image registration.

[0036] The following is a detailed introduction.

[0037] like Figure 1 As shown, precise alignment of the moving image to the fixed image can be achieved simply by acquiring both moving and fixed lung CT images. The specific registration process is as follows: like Figure 1 As shown, the registration network performs the registration process in parallel at three different scale levels: the original 3D coordinate grid is downsampled to 1 / 8 of the original resolution and then input into the MLP network to predict large-scale deformation, generating the deformation field. Simultaneously, the grid is downsampled to 1 / 4 of its original resolution and then input into another MLP network to predict mesoscale deformation, generating a deformation field. In parallel, the fixed and moving images are downsampled to half their original resolution and then fed into a CNN network to predict fine-scale deformation, generating a deformation field. The three deformation fields are fused to obtain the final deformation field. The final deformed image is generated through a Spatial Transformer Network (STN).

[0038] The entire registration process is completed end-to-end without any intermediate processing steps.

[0039] First, we will introduce in detail the implicit neural representation (INR) for multi-scale deformation modeling.

[0040] In this model, the Implicit Neural Representation (INR) serves as the core component, efficiently encoding the continuously changing features of the image through a Multilayer Perceptron (MLP). Specifically, the INR learns a mapping function from spatial coordinates to a high-dimensional feature space via an MLP network. In this embodiment, an MLP network is used to model the displacement field. ,in This represents the set of network parameters. The neural displacement field... With a three-dimensional coordinate grid Location point in As input, output the displacement vector corresponding to that point. ,Right now:

[0041] By moving the original position of each point in image M Adding it to its displacement vector yields the new position after deformation. ,Right now: Traditional implicit neural representations typically use ReLU as the activation function when processing images. However, ReLU is prone to "spectral bias" when learning high-frequency details, limiting the network's ability to handle small-scale deformations. Therefore, this embodiment uses a sine function as the activation function. (Based on a coordinate grid...) Generated neural displacement field It can be represented as:

[0042]

[0043] in, This represents the output of the i-th layer of the network. Let be the weight matrix of the i-th layer. For bias vectors, The input is a spatial coordinate vector. To achieve effective representation of neural displacement fields, this model constructs two MLP networks of the same size, each containing four hidden layers with 512 hidden units per layer. The depth and width design of this network ensures both the ability to capture complex displacement dynamics in medical images (providing sufficient representation capabilities) and the sensitivity to high-frequency details.

[0044] Secondly, we will introduce in detail the convolutional neural network (CNN) for fine-scale deformation prediction.

[0045] Although the model uses an MLP network with periodic activation functions for high-frequency detail learning to predict large-scale deformations in lung CT images, this embodiment optimizes fine-grained structural accuracy through a convolutional neural network (CNN) at higher resolution levels, enabling refined adjustments to local deformations.

[0046] The proposed CNN network primarily follows the U-Net information processing principle with a CNN backbone structure. At the beginning of the registration process, both the moving and stationary images are input into the network at half the original resolution. The network first passes through three downsampling layers: using... The convolutional kernel performs initial feature extraction on the image, followed by instance normalization and the Leaky ReLU activation function for feature enhancement. Then, trilinear interpolation is used to reduce the feature map size to half of its original size. Finally, after two such feature extraction and size reduction steps, a sequence of feature maps with higher dimensions and smaller size is obtained.

[0047] In the upsampling path, this embodiment continues the skip connection setting of U-Net, concatenating the tensor sequence in the upsampling path with the low-dimensional feature map sequence at the same level channel dimension one by one. The concatenated feature map is then processed... The convolution kernel is used for processing, and the feature map size is upsampled to twice the original size through trilinear interpolation.

[0048] In the final layer of the network, the feature maps are mapped channel-by-channel to the deformation field in the time dimension. The kernel size of this layer is [size missing]. And no other operations are performed. At this point, the CNN network has completed the fine registration task.

[0049] In the optimization section, this embodiment selects a similarity loss function respectively. Smoothness loss function Compared with Jacobian determinant loss function .

[0050] Similarity loss For similarity loss function In this embodiment, normalized cross-correlation (NCC) is used as a similarity metric to evaluate the degree of similarity between the deformed moving image and the stationary image.

[0051] in, For fixed image The point in the middle, Indicated by point Centered on, with side length as Cube local window The set of points within (set in this model) ). and Representing fixed images With deformed images The average intensity within a local window. Negative NCC is used as a similarity loss measure; the smaller the loss value, the more similar the two images are.

[0052] Smoothing loss Although minimizing the similarity loss function This can make the deformed image approximate a fixed image, but it may lead to physically unreasonable or discontinuous situations in the deformation field. To ensure the continuity and stability of the deformation field, a smoothness loss function is introduced. It suppresses the roughness of the deformation field by calculating the L2 norm of the spatial gradient of the deformation field. The specific formula is defined as follows:

[0053] here, Voxel representation Deformation field The gradient is calculated. In the discrete case, this embodiment uses the difference between adjacent elements to approximate the magnitude of the spatial gradient. If the spatial variation is more drastic, the corresponding loss value will also be larger.

[0054] Jacobian determinant loss In the proposed model, the Jacobian loss function The Jacobian determinant is used to ensure the biomedical feasibility of deformable fields. Maintaining its positive orientation is crucial to avoiding folding and tearing. By penalizing negative Jacobian values ​​using the ReLU function, It can force smooth and continuous deformation, maintain topological consistency, and prevent non-physical deformation. The formula is defined as follows:

[0055] Based on the above discussion, the comprehensive loss function proposed in this embodiment is as follows, where and The weights representing the smoothing loss and the Jacobian determinant loss are respectively:

[0056] This embodiment uses the 4D CT dataset provided by DIR-Lab, such as... Figure 2 As shown in the image, this dataset contains lung CT scans of 10 patients at different respiratory stages, with each group containing data from 10 different respiratory cycles. These images are high-resolution and detailed in their anatomical structure. The dataset not only contains three-dimensional spatial information but also records the temporal dimension changes during respiration. Therefore, it is widely used to evaluate and compare the effectiveness of different registration algorithms, develop and validate new algorithms, and in clinical applications for the diagnosis and treatment of lung diseases.

[0057] In practice, this embodiment extracts images at the start (T00) and end (T50) of each patient's respiratory cycle as input pairs for the model. Each scan pair is matched with 30 pairs of lung landmarks manually annotated by professionals. However, the original four-dimensional CT dataset is too large in both spatial and temporal dimensions, which not only increases computational complexity but also introduces irrelevant noise. Therefore, this embodiment first crops the image data, retaining only the lung-related parts, aiming to preserve as much lung activity information as possible. This method reduces computational complexity while encapsulating key data information.

[0058] Then, the cropped image is normalized, adjusting the pixel value range to... This provides standardized input data for the subsequent registration process.

[0059] For each set of parameters, this embodiment applies uniform parameter settings to the model. When training MS-HIENet, considering the differences in network structure at different scales, CNN and MLP employ different learning rates. Specifically, the learning rate for CNN is set to 0.001, while the learning rate for MLP is set to 0.0001, both adjusted using the Adam optimizer. The entire model undergoes 1000 iterations of optimization. Model construction and training are completed within the PyTorch framework. Evaluation is performed on a system equipped with an NVIDIA RTX 3090 Ti GPU and an Intel i7-7700K CPU to ensure sufficient computing power and efficiency.

[0060] Given that each scan pair in the DIRLAB dataset includes 300 anatomically important landmarks, this embodiment uses target registration error (TRE) as the core evaluation metric to assess the performance of MS-HIENet compared to other registration methods in the DIRLAB dataset. TRE is a widely used metric in medical image registration evaluation. It assesses the accuracy of registration by measuring the Euclidean distance between landmarks in the registered image and their positions in the original image. A lower TRE value generally indicates a more accurate registration result. The calculation formula is as follows:

[0061] Where N is the total number of landmarks (300 in the DIR-LAB dataset). It is the first The location of an actual landmark It is the location of the landmark after the registration model has been converted.

[0062] This embodiment comprehensively compares the proposed model with various methods, including traditional optimization-based methods (such as pTVreg), state-of-the-art unsupervised learning methods (such as VoxelMorph

[16] , HPRN, LungRegNet, MJ-CNN], GroupRegNet), CNN-based iterative methods, IDIR methods using implicit neural representations, and methods based on circular consistent implicit neural representations, to verify the superiority of the proposed model.

[0063] Table 1 lists a quantitative comparison of registration accuracy based on TRE. Although the accuracy may be slightly lower than some traditional methods in certain cases, the MS-HIENet model proposed in this embodiment shows higher registration accuracy and stability, especially performing well in large deformation tasks. (In Case 6) Case 7 And Case 8 In this study, the model utilizing MS-HIENet outperforms other multi-scale algorithms. Notably, while CNN-based iterative methods also employ multi-scale strategies, the method using implicit neural representations captures complex image features more effectively. Furthermore, compared to IDIR and models using periodic consistency, the proposed model achieves lower TRE, especially under large deformation conditions. Its main advantage lies in the end-to-end registration process, eliminating the need for lung masks in auxiliary registration. This simplifies preprocessing, reduces time consumption, and minimizes parameter tuning, thereby enhancing the model's robustness.

[0064] Table 1 Comparison Table

[0065] In terms of computational efficiency and model parameters, although MS-HIENet has a longer computation time per iteration, it outperforms the IDIR and periodic consistency IDIR methods in overall computation time by significantly reducing the number of iterations required. Specifically, the IDIR method requires 2500 iterations, averaging 13 minutes per case, while the periodic consistency IDIR method also requires 2500 iterations, averaging 9 minutes per case. In contrast, MS-HIENet only requires 1000 iterations, averaging 6 minutes per case, to achieve excellent registration results. Regarding model parameters, both IDIR and periodic consistency IDIR methods have 133,379 parameters, while MS-HIENet uses 447,617 parameters. Although MS-HIENet has a larger number of parameters, its advantage lies in using a completely low-resolution global coordinate grid input, avoiding masking limitations and high-resolution sampling explosion problems. By capturing global deformation consistency and anatomical background information, it can significantly improve the prediction ability and registration accuracy of large deformations. Experimental results show that MS-HIENet has a lower TRE value compared to other methods.

[0066] To comprehensively evaluate the effectiveness of each module in the MS-HIENet model, this embodiment conducted a series of ablation experiments. These experiments aimed to verify the following key hypotheses: First, single-scale INR outperforms CNNs in handling large deformations; second, CNNs can effectively compensate for the shortcomings of INRs in detail processing, leveraging the advantages of CNNs in capturing local features and processing high-frequency information. Furthermore, the study explored the impact of multi-scale strategies on registration performance, confirming that multi-scale methods based on INR can significantly improve registration accuracy and the ability to handle large deformations.

[0067] Figure 3This paper demonstrates a layered feature extraction strategy for transverse CT images of the lungs at different spatial resolutions. Experimental results show that at low-resolution levels (downsampling factors of 1 / 8 and 1 / 4), the large field of view forces the neural network to prioritize learning global deformation features of the lung parenchyma rather than local texture details; while at high-resolution levels (downsampling factor of 1 / 2), the morphological features of key anatomical structures such as the bronchial tree and vascular branches are preserved with enhanced integrity. To clearly illustrate the configuration of different model variants, Table 2 lists the network type and input settings used by each model variant at different scales. Specifically, low resolution corresponds to 1 / 8 downsampling of the original image, medium resolution to 1 / 4 downsampling, and high resolution to 1 / 2 downsampling. These scales are consistent with the resolution levels used in the MS-HIENet model and were determined based on memory usage considerations.

[0068] Table 2. Configuration of model variants in ablation experiments

[0069] In the table, NoMS = No Multi-Scale.

[0070] Table 3. Target registration error (TRE) and its mean for different ablation models in 10 cases.

[0071] Table 3 shows the target registration error (TRE) and its average value for different ablation models in 10 cases. Analysis shows that the single-scale INR model (MS-HIENet_NoMS_INR) outperforms the single-scale CNN model (MS-HIENet_NoMS_CNN) in reducing the target registration error (TRE), confirming the advantage of INR in handling large deformations. Adopting a multi-scale strategy further improves registration performance through the combined use of INR and CNN—the multi-scale model can handle global and local deformations at different resolutions, fully leveraging the large-scale deformation prediction capability of INR and the detail optimization capability of CNN. However, some multi-scale variants (such as MS-HIENet_INR3) perform poorly in cases with large deformations (such as Case 8) due to the detail processing limitations of INR. Ultimately, the MS-HIENet model, with the lowest average TRE (1.00±1.15), validates the significant effectiveness of the multi-scale INR strategy in improving registration accuracy and handling large deformations.

[0072] Figure 4A scatter plot of the registration results in the ablation experiments is presented to assess the concentration and dispersion of errors. Since the results of cases 1 to 4 show relatively small differences, the analysis focuses on the target registration error (TRE) of cases 5 to 10. In the figure, the horizontal axis represents TRE (unit: mm), the vertical axis corresponds to the z-axis direction of the image, and each scatter point represents the registration error of a single sample. The comparison reveals that the MS-HIENet model has the most concentrated TRE distribution, with approximately 95% of the scatter points concentrated in the low error range (<2 mm), indicating high registration accuracy and strong stability. In contrast, in large deformation scenarios (such as case 8), the multi-scale INR-based registration method only shows a few samples with high TRE (>7 mm), while other models (such as pure CNN or single-scale INR) show a significant increase in abnormal error points, verifying the adaptability of the multi-scale strategy to extreme deformations. Furthermore, the error distribution of all models along the main direction of respiratory motion (z-axis) has low dispersion, indicating that the deformation field is continuous in the time dimension. This result further confirms that the hybrid architecture integrating multi-scale INR and CNN can effectively balance global deformation modeling and local detail preservation, thereby improving the registration reliability under complex breathing motion.

[0073] Figure 5 This paper presents the cross-sectional intensity difference map of slice 70 in Case 8 of the DIR-Lab dataset, comparing the intensity differences between the stationary image (EI), the moving image (EE), and deformed images generated by different models with the stationary image. The intensity difference map uses color gradients to visualize local differences, with darker colors indicating greater differences. The results show that the deformed image generated by the full MS-HIENet model has the smallest intensity difference compared to the stationary image, especially in the lung and thoracic regions, where the difference is significantly lower than other models. In contrast, models using only INR or single-scale methods still exhibit significant intensity differences in detail processing (such as lung boundaries). Furthermore, the INR method outperforms CNN in handling large deformations. In summary, MS-HIENet, by effectively fusing CNN and INR, more accurately captures image details (especially in areas of large deformation), thus achieving superior registration performance.

[0074] Figure 6The visualization results of the deformation field of slice 55 in Case 8 from the DIR-Lab dataset are shown, with the deformation field overlaid in light blue on the slice image. As shown in the figure, the deformation field generated by the complete MS-HIENet model exhibits higher smoothness and consistency in the lung and thoracic regions, especially in areas of large deformation (such as the lung boundary), where the deformation field distribution is more reasonable, and the deformation vector direction highly matches the movement trend of the anatomical structure. In contrast, the deformation fields of other models (such as pure INR or single-scale CNN) show local orientation disorder in the diaphragm region (arrows), and some areas exhibit excessive compression or stretching. This indicates that the multi-scale explicit-implicit hybrid strategy can effectively coordinate global deformation constraints and local detail adjustments, generating a smooth deformation field that conforms to biomechanical properties and avoiding non-physical folding or tearing artifacts common in traditional methods.

[0075] Table 4 quantifies the proportion of negative Jacobian determinants for each model in 10 cases from the DIR-Lab dataset (reflecting the smoothness and topological consistency of the deformation field). Experimental results show that the full MS-HIENet model has the lowest average proportion of negative Jacobian determinants (0.00017), significantly outperforming other model variants, especially in case 8 (0.00032). Furthermore, the MS-HIENet_INR3 model has a negative proportion of 0.00008, better than MS-HIENet_CNN3 (0.00051), further validating the advantage of implicit neural representations in generating smooth deformation fields. However, it is noteworthy that while the single-scale INR and CNN models achieve a zero proportion of negative Jacobian determinants (indicating no folding or tearing in the deformation field), their TRE performance is poor—this indicates that while single-scale models can maintain the topological consistency of the deformation field, the lack of global-local feature capture capabilities of multi-scale strategies limits registration accuracy. Therefore, multi-scale strategies play a crucial role in improving registration performance.

[0076] Table 4. Comparison of the percentage of negative Jacobian determinants among different models ( )

[0077] It should be noted that values ​​< 0.00001 in the table are marked as "-".

[0078] Example 2 A multi-scale implicit-explicit hybrid medical image registration system, comprising: The image acquisition module is configured to acquire CT images of both moving and stationary images; The image registration module is configured to process CT images using a trained registration network model, specifically including: The multi-scale registration module is configured to downsample the three-dimensional coordinate grid to different set ratios of the original resolution, and then use a multilayer perceptron network model to predict the deformation at different scales to generate the first and second deformation fields. The fine-scale registration module is configured to downsample the fixed image and the moving image to a third set ratio of the original resolution, and then use a CNN network model to predict the third-scale deformation to generate the third deformation field. The fusion module is configured to fuse three deformation fields to obtain the final deformation field, and then generate the final deformation image through a spatial transformation network.

[0079] Example 3 A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps in the method provided in Embodiment 1.

[0080] Example 4 An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps in the method provided in Embodiment 1.

[0081] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of one or more computer-usable storage media (including, but not limited to, disk storage, etc.) containing computer-usable program code. CD - ROM It takes the form of a computer program product implemented on (such as optical memory, etc.).

[0082] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-scale implicit-explicit hybrid medical image registration method, characterized in that, Includes the following steps: Acquire CT images of both moving and stationary images; The trained registration network model is used to process CT images, specifically including: After downsampling the three-dimensional coordinate grid to different set ratios of the original resolution, the deformation at different scales is predicted using a multilayer perceptron network model, generating the first and second deformation fields. After downsampling the fixed image and the moving image to a third predetermined ratio of the original resolution, the third-scale deformation is predicted using a CNN network model to generate the third deformation field. The three deformation fields are fused to obtain the final deformation field, and the final deformation image is generated through a spatial transformation network.

2. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, After downsampling the three-dimensional coordinate grid to different set ratios of the original resolution, the process of predicting deformations at different scales using a multilayer perceptron network model to generate the first and second deformation fields includes: After downsampling the three-dimensional coordinate grid to a first set ratio of the original resolution, the first-scale deformation is predicted using a multilayer perceptron network model to generate the first deformation field. After downsampling the three-dimensional coordinate grid to a second predetermined scale of the original resolution, another multilayer perceptron network model is used to predict the second-scale deformation and generate the second deformation field.

3. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 2, characterized in that, The first scale is larger than the second scale, the second scale is larger than the third scale, the first set ratio is smaller than the second set ratio, and the second set ratio is smaller than the third set ratio.

4. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, The two multilayer perceptron network models have the same structure, both containing four hidden layers, each with 512 hidden units.

5. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, After downsampling the three-dimensional coordinate grid to different set ratios of the original resolution, the process of predicting deformations at different scales using a multilayer perceptron network model to generate the first and second deformation fields includes: With a three-dimensional coordinate grid Location point in As input, output the displacement vector corresponding to that point. ,Right now: By moving the original position of each point in image M Adding it to its displacement vector yields the new position after deformation. ,Right now: Using a sine function as the activation function, based on a coordinate grid... Generated neural displacement field Represented as: in, This represents the output of the i-th layer of the network. Let be the weight matrix of the i-th layer. For bias vectors, The input is a spatial coordinate vector.

6. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, The process of downsampling the fixed and moving images to a third set ratio of the original resolution and then using a CNN network model to predict the third-scale deformation to generate the third deformation field includes: downsampling the fixed and moving images to half of the original resolution and then inputting them into a CNN network model to predict fine-scale deformation to generate the third deformation field.

7. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, Moving and fixed images are input into the CNN network model at half the original resolution. After three downsampling layers: initial feature extraction is performed on the image using convolutional kernels, feature enhancement is performed by applying instance normalization and Leaky ReLU activation function, and then the feature map size is reduced to half the original size by trilinear interpolation. After two such feature extraction and size reduction, the feature map sequence is obtained.

8. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, In the upsampling path of the CNN network model, the tensor sequence in the upsampling path is concatenated one by one with the low-dimensional feature map sequence in the same channel dimension. The concatenated feature map is processed by convolution kernels, and the feature map size is upsampled to twice the original size through trilinear interpolation. Finally, the feature map is mapped to the deformation field in the time dimension channel by channel.

9. The multi-scale implicit-explicit hybrid medical image registration method as described in claim 1, characterized in that, During the training and optimization process of the registration network model, the final loss function is optimized by weighting the similarity loss function, the smoothness loss function, and the Jacobian determinant loss function.

10. A multi-scale implicit-explicit hybrid medical image registration system, characterized in that, include: The image acquisition module is configured to acquire CT images of both moving and stationary images; The image registration module is configured to process CT images using a trained registration network model, specifically including: The multi-scale registration module is configured to downsample the three-dimensional coordinate grid to different set ratios of the original resolution, and then use a multilayer perceptron network model to predict the deformation at different scales to generate the first and second deformation fields. The fine-scale registration module is configured to downsample the fixed image and the moving image to a third set ratio of the original resolution, and then use a CNN network model to predict the third-scale deformation to generate the third deformation field. The fusion module is configured to fuse three deformation fields to obtain the final deformation field, and then generate the final deformation image through a spatial transformation network.

Images