Image registration method and device fusing physical prior and adaptive parameter learning

Abstract

The present application relates to a kind of image registration method and device of fusing physical priori and adaptive parameter learning, the method includes the following steps: obtaining image pair, including fixed image as reference graph and the mobile image to be registered;The image pair is input into multiple cascade registration iteration units and is refined gradually, and the final displacement field is output, the final displacement field includes the superposition result of single-step displacement field update amount output by each level registration iteration unit;The final displacement field output is used as registration displacement field, and the mobile image is resampled, and registration result is obtained.The present application explicitly encodes the analytical intermediate quantity of physical iterative registration algorithm as neural network input, physical constraint is explainable, and two-stage mechanism of prediction correction is introduced, so that network can adaptively adjust the regularization intensity according to local image structure, without manual parameter adjustment, improve the generalization ability when training data is limited and registration precision, significantly shorten registration time-consuming.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and relates to high-precision non-rigid registration technology for medical images such as computed tomography (CT) and magnetic resonance imaging (MRI). Specifically, it relates to an image registration method and system that integrates physical priors and adaptive parameter learning. Background Technology

[0002] Image registration aims to establish a precise mapping relationship between the spatial coordinates of two or more images, so that images acquired at different times, from different imaging devices, or from different individuals can be aligned in space to achieve consistent anatomical alignment. It is a core and key technology in precision medicine applications such as radiotherapy planning, intraoperative navigation, multimodal fusion diagnosis, and efficacy evaluation. Its registration accuracy and efficiency directly affect the reliability of subsequent clinical decisions.

[0003] The current mainstream technologies for medical image registration can be divided into three categories.

[0004] The first category is the traditional physical-driven iterative registration algorithm, with references to the Demons algorithm and its variants. References [1] (Thirion J P. Image matching as a diffusion process: an analogy with Maxwell's demons[J]. Medical image analysis, 1998, 2(3): 243-260), [2] (Wang H, Dong L, O'Daniel J, et al. Validation of an accelerated 'demons' algorithm for deformable image registration in radiation therapy[J]. Physics in Medicine & Biology, 2005, 50(12): 2887-2905), and [3] (Santos-Ribeiro A, NuttD J, McGonigle J. Inertial demons: A momentum-based diffeomorphic registration framework[C] / / International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2016: Represented by methods 37-45, inspired by thermodynamic diffusion models, image alignment is achieved by iteratively updating the displacement field based on image gradient and intensity differences. These methods have clear physical meaning and strong interpretability, making them highly reliable in clinical settings. However, parameter tuning relies on human experience, and computation is time-consuming.

[0005] The second category is pure data-driven deep learning registration methods, represented by references [4] (Balakrishnan G, Zhao A, Sabuncu MR, et al. Voxelmorph: a learning framework for deformable medical image registration[J]. IEEE transactions on medical imaging, 2019, 38(8): 1788-1800) and [5] (Ranjan A, Black M J. Optical flow estimation using a spatial pyramid network[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4161-4170). These methods utilize convolutional neural networks to directly regress the displacement field from image pairs, resulting in fast inference speed, but insufficient interpretability and inadequate stability and generalization ability in medical image scenarios with limited labeled data.

[0006] The third category is the hybrid registration method based on algorithm expansion, represented by the deep expansion network for brain image registration in reference [6] (Liao R, Wang P, Cao W, et al. MBURegNet: an unsupervised model-based deep unrolling registration network for deformable image registration in dual-energy CT[C] / / MedicalImaging 2024: Physics of Medical Imaging. SPIE, 2024, 12925: 888-892) and reference [7] (Sun Jiaying, Yang Shuangyan, Ying Shihui. Research on deep expansion network for brain image registration[J / OL]. Journal of Image and Graphics, 2026, 1-15. DOI: 10.11834 / jig.250283). This method maps the traditional iterative optimization steps to neural network layers, which has both interpretability and learnability, but its registration accuracy still needs to be further improved. Summary of the Invention

[0007] Technical objective: To address the aforementioned technical problems, this invention proposes an image registration method and system that integrates physical priors and adaptive parameter learning.

[0008] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0009] An image registration method integrating physical priors and adaptive parameter learning includes the following steps:

[0010] Acquire image pairs, including a fixed image as a reference and a moving image to be registered;

[0011] The image is refined step by step by inputting multiple cascaded registration iteration units, and the final displacement field is output. The final displacement field includes the superposition result of the single-step displacement field update output by each registration iteration unit.

[0012] The final displacement field output is used as the registration displacement field, and the moving image is resampled to obtain the registration result;

[0013] Each registration iteration unit executes the following process:

[0014] Get the current cumulative displacement field and the single-step displacement field update of the previous unit;

[0015] The deformation image is obtained by resampling the moving image based on the current cumulative displacement field;

[0016] Extract the gradient fields of the fixed image and the deformed image;

[0017] Substituting the initial regularization parameter into the analytical formula of the physical iterative registration algorithm, the first physical prior intermediate quantity is calculated; the physical iterative registration algorithm is an iterative registration method based on image gradient and intensity residual with clear physical meaning;

[0018] The first physical prior intermediate quantity, gradient field, current cumulative displacement field and single-step position field update quantity of the previous unit are concatenated into a multi-channel feature tensor, which is then input into the deformation estimation neural network. The deformation estimation neural network outputs a regularization parameter map and displacement field correction quantity.

[0019] Substituting the regularized parameter graph into the analytical formula of the physical iterative registration algorithm, the second physical prior intermediate quantity is calculated;

[0020] Add the second physical prior intermediate quantity to the displacement field correction quantity to obtain the single-step displacement field update quantity of the current element;

[0021] The single-step displacement field update of the current unit, the single-step displacement field update of the previous unit, the current cumulative displacement field, and the second physical prior intermediate quantity are concatenated and then input into the inertial fusion neural network to obtain the inertial correction quantity.

[0022] The final displacement field is obtained by summing the inertial correction, the current cumulative displacement field, and the single-step displacement field update of the current element.

[0023] Furthermore, the current cumulative displacement field input to each registration iteration unit includes the cumulative displacement field in the horizontal direction and the cumulative displacement field in the vertical direction, and the extracted gradient field includes the horizontal gradient field and the vertical gradient field of the fixed image and the deformed image.

[0024] Furthermore, the physical iterative registration algorithm is a series of Demons algorithms, including one or a combination of the Thirion primitive Demons algorithm, the Wang symmetric Demons algorithm, or the Santos-Ribeiro inertial Demons algorithm;

[0025] The first and second physical prior intermediate quantities include at least: the intensity residual field of the fixed image and the deformed image, the horizontal and vertical offsets of the fixed image side, the horizontal and vertical offsets of the deformed image side, the confidence map of the fixed image side, and the confidence map of the deformed image side.

[0026] Furthermore, the physical iterative registration algorithm is the Horn-Schunck optical flow method, and the first physical prior intermediate quantity and the second physical prior intermediate quantity are the analytical intermediate quantities of the corresponding algorithm.

[0027] Furthermore, the deformation estimation neural network is a dual-output neural network containing a regularization intensity head and a residual correction head, and the regularization parameter map is constrained by the Softplus activation function to ensure a positive value.

[0028] Furthermore, the weights and biases of the last layer of the inertial fusion neural network are initialized to zero.

[0029] Furthermore, the multiple registration iteration units employ a multi-level intermediate supervision strategy to design the loss function. :

[0030]

[0031] in, For the first Loss function of each registration iteration unit The weighting coefficients.

[0032] An image registration device integrating physical priors and adaptive parameter learning includes:

[0033] The image acquisition module is configured to acquire image pairs, including a fixed image as a reference image and a moving image to be registered;

[0034] The displacement field calculation module is configured to refine the image pair by inputting multiple cascaded registration iteration units step by step, and output the final displacement field, wherein the final displacement field includes the superposition result of the single-step displacement field update output by each level of the registration iteration unit;

[0035] The registration module is configured to use the output final displacement field as the registration displacement field, resample the moving image, and obtain the registration result.

[0036] The displacement field calculation module includes:

[0037] The input unit is configured to obtain the current cumulative displacement field and the single-step displacement field update of the previous unit;

[0038] An image resampling unit is configured to resample the moving image based on the current cumulative displacement field to obtain a deformed image;

[0039] The gradient field extraction unit is configured to extract the gradient fields of the fixed image and the deformed image;

[0040] The physical prior calculation unit is configured to substitute the initial regularization parameters into the analytical formula of the physical iterative registration algorithm to calculate the first physical prior intermediate quantity; the physical iterative registration algorithm is an iterative registration method based on image gradient and intensity residual with clear physical meaning; and to substitute the regularization parameter map into the analytical formula of the physical iterative registration algorithm to calculate the second physical prior intermediate quantity.

[0041] The feature splicing unit is configured to splice the first physical prior intermediate quantity, gradient field, current cumulative displacement field and single-step position field update quantity of the previous unit into a multi-channel feature tensor.

[0042] The deformation estimation neural network is configured as a dual-output neural network containing a regularization intensity head and a residual correction head. It processes the output of the feature splicing unit and outputs a regularization parameter map and a displacement field correction.

[0043] The prediction and correction unit is configured to add the second physical prior intermediate quantity to the displacement field correction quantity to obtain the single-step displacement field update quantity of the current unit;

[0044] The physics and data fusion unit is configured to concatenate the single-step displacement field update of the current unit, the single-step displacement field update of the previous unit, the current cumulative displacement field, and the second physical prior intermediate quantity into a multi-channel feature tensor.

[0045] An inertial fusion neural network is configured to process the output of the physical and data fusion unit to obtain the inertial correction amount;

[0046] The cumulative displacement field update unit is configured to sum the inertial correction amount, the current cumulative displacement field, and the single-step displacement field update amount of the current unit to obtain the final displacement field.

[0047] An electronic device includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the steps of the method.

[0048] A computer program product includes a computer program that, when executed by a processor, implements the steps of the method.

[0049] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0050] The image registration method and apparatus proposed in this invention integrate physical priors and adaptive parameter learning. It explicitly encodes the analytical intermediate quantities of the physical iterative registration algorithm into neural network inputs, making the physical constraints interpretable and exhibiting stronger generalization ability in medical image limited annotation scenarios. It replaces the regularization parameter with a learnable spatial adaptive parameter map and introduces a two-stage prediction and correction mechanism, enabling the network to adaptively adjust the regularization intensity according to the local image structure without manual parameter tuning. The inertial fusion parameter is replaced by an independent neural network, which directly predicts the inertial correction amount using the current state and historical states as inputs. This can adapt to situations where fixed parameters cannot handle such as sudden changes in motion direction and oscillations during the iteration process, significantly reducing the learning difficulty, improving the generalization ability and registration accuracy when training data is limited, and significantly shortening the registration time. Attached Figure Description

[0051] Figure 1 This is a flowchart of the image registration method that integrates physical priors and adaptive parameter learning proposed in Embodiment 1 of the present invention.

[0052] Figure 2 This is a schematic diagram of the overall structure of the image registration system that integrates physical priors and adaptive parameter learning, as proposed in Embodiment 2 of the present invention.

[0053] Figure 3 This is a schematic diagram of the deformation estimation neural network example based on the encoder-decoder structure proposed in this invention.

[0054] Figure 4 This is a schematic diagram of an example of a deformation estimation neural network based on residual structure proposed in this invention.

[0055] Figure 5 This is a schematic diagram of the structure of an example of the inertial fusion neural network proposed in this invention.

[0056] Figure 6 This is a schematic diagram of the training data generation process in the registration method proposed in this invention.

[0057] Figure 7 This is a comparison chart of the registration method proposed in this invention with the Demons algorithm (100 iterations, 300 iterations) and the SpyNet deep learning method on a public dataset. Detailed Implementation

[0058] Example 1

[0059] Figure 1 This is a flowchart of the image registration method that integrates physical priors and adaptive parameter learning proposed in Embodiment 1 of the present invention. Figure 1 As shown, the non-rigid image registration method of this embodiment includes the following steps:

[0060] Acquire image pairs of fixed and moving images, and initialize the cumulative displacement field and single-step displacement field update to zero;

[0061] The image is used as the registration displacement field by refining the displacement field step by step through N cascaded registration iteration units. The moving image is then resampled to obtain the registration result.

[0062] Each registration iteration unit executes the following process:

[0063] The deformation image is obtained by resampling the moving image based on the current cumulative displacement field;

[0064] Extract the gradient fields from the fixed image and the deformed image;

[0065] Substitute the initial regularization parameter into the analytical formula of the physical iterative registration algorithm to calculate the first physical prior intermediate quantity;

[0066] The first physical prior intermediate quantity, gradient field, cumulative displacement field and single-step displacement field update quantity of the previous unit are concatenated into a multi-channel feature tensor, which is input into the deformation estimation neural network. The deformation estimation neural network outputs a regularized parameter map and displacement field correction quantity.

[0067] The initial regularization parameters are replaced with the regularization parameter graph, and the second physical prior intermediate quantity is calculated.

[0068] Add the recalculated second physical prior intermediate quantity to the displacement field correction quantity to obtain the single-step displacement field update quantity of the current element;

[0069] The single-step displacement field update of the current unit, the single-step displacement field update of the previous unit, the current cumulative displacement field, and the second physical prior intermediate quantity are concatenated and input into the inertial fusion neural network to obtain the inertial correction quantity, which is then added to the update quantities of the previous unit and the current unit to update the cumulative displacement field.

[0070] The cumulative displacement field output after N units is used as the final registration displacement field. The moving image is then resampled to obtain the registration result.

[0071] In the above steps, the fixed image serves as the reference image for registration and as the alignment standard; the moving image serves as the image to be registered, and is aligned with the fixed image through deformation. The two together are called a set of input image pairs. The cumulative displacement field is used to record the total deformation of the moving image from its initial position to the current iteration unit.

[0072] Example 2

[0073] Figure 2 This is a schematic diagram of the overall structure of the non-rigid image registration system that integrates physical priors and adaptive parameter learning, as proposed in Embodiment 2 of the present invention.

[0074] 1. System Overall Structure

[0075] like Figure 2 As shown, the registration system in this embodiment includes N cascaded registration iteration units. The parameters of each registration iteration unit can be shared (i.e., parameter sharing mode, the number of parameters does not increase with N) or independent (i.e., parameter independence mode, higher accuracy), both modes are within the protection scope of this invention.

[0076] The system data flow conventions are as follows:

[0077] Input: Fixed image (Reference image), moving image (Image to be registered);

[0078] Output: Cumulative displacement field (Including the cumulative displacement field in the horizontal direction) and the cumulative displacement field in the vertical direction ), moving image through After resampling, it is aligned with the fixed image F.

[0079] 2. Detailed process of a single registration iteration unit

[0080] Furthermore, with the first Taking one unit as an example, each registration iteration unit performs the following steps sequentially:

[0081] Step 1: Moving Image Resampling (Deformed Image Generation)

[0082] Using the current cumulative displacement field For moving images Resampling is performed to obtain the deformed image. In particular, when At that time, the cumulative displacement field is initialized as an all-zero matrix. Equivalent to .

[0083] Step 2: Gradient Field Extraction

[0084] Extract fixed images separately With deformed images Horizontal gradient field , gradient field in the vertical direction , Gradient extraction methods can include the Sobel operator, the Scharr operator, the Prewitt operator, or the differential gradient method.

[0085] Step 3: Pre-calculation of physical prior features (prediction stage)

[0086] With the initial regularization coefficient graph Substituting the (all-1 matrix) into the analytical update formula of the Demons algorithm derived from the thermodynamic diffusion model, calculate the following 7 physical prior intermediate quantities:

[0087] Intensity residual field:

[0088] Fixed image side horizontal offset:

[0089] Fixed image side vertical offset:

[0090] Horizontal offset of deformed image side:

[0091] Vertical offset of the deformed image side:

[0092] Fixed image side confidence plot:

[0093] Deformation image side confidence plot:

[0094] in, Indicates the offset. Subscript , These represent the horizontal and vertical directions, respectively.

[0095] , These represent the forward horizontal gradient field (based on the right pixel) and the backward horizontal gradient field (based on the left pixel), respectively. , These represent the forward vertical gradient field (based on the lower pixel) and the backward vertical gradient field (based on the upper pixel), respectively.

[0096] The confidence map is obtained by normalizing the gradient energy and regularization term, with a value range of [0, 1], representing the confidence level of "gradient-driven update" at the corresponding pixel. A high confidence level means that the pixel is in a strong edge or high-texture region (large gradient) and the registration residual D is small; conversely, the pixel is in a flat region (small gradient) or the registration is severely misaligned, and the registration residual D is large.

[0097] The above 7 physical quantities and the 4 gradient fields in Step 2 ( ), and the current cumulative displacement field and The single-step displacement field update of the previous unit and They are concatenated along the channel dimension to form a 15-dimensional feature tensor, which serves as the input to the neural network.

[0098] In particular, when hour, and Initialize as a matrix of all zeros.

[0099] Step 4: Deformation estimation neural network inference (dual output)

[0100] The 15-dimensional feature tensor from Step 3 is fed into a deformation estimation neural network (such as an encoder-decoder structure, residual network, etc.). The network has two functionally independent output heads:

[0101] Regularization intensity head (single channel): Outputs a learnable spatial adaptive regularization parameter graph. Guaranteed by the Softplus activation function constraint In the early stages of training, the network output is close to the initial parameter values, forming a smooth transition from the initial parameters to the adaptive parameters.

[0102] Residual correction head (dual channel): Outputs learnable displacement field corrections. (Horizontal component) and vertical components ).

[0103] This invention, through this design, directly uses analytical intermediate quantities in the Demons formula, such as intensity residual field and offset, as input channels for the neural network. This simplifies the network's learning task from "learning the complete nonlinear mapping from image pairs to displacement fields from 0" to "learning the nonlinear correction of the physical analytical solution," which can significantly reduce the learning difficulty and improve the generalization ability when training data is limited.

[0104] In this embodiment of the invention, the traditional global scalar regularization parameter is first extended to a spatially distributed regularization parameter map, allowing different regularization parameters to be applied to different locations in the image. Furthermore, this parameter map is adaptively generated through a neural network, transforming the regularization parameter from a globally uniform, manually set parameter to a spatially variable, data-driven parameter, thereby reducing reliance on manual parameter tuning and repeated trials, while also considering the effective displacement updates and iterative stability of different local regions. Moreover, both the regularization parameter map and the displacement field correction are calculated based on the same 15-dimensional input features and jointly serve the displacement update of the current registration iteration unit. Both are input into the same deformation estimation neural network for processing, and the two output heads share the main feature extraction part of the network, avoiding redundant feature extraction caused by setting two independent networks, and reducing the number of network parameters, computational load, and storage overhead.

[0105] It should be noted that this invention does not limit the specific structure of the deformation estimation neural network. Any neural network capable of accepting 15-dimensional feature input and producing a single-channel regularization parameter map and a dual-channel displacement field correction output is acceptable. Its backbone network for feature extraction can be an "encoder-decoder" structure, such as... Figure 3 As shown, Figure 3 This is a schematic diagram illustrating an example of a deformation estimation neural network based on an encoder-decoder structure proposed in this invention. It can also be a residual structure, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of an example of a deformation estimation neural network based on residual structure proposed in this invention. Alternatively, other network structures that meet the input and output requirements may also be used.

[0106] Step 5: Recalculation of physical prior features (correction phase)

[0107] Output from Step 4 Replace Step 3 Substituting the same Demons analytical formula, the seven intermediate physical quantities are recalculated to obtain the updated physical state set. .

[0108] Step 6: Fusion of physical update quantity and neural network correction quantity

[0109] Adjust the offset in the physical state set of Step 5 correction phase. Adding the displacement field correction amount output by the residual correction head in Step 4, we obtain the single-step displacement field update amount for this unit. .

[0110] This invention replaces the globally fixed regularization coefficients in the Demons formula with a learnable spatial adaptive parameter map regularization intensity map, employing a two-stage prediction-correction mechanism: In the first stage, physical prior features are calculated using initial parameters (an all-one matrix) and fed into the neural network, which predicts the regularization intensity map; in the second stage, the initial parameters are replaced with this parameter map, and the physical intermediate quantities are recalculated to obtain accurate physical updates. This mechanism enables the network to adaptively adjust the regularization intensity based on local image structures (such as high-contrast boundaries and uniformly organized regions) without manual parameter tuning.

[0111] Step 7: Learnable Nonlinear Inertial Fusion

[0112] The single-step displacement field update of the previous unit, the single-step displacement field update of the current unit, the current cumulative displacement field, and the intensity residual field D in Step 5 are concatenated with the confidence map by channel to form a 9-dimensional feature tensor:

[0113] ;

[0114] It is fed into an inertial fusion neural network, which outputs a dual-channel inertial correction value. (Horizontal component) Vertical component Based on the single-step displacement field update of the previous unit, the final inertial fusion result is as follows:

[0115] .

[0116] This invention replaces the fixed inertial fusion parameter β in traditional Demons with a learnable inertial fusion neural network through this step. This network takes the 9-dimensional feature tensor from the current and historical states as input, directly predicts the inertial correction, outputs a dual-channel inertial correction, and uses it to update the current cumulative displacement field. This allows inertial fusion to perform data-driven adaptive adjustment based on the current and historical registration states, adapting to situations where fixed parameters cannot handle sudden changes in motion direction or oscillations during iteration. The weights and biases of the last layer of the inertial fusion neural network are initialized to 0, making the inertial correction close to 0 in the early stages of training, effectively degenerating into a traditional fixed linear inertial fusion method, and gradually learning a nonlinear fusion strategy as training progresses. The inertial fusion neural network is used to generate additional inertial corrections based on the current and historical displacement states, correcting the fusion result of the single-step displacement field update between the current unit and the previous unit. Initializing its last layer weights and biases to 0 ensures that the inertial correction is 0 or close to 0 in the early stages of training. Therefore, before the network has learned an effective inertial fusion strategy, it will not generate additional displacement corrections without physical basis due to random initialization. The system first runs according to the basic update path without introducing additional network inertial corrections, and gradually learns nonlinear inertial correction strategies as training progresses. This helps to reduce the risk of abnormal displacements or iterative oscillations in the early stages of training.

[0117] Figure 5 This is a schematic diagram of the structure of an example of the inertial fusion neural network proposed in this invention. Figure 5 A simple network structure diagram for implementation is shown, including convolutions and activation functions. In practice, the inertial fusion neural network used in this invention only needs to include a feature extraction module and an output mapping layer. The feature extraction module is used to extract the correlation features between the current displacement state, historical displacement states, and physical priors. The output mapping layer is used to generate dual-channel inertial correction values, with the two channels corresponding to the horizontal and vertical inertial correction values, respectively. The limitation of the inertial fusion neural network in this invention is mainly reflected in its input, output, and their functional relationship in the cascade registration process, rather than relying on a specific network structure. "Encoder-decoder" networks, residual networks, or other neural networks that can satisfy the input-output mapping can be used.

[0118] Step 8: Cumulative Displacement Field Update

[0119] The inertial fusion result is added to the single-step displacement field update of this element to obtain the cumulative displacement field of this element. The cumulative displacement field of this unit (Horizontal component) and vertical components ) and the single-step displacement field update of this unit The process is then passed to the next registration iteration unit, and the current unit's process ends.

[0120] 3. Training methods

[0121] This invention supports both supervised and unsupervised training paradigms. The training data can be various types of images or image patches, such as medical images (CT, MRI, etc.), remote sensing images, and natural images.

[0122] Supervised paradigm: Given a fixed image Randomly generate displacement field Φ (containing U and V), by The moving image is obtained by Φ resampling. This constitutes the paired training data. The process of generating paired data is as follows: Figure 6 As shown. Figure 6 This is a schematic diagram of the training data generation process in the registration method proposed in this invention.

[0123] Unsupervised paradigm: No known displacement field annotations are required; only fixed and moving image pairs are needed. , During training, image similarity and displacement field regularization constraints are used as loss functions.

[0124] Furthermore, the present invention contains Each of the registration iteration units outputs a cumulative displacement field, and a multi-level intermediate supervision strategy is used to design the loss function. The loss function can be expressed as:

[0125]

[0126] In the formula, For the first Unit loss function The weighting coefficients, Average value can be taken It can also be followed Incremental weighting (higher weight for later units; this setting is recommended). More specifically, the convention is... The cumulative displacement field output by each unit is (Include and ), moving image After the first The deformation image after resampling of the cumulative displacement field of each unit is .

[0127] Furthermore, unsupervised training does not require displacement field annotation; it is trained only by multi-level intermediate supervision using image similarity loss and displacement smoothing regularization loss. The unsupervised paradigm loss is expressed as:

[0128] .

[0129] Supervised training synthesizes paired training data by applying randomly generated displacement fields to fixed images. The outputs of each iterative unit are jointly constrained by a multi-level intermediate supervised loss function, which includes an image similarity term, a displacement field endpoint error term, and a smoothing regularization term. The supervised paradigm loss is expressed as:

[0130] ;

[0131] Supervised paradigm loss adds displacement field prediction loss to the unsupervised paradigm. In the formula, The similarity loss can be achieved using mean squared error (MSE) or normalized cross-correlation (NCC). This is the displacement smoothing regularization term (the L2 norm is taken after calculating the gradient of the displacement field). This refers to the endpoint error (the mean square error between the predicted displacement field and the actual displacement field). and These are their respective coefficients.

[0132] 4. Optimizers and Data Augmentation

[0133] In this embodiment of the invention, the Adam optimizer is used for iterative parameter updates during the training phase, with an initial learning rate set to 0.0005 and a learning rate decay strategy configured. This embodiment also introduces data augmentation strategies during training, specifically including applying rigid geometric transformations such as rotation and mirroring to the input image to augment the data and improve the model's generalization ability.

[0134] Figure 7 This is a comparison chart showing the registration method proposed in this invention, along with the Demons algorithm (100 iterations, 300 iterations) and the SpyNet deep learning method, on a public dataset. Figure 7 As shown, the experimental results indicate that the method of this invention is superior to other methods in the prediction accuracy of the U and V components of the displacement field. The deformation image is highly consistent with the gold standard, the residual field amplitude is significantly smaller than that of the Demons algorithm and SpyNet, and the registration accuracy and deformation field smoothness are significantly improved.

[0135] This invention can be based on the inertial Demons algorithm, or on the original Demons algorithm, the Wang symmetric Demons algorithm, or other Demons variants. It can also be based on iterative methods with clear physical significance, such as the Horn-Schunck optical flow method, with design modifications.

[0136] Network structure: Both dual-output networks and inertial fusion networks can adopt an "encode-decode" structure (such as U-Net), residual networks, or other network structures that meet the input and output size requirements.

[0137] Parameter mode: The N registration iteration units can share parameters or have independent parameters. That is, the neural network parameters of the N registration iteration units are shared with each other, and the total number of model parameters does not increase with N; or the neural network parameters of the N registration iteration units are independent with each other, and each unit has independent network weights.

[0138] Example 3

[0139] This embodiment proposes an image registration device that integrates physical priors and adaptive parameter learning, and executes the registration method described in Embodiment 1. The device includes:

[0140] The image acquisition module is configured to acquire image pairs, including a fixed image as a reference image and a moving image to be registered;

[0141] The displacement field calculation module is configured to refine the image pair by inputting multiple cascaded registration iteration units step by step, and output the final displacement field, wherein the final displacement field includes the superposition result of the single-step displacement field update output by each level of the registration iteration unit;

[0142] The registration module is configured to use the output final displacement field as the registration displacement field to resample the moving image and obtain the registration result.

[0143] Furthermore, the displacement field calculation module includes:

[0144] The input unit is configured to obtain the current cumulative displacement field and the single-step displacement field update of the previous unit;

[0145] An image resampling unit is configured to resample the moving image based on the current cumulative displacement field to obtain a deformed image;

[0146] The gradient field extraction unit is configured to extract the gradient fields of the fixed image and the deformed image;

[0147] The physical prior calculation unit is configured to substitute the initial regularization parameters into the analytical formula of the physical iterative registration algorithm to calculate the first physical prior intermediate quantity; the physical iterative registration algorithm is an iterative registration method based on image gradient and intensity residual with clear physical meaning; and to substitute the regularization parameter map into the analytical formula of the physical iterative registration algorithm to calculate the second physical prior intermediate quantity.

[0148] The feature splicing unit is configured to splice the first physical prior intermediate quantity, gradient field, current cumulative displacement field and single-step position field update quantity of the previous unit into a multi-channel feature tensor.

[0149] The deformation estimation neural network is configured as a dual-output neural network containing a regularization intensity head and a residual correction head. It processes the output of the feature splicing unit and outputs a regularization parameter map and a displacement field correction.

[0150] The prediction and correction unit is configured to add the second physical prior intermediate quantity to the displacement field correction quantity to obtain the single-step displacement field update quantity of the current unit;

[0151] The physics and data fusion unit is configured to concatenate the single-step displacement field update of the current unit, the single-step displacement field update of the previous unit, the current cumulative displacement field, and the second physical prior intermediate quantity into a multi-channel feature tensor.

[0152] An inertial fusion neural network is configured to process the output of the physical and data fusion unit to obtain the inertial correction amount;

[0153] The cumulative displacement field update unit is configured to sum the inertial correction amount, the current cumulative displacement field, and the single-step displacement field update amount of the current unit to obtain the final displacement field.

[0154] It should be noted that the registration framework of the present invention is also applicable to other scenarios with similar registration requirements, such as remote sensing image registration, video frame inter-frame optical flow estimation, and industrial inspection image alignment. Those skilled in the art can make corresponding transfers based on the description in this specification.

[0155] Another embodiment of the present invention provides an electronic device including at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the steps of the method described in Embodiment 1.

[0156] Another embodiment of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0157] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. An image registration method integrating physical priors and adaptive parameter learning, characterized in that, Including the following steps: Acquire image pairs, including a fixed image as a reference and a moving image to be registered; The image is refined step by step by inputting multiple cascaded registration iteration units, and the final displacement field is output. The final displacement field includes the superposition result of the single-step displacement field update output by each registration iteration unit. The final displacement field output is used as the registration displacement field, and the moving image is resampled to obtain the registration result; Each registration iteration unit executes the following process: Get the current cumulative displacement field and the single-step displacement field update of the previous unit; The deformation image is obtained by resampling the moving image based on the current cumulative displacement field; Extract the gradient fields of the fixed image and the deformed image; Substituting the initial regularization parameter into the analytical formula of the physical iterative registration algorithm, the first physical prior intermediate quantity is calculated; the physical iterative registration algorithm is an iterative registration method based on image gradient and intensity residual with clear physical meaning; The first physical prior intermediate quantity, gradient field, current cumulative displacement field and single-step position field update quantity of the previous unit are concatenated into a multi-channel feature tensor, which is then input into the deformation estimation neural network. The deformation estimation neural network outputs a regularization parameter map and displacement field correction quantity. Substituting the regularized parameter graph into the analytical formula of the physical iterative registration algorithm, the second physical prior intermediate quantity is calculated; Add the second physical prior intermediate quantity to the displacement field correction quantity to obtain the single-step displacement field update quantity of the current element; The single-step displacement field update of the current unit, the single-step displacement field update of the previous unit, the current cumulative displacement field, and the second physical prior intermediate quantity are concatenated and then input into the inertial fusion neural network to obtain the inertial correction quantity. The final displacement field is obtained by summing the inertial correction amount, the current cumulative displacement field, and the single-step displacement field update amount of the current element. The physical iterative registration algorithm is a series of Demons algorithms, including one or a combination of the Thirion original Demons algorithm, the Wang symmetric Demons algorithm, or the Santos-Ribeiro inertial Demons algorithm; The first and second physical prior intermediate quantities include at least: the intensity residual field of the fixed image and the deformed image, the horizontal and vertical offsets of the fixed image side, the horizontal and vertical offsets of the deformed image side, the confidence map of the fixed image side, and the confidence map of the deformed image side.

2. The image registration method integrating physical priors and adaptive parameter learning as described in claim 1, characterized in that: The current cumulative displacement field input to each registration iteration unit includes the cumulative displacement field in the horizontal direction and the cumulative displacement field in the vertical direction. The extracted gradient field includes the horizontal gradient field and the vertical gradient field of the fixed image and the deformed image.

3. The image registration method integrating physical priors and adaptive parameter learning as described in claim 1, characterized in that: The physical iterative registration algorithm can also be the Horn-Schunck optical flow method, where the first physical prior intermediate and the second physical prior intermediate are the analytical intermediates of the corresponding algorithms.

4. The image registration method integrating physical priors and adaptive parameter learning as described in claim 1, characterized in that: The deformation estimation neural network is a dual-output neural network containing a regularization intensity head and a residual correction head. The regularization parameter map is constrained by the Softplus activation function to ensure positive values.

5. The image registration method integrating physical priors and adaptive parameter learning as described in claim 1, characterized in that: The weights and biases of the last layer of the inertial fusion neural network are initialized to zero.

6. The image registration method integrating physical priors and adaptive parameter learning as described in claim 1, characterized in that: The multiple registration iteration units employ a multi-level intermediate supervision strategy to design the loss function. : ； in, For the first The loss function of each registration iteration unit. For the first Loss function of each registration iteration unit The weighting coefficients.

7. An image registration device integrating physical priors and adaptive parameter learning, characterized in that, include: The image acquisition module is configured to acquire image pairs, including a fixed image as a reference image and a moving image to be registered; The displacement field calculation module is configured to refine the image pair by inputting multiple cascaded registration iteration units step by step, and output the final displacement field, wherein the final displacement field includes the superposition result of the single-step displacement field update output by each level of the registration iteration unit; The registration module is configured to use the output final displacement field as the registration displacement field, resample the moving image, and obtain the registration result. The displacement field calculation module includes: The input unit is configured to obtain the current cumulative displacement field and the single-step displacement field update of the previous unit; An image resampling unit is configured to resample the moving image based on the current cumulative displacement field to obtain a deformed image; The gradient field extraction unit is configured to extract the gradient fields of the fixed image and the deformed image; The physical prior calculation unit is configured to substitute the initial regularization parameters into the analytical formula of the physical iterative registration algorithm to calculate the first physical prior intermediate quantity; the physical iterative registration algorithm is an iterative registration method based on image gradient and intensity residual with clear physical meaning; and to substitute the regularization parameter map into the analytical formula of the physical iterative registration algorithm to calculate the second physical prior intermediate quantity. The feature splicing unit is configured to splice the first physical prior intermediate quantity, gradient field, current cumulative displacement field and single-step position field update quantity of the previous unit into a multi-channel feature tensor. The deformation estimation neural network is configured as a dual-output neural network containing a regularization intensity head and a residual correction head. It processes the output of the feature splicing unit and outputs a regularization parameter map and a displacement field correction. The prediction and correction unit is configured to add the second physical prior intermediate quantity to the displacement field correction quantity to obtain the single-step displacement field update quantity of the current unit; The physics and data fusion unit is configured to concatenate the single-step displacement field update of the current unit, the single-step displacement field update of the previous unit, the current cumulative displacement field, and the second physical prior intermediate quantity into a multi-channel feature tensor. An inertial fusion neural network is configured to process the output of the physical and data fusion unit to obtain the inertial correction amount; The cumulative displacement field update unit is configured to sum the inertial correction amount, the current cumulative displacement field, and the single-step displacement field update amount of the current unit to obtain the final displacement field; The physical iterative registration algorithm is a series of Demons algorithms, including one or a combination of the Thirion original Demons algorithm, the Wang symmetric Demons algorithm, or the Santos-Ribeiro inertial Demons algorithm; The first and second physical prior intermediate quantities include at least: the intensity residual field of the fixed image and the deformed image, the horizontal and vertical offsets of the fixed image side, the horizontal and vertical offsets of the deformed image side, the confidence map of the fixed image side, and the confidence map of the deformed image side.

8. An electronic device, characterized in that: The method includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that: Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images