Combining implicit neural representations with bidirectional multi-scale image registration methods and systems

Abstract

The present application relates to the technical field of image registration, and provides a method and system for image registration combining implicit neural representation and bidirectional multi-scale, which comprises the following steps: obtaining floating images and reference images at multiple different low-resolution scales; obtaining an initial deformation field through a continuous-domain coarse registration network based on implicit neural representation; obtaining a high-resolution deformed image through a spatial transformation network; combining the high-resolution deformed image with the reference image; obtaining a high-resolution displacement field through a high-resolution branch U-shaped hierarchical encoding-decoding network; obtaining a low-resolution displacement field through a low-resolution branch U-shaped hierarchical encoding-decoding network after multi-scale gate fusion processing based on the floating images at the low-resolution scales and the initial deformation field; and obtaining a registration field based on the high-resolution and low-resolution displacement fields, so as to realize the alignment of the floating images and the reference images. The image registration precision is improved.

Description

Technical Field

[0001] This invention belongs to the field of image registration technology, and particularly relates to an image registration method and system that combines implicit neural representation with bidirectional multi-scale. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, fast and accurate registration methods for images such as lung 4D-CT (four-dimensional computed tomography) have achieved significant accuracy improvements in large deformation capture and topology preservation thanks to the continuous domain modeling capabilities of implicit neural representation (INR). However, they are essentially all instance-level optimization methods. Traditional INR methods usually rely on building a specific multilayer perceptron (MLP) to fit the coordinate mapping relationship of a single image pair. Although this one-example-one-training paradigm can generate high-quality smooth deformation fields, it means that for each new pair of clinical image data, the network needs to undergo time-consuming iterative optimization again. This characteristic greatly limits the application of INR technology in clinical scenarios with stringent requirements for real-time and high-throughput processing.

[0004] Therefore, the primary motivation for this research is to overcome the inherent instance-level fitting limitations of INR and construct a generalized registration model that can be trained once and inferred quickly multiple times, while retaining the advantages of INR in generating smooth deformation fields. However, constructing a generalized registration network faces two major unresolved challenges, which are also the bottlenecks of existing cascaded or pyramidal registration methods: First, existing multi-scale registration architectures lack bidirectional information interaction between scales: In traditional deep learning registration frameworks (such as LungRegNet, an unsupervised deep learning method for lung image registration), solving large deformation problems usually relies on a coarse-to-fine cascade strategy. Although these multi-scale methods alleviate the difficulty of deformation capture to some extent, they are generally limited by error propagation caused by unidirectional information flow. Under this unidirectional mechanism, the alignment error estimated in the coarse-scale stage will be irreversibly propagated to the fine-scale, making it difficult for subsequent networks to correct global misalignment. In addition, feature extraction at each scale is often isolated, cutting off the path of deep features to shallow layers and ignoring the strong coupling relationship between global semantics and local texture. Second, pure data-driven generalized registration lacks explicit biophysical constraints: Most current registration networks rely on image similarity loss (such as mean square error MSE or waveform similarity parameter NCC) to drive the process. Although they can achieve pixel-level alignment, they often ignore the anatomical and physiological characteristics of lung tissue (e.g., the lung is a highly complex heterogeneous organ containing bronchi, vascular trees, and lung parenchyma, and its movement has obvious topology preservation characteristics and tissue incompressibility). Although existing generalization methods introduce smoothing regularization terms, they lack explicit constraints for specific anatomical structures (such as lung lobe boundaries and tumor regions), which can easily lead to non-physical deformations (such as mesh folding or tissue tearing) in areas lacking texture. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides an image registration method and system that combines implicit neural representation with bidirectional multi-scale representation. The method transforms the INR into a conditional inference network, which retains the advantages of stepless resolution and continuous deformation modeling while achieving generalized inference with a single forward pass. Furthermore, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch, enabling the low-resolution branch to utilize high-level semantic information when recovering features, thereby improving the image registration accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A first aspect of the present invention provides a method for image registration combining implicit neural representation and bidirectional multi-scale methods, comprising: Multiple floating images and reference images at different low-resolution scales are acquired. After obtaining the initial deformation field through a continuous domain coarse registration network based on implicit neural representation, a high-resolution deformed image is obtained through a spatial transformation network. Combined with the high-resolution reference image, a high-resolution displacement field is obtained through a U-shaped hierarchical encoder-decoder network with a high-resolution branch. The continuous domain coarse registration network based on implicit neural representation includes: multi-scale gated fusion of the input floating image and reference image to obtain fusion features, which are then concatenated with a Fourier-encoded spatial coordinate grid to construct a vector containing position and texture information. Finally, the initial deformation field is regressed using a fully connected network based on a sinusoidal activation function. Based on the floating image and initial deformation field at a low resolution scale, after multi-scale gating fusion processing, a low-resolution displacement field is obtained through a U-shaped hierarchical encoder-decoder network with a low-resolution branch; among them, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through convolutional block attention. Based on the high-resolution and low-resolution displacement fields, a registration field is obtained, enabling the alignment of the floating image with the reference image.

[0007] Furthermore, after the U-shaped hierarchical encoder-decoder network of the high-resolution branch processes the input image into blocks, it obtains high-resolution bottleneck layer semantic features through the Transformer encoder, and then uses the decoder to upsample to obtain a high-resolution displacement field.

[0008] Furthermore, an image registration model is constructed based on a continuous domain coarse registration network, a spatial transformation network, a U-shaped hierarchical encoder-decoder network with a high-resolution branch, and a U-shaped hierarchical encoder-decoder network with a low-resolution branch, all based on implicit neural representation. The loss function used for training the image registration model is a weighted sum of image similarity loss, smoothing regularization loss, Jacobi determinant constraint, and anatomical structure-aware constraint. The anatomical structure-aware constraint adopts the introduced Sobel operator and structural similarity.

[0009] Furthermore, the multi-scale gating fusion includes: for the reference image and the floating image, a dual-branch feature extraction architecture is used to capture local and global information in parallel, and multi-scale feature representations are generated through channel concatenation and convolution aggregation; after obtaining the multi-scale feature representations of the reference image and the floating image, a differential-aware gating fusion mechanism is introduced to obtain the fused features.

[0010] Furthermore, the Transformer encoder includes multiple hierarchical visual transformers connected in sequence, and the hierarchical visual transformers introduce a window-based self-attention mechanism and a moving window self-attention mechanism.

[0011] A second aspect of the invention provides an image registration system combining implicit neural representation and bidirectional multi-scale methods, comprising: The high-resolution module is configured to: acquire multiple floating images and reference images at different low-resolution scales; obtain the initial deformation field through a continuous domain coarse registration network based on implicit neural representation; obtain a high-resolution deformed image through a spatial transformation network; combine the high-resolution reference image with a U-shaped hierarchical encoder-decoder network with a high-resolution branch to obtain a high-resolution displacement field; wherein, the continuous domain coarse registration network based on implicit neural representation includes: performing multi-scale gated fusion on the input floating image and reference image to obtain fusion features, and concatenating them with a Fourier-encoded spatial coordinate grid to construct a vector containing position and texture information; and then using a fully connected network based on a sinusoidal activation function to regress the initial deformation field; The low-resolution module is configured to: based on the floating image and initial deformation field at a low-resolution scale, after multi-scale gating fusion processing, obtain the low-resolution displacement field through a U-shaped hierarchical encoder-decoder network of the low-resolution branch; wherein, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through convolutional block attention. The registration module is configured to obtain a registration field based on high-resolution and low-resolution displacement fields, thereby aligning the floating image with the reference image.

[0012] Furthermore, after the U-shaped hierarchical encoder-decoder network of the high-resolution branch processes the input image into blocks, it obtains high-resolution bottleneck layer semantic features through the Transformer encoder, and then uses the decoder to upsample to obtain a high-resolution displacement field.

[0013] Furthermore, an image registration model is constructed based on a continuous domain coarse registration network, a spatial transformation network, a U-shaped hierarchical encoder-decoder network with a high-resolution branch, and a U-shaped hierarchical encoder-decoder network with a low-resolution branch, all based on implicit neural representation. The loss function used for training the image registration model is a weighted sum of image similarity loss, smoothing regularization loss, Jacobi determinant constraint, and anatomical structure-aware constraint. The anatomical structure-aware constraint adopts the introduced Sobel operator and structural similarity.

[0014] Furthermore, the multi-scale gating fusion includes: for the reference image and the floating image, a dual-branch feature extraction architecture is used to capture local and global information in parallel, and multi-scale feature representations are generated through channel concatenation and convolution aggregation; after obtaining the multi-scale feature representations of the reference image and the floating image, a differential-aware gating fusion mechanism is introduced to obtain the fused features.

[0015] Furthermore, the Transformer encoder includes multiple hierarchical visual transformers connected in sequence, and the hierarchical visual transformers introduce a window-based self-attention mechanism and a moving window self-attention mechanism.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention transforms the INR into a conditional inference network, which retains the advantages of stepless resolution and continuous deformation modeling while achieving generalized inference with a single forward pass. Furthermore, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch, enabling the low-resolution branch to utilize high-level semantic information when recovering features, thereby improving image registration accuracy. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is a flowchart of the image registration method combining implicit neural representation and bidirectional multi-scale in Embodiment 1 of the present invention; Figure 2 This is a structural diagram of the Multi-Scale Gated Fusion (MSGF) module of Embodiment 1 of the present invention; Figure 3 This is a structural diagram of the U-shaped hierarchical encoding-decoding network of Embodiment 1 of the present invention; Figure 4 This is a structural diagram of two consecutive Swing Transformer blocks in Embodiment 1 of the present invention; Figure 5 This is a structural diagram of the CBAM according to Embodiment 1 of the present invention; Figure 6 This is a structural diagram of the gated feature fusion according to Embodiment 1 of the present invention; Figure 7 This is a schematic diagram of the source image and the image slices after edge extraction in Embodiment 1 of the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0019] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] Example 1 This embodiment provides a method for image registration that combines implicit neural representation with bidirectional multi-scale representation.

[0021] The image registration method combining implicit neural representation and bidirectional multi-scale provided in this embodiment aims to complete the alignment from large global deformations to subtle local deformations through a single forward inference.

[0022] The image registration method combining implicit neural representation and bidirectional multi-scale provided in this embodiment addresses the problems of low utilization of high-level semantics and progressive error accumulation in the traditional "coarse-to-fine" unidirectional cascade strategy by designing a bidirectional feedback mechanism: the initial deformation field obtained by the continuous domain coarse registration network with implicit neural representation at the low-resolution scale provides basic localization support for the accurate registration of the high-resolution branch, avoiding registration failure or accuracy reduction due to initial localization deviation of the high-resolution branch, and significantly improving the efficiency and stability of solving the high-resolution displacement field; the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch as prior knowledge through the convolutional block attention module. This design enables the low-resolution branch to utilize high-level semantic information when recovering features, effectively compensating for the lack of detailed information in the low-resolution image, improving the accuracy of the low-resolution displacement field, and simultaneously achieving deep fusion and complementarity of features between high and low resolution scales, thus improving the image registration accuracy.

[0023] The image registration method provided in this embodiment, which combines implicit neural representation with bidirectional multi-scale, utilizes the Sobel operator to extract significant anatomical edge information in the image and combines it with the structural similarity loss function to impose topological and texture consistency constraints on the registration deformation field. This effectively compensates for the shortcomings of traditional pixel-level MSE loss in handling blurred boundaries, forces the model to focus on anatomical contours with clear biological significance, and ensures that the registration results retain high-frequency details while conforming to the biomechanical continuity of lung tissue.

[0024] This embodiment argues that the registration geometry also requires a top-down interaction mechanism across scales. An ideal registration network not only needs to transfer the deformation field from low resolution to high resolution, but also needs to establish a bidirectional feedback mechanism so that high-level anatomical semantics can guide the feature extraction of lower levels, thereby simultaneously achieving the capture of large global deformations and the refinement of local fine textures in a single network.

[0025] On the other hand, to overcome the limitations of traditional discrete grids in handling complex deformations, we explored registration methods combining explicit and implicit methods, as well as dynamic registration methods that enhance temporal features. While these methods based on implicit neural representations have achieved significant results in maintaining topological smoothness due to their advantage of continuous coordinate modeling, traditional implicit neural representation methods are inherently limited by instance-level optimization. Their one-example-one-training mechanism requires re-optimizing network weights for each new image pair, resulting in high computational costs and difficulty meeting the real-time requirements of clinical practice. Therefore, the primary motivation for this chapter's research is to break through the instance-fitting limitations of INR and deeply integrate its continuous modeling advantages with a generalized multi-scale architecture.

[0026] Furthermore, both multi-scale CNNs (Convolutional Neural Networks) and generalized implicit neural representation networks lack explicit biophysical constraints in purely data-driven generalized registration. Most current registration networks rely on image similarity losses (such as MSE or NCC), which, while achieving pixel-level alignment, often ignore the anatomical and physiological characteristics of lung tissue. The lung is a highly complex and heterogeneous organ, comprising bronchi, vascular trees, and lung parenchyma, exhibiting significant topology-preserving properties and tissue incompressibility. Although existing research has attempted to introduce anatomical constraints—DART (Dynamic Adaptive Reconstruction and Transformation Model) utilizes keypoint supervision to maintain structural correspondences—it typically relies heavily on cumbersome annotation or upstream segmentation networks. Therefore, how to embed biophysical constraints consistent with anatomical understanding into deep networks to guide the network in generating deformation fields that are both accurate and physiologically sound is another core issue addressed in this embodiment.

[0027] To address the shortcomings of existing registration networks in multi-scale interaction mechanisms, slow INR instance-level inference, and limited ability to preserve edge structures, this embodiment proposes a bidirectional multi-scale registration network based on implicit neural representation: A generalized conditional implicit neural representation framework was constructed: a multi-scale gated fusion (MSGF) module was proposed, which abandoned the traditional pure coordinate mapping method and instead extracted spatially aligned local anatomical features as prior conditions and injected them into the network. This design transformed the INR into a conditional inference network, which, while retaining the advantages of stepless resolution and continuous deformation modeling, achieved generalized inference with a single forward pass and an inference speed of 0.33 seconds, greatly improving clinical usability. A bidirectional feature interaction architecture based on an attention mechanism was designed: abandoning the traditional unidirectional concatenation strategy, a feature pyramid with a bidirectional feedback mechanism was constructed based on the Swin Transformer. By introducing the convolutional block attention mechanism (CBAM), a top-down semantic interaction path was established, injecting deep high-level anatomical semantic priors into the low-resolution decoding stream. This cross-scale semantic guidance mechanism can effectively correct local biases at the coarse scale and significantly improve the robustness of the network in dealing with complex respiratory movements. A lightweight anatomical structure-aware constraint strategy is proposed: to address the problem of weak constraint force of pixel-level loss in homogeneous regions, a hybrid loss function integrating the Sobel edge operator and structural similarity (SSIM) is designed. This mechanism directly and explicitly captures the boundaries of blood vessels and bronchi in the gradient domain, forcibly maintaining the topological consistency of biological tissues. Experiments on the authoritative DIR-Lab dataset show that this method not only achieves a low target registration error (TRE) of 1.28 mm, but also controls the folding rate of the deformation field at a low level of 0.009%, ensuring the biomechanical rationality and continuity of the registration results.

[0028] The image registration method combining implicit neural representation and bidirectional multi-scale provided in this embodiment designs an end-to-end registration network that combines implicit neural representation and bidirectional feature feedback. This network aims to complete the alignment from large global deformations to subtle local deformations through a single forward inference. The overall registration process mainly includes the following two stages: The first stage is a coarse registration in the continuous domain based on implicit neural representations: First, the input floating image and the reference image are processed by a multi-scale gated fusion module (MSGF) to extract deep features at low resolution scales of 1 / 8 and 1 / 4. In order to take advantage of the fitting characteristics of implicit neural representations in the continuous domain, the network concatenates these fused features with a spatial coordinate grid encoded by Fourier transform to construct an input vector containing position and texture information. Subsequently, a fully connected network based on a sinusoidal activation function is used to regress a continuous displacement field. The initial deformation field output in this stage is mainly used to capture large global deformations between images and to pre-transform the moving image to provide an alignment basis for the next stage. The second stage is fine registration based on bidirectional feature interaction: In this stage, the network runs two SwinTransformer branches in parallel, processing image features at 1 / 2 high resolution and 1 / 4 low resolution respectively. To achieve information interaction between scales, the network is designed with a bidirectional feedback mechanism: the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through the Convolutional Block Attention (CBAM) module. This design allows the low-resolution branch to utilize high-level semantic information when recovering features. Finally, the residual deformation field generated in the fine registration stage is combined with the initial deformation field in the coarse registration stage at multiple scales to output the final registered deformation field. .

[0029] The image registration method combining implicit neural representation and bidirectional multi-scale provided in this embodiment, such as... Figure 1 As shown, it includes the following steps: Step 1: Obtain floating images and reference images at multiple different low-resolution scales (1 / 8 and 1 / 4).

[0030] Step 2: Based on the floating images at low resolution scales of 1 / 8 and 1 / 4 and the reference image, coarse registration in the continuous domain based on implicit neural representation (INR) is performed to obtain the initial deformation field.

[0031] First, the input floating image and the reference image are processed by a multi-scale gated fusion (MSGF) module, which extracts deep features at low resolution scales of 1 / 8 and 1 / 4 to obtain fused features. In order to take advantage of the fitting characteristics of implicit neural representations in the continuous domain, the fused features are concatenated with a spatial coordinate grid encoded by Fourier transform to construct an input vector containing position and texture information. Subsequently, a continuous displacement field is regressed using a fully connected network based on a sinusoidal activation function. The initial deformation field output in this stage is mainly used to capture large global deformations between images and to pre-transform the floating image to provide an alignment basis for the next stage.

[0032] Step 101: Multi-scale feature gating fusion.

[0033] To overcome the inherent limitations of traditional implicit neural representation methods in instance-level optimization during registration tasks, the Multi-Scale Gated Fusion (MSGF) module employs a feature extraction and fusion strategy based on conditional injection. The structure of the MSGF module is as follows: Figure 2 As shown.

[0034] (1) First, a dual-branch feature extraction architecture was designed to capture local and global information in parallel.

[0035] Local detail branches are used The small convolutional kernels are stacked to focus on extracting high-frequency boundaries such as blood vessel texture; the global context branch uses... Large convolutional kernels are used to expand the receptive field and capture large-scale topological structures such as lung lobe contours.

[0036] Combining the residual connection mechanism, the output characteristics of the two branches and (Its length, width, and height are c2, w, and h respectively) can be calculated as follows: ; ; in, Represents nonlinear convolution transformation. These are learnable weight parameters. The floating image and the reference image have length, width, and height of c1, w, and h, respectively.

[0037] (2) Channel splicing Concat and convolution aggregation C fuse This generates multi-scale feature representations containing rich spatial context (i.e., multi-scale feature extraction). This lays the foundation for subsequently introducing image priors into the implicit continuous domain: ; (3) Obtaining multi-scale feature representations of the reference image (reference image features) Multi-scale feature representation of floating images (moving image features) Subsequently, to ensure that the conditional features of the injected INR can reflect the current registration error, a differential-aware gated fusion mechanism was introduced, such as... Figure 6 As shown.

[0038] Unlike simple feature overlay, the difference-aware gated fusion mechanism first calculates the difference map in the feature space. and compare it with the original features ( and The components are concatenated along the channel dimension to construct a joint tensor. Subsequently, a spatial gating map is generated using this joint feature. This is used to adaptively evaluate the degree of mismatch at each voxel location: ; in, It is a sigmoid activation function. For gated convolution weights, This represents the convolution operation; Using spatial gating maps, fused features are obtained. The following formula is used to calculate and ensure that the network assigns higher weights to the difference features in regions with significant registration differences: ; in, This represents the normalization and activation function (Norm+ReLU). This represents the weights of the convolution kernel parameters during the feature fusion process. This indicates multiplication by channel.

[0039] Ultimately, the gating fusion generated Beyond simply transmitting information between layers, the crucial role is to serve as a spatial-feature condition for subsequent implicit neural representation modules. In traditional optimized INRs, only a mapping between coordinates and deformation fields is established; however, in this embodiment, these fused features, containing multi-scale anatomical information and explicit error cues, are concatenated with the coordinate grid after Fourier position encoding to jointly constitute the INR's input vector. This design essentially transforms the implicit network's task from fitting coordinate mappings for specific image pairs to learning deformation regression rules based on image features. This enables the model to generalize, allowing it to directly perform forward inference on unseen image pairs without requiring instance-level iterative optimization.

[0040] Step 102, Implicit neural representation branches within the scale.

[0041] In the coarse registration stage, to enable continuous modeling of large global deformations of lung tissue in a resolution-independent manner, a regression branch based on Conditional Implicit Neural Representation (INR) was designed. Unlike traditional discretized displacement field estimation methods, this INR-based regression branch utilizes a multilayer perceptron (MLP) to construct a nonlinear mapping function from spatial coordinates to deformation displacement. To enhance the network's ability to capture high-frequency details and achieve cross-subject generalization inference, the INR-based regression branch underwent the following targeted design in input construction and network structure.

[0042] (1) Fourier position coding.

[0043] Because multilayer perceptrons are inherently biased towards learning low-frequency functions (i.e., spectral bias), they directly use low-dimensional spatial coordinates... The input to a multilayer perceptron (MLP) can result in an overly smooth output deformation field, making it difficult to fit regions with high-frequency variations, such as the lung lobe edges. To overcome this bottleneck, a Fourier feature mapping strategy is introduced, projecting the coordinates onto a high-dimensional frequency space before inputting them into the MLP. For the input coordinates (i.e., the coordinates of the pixels in the input image),... Using a set of linearly spaced frequency components Encode, encoding function The definition is as follows: ; Each frequency component It includes both sine and cosine components to ensure the completeness of the position information: ; Through the above mapping, the original 3D coordinates are expanded to... The operation of adding high-frequency feature vectors significantly expands the bandwidth of the input signal, enabling the implicit network to more sensitively perceive subtle changes in spatial location, thereby improving the fitting accuracy for complex deformations.

[0044] (2) Spatial-feature condition injection.

[0045] Traditional implicit neural representation methods typically rely solely on coordinate input to fit a single scene, failing to handle unseen image data. To endow the network with generalized inference capabilities, a "conditional injection" strategy is employed. This involves integrating the fused features extracted by the multi-scale gating module in step 101. As a priori condition, it is concatenated with the coordinate vector after position encoding along the channel dimension, and the sampling point is set. The image features at that location are Then the final vector input to the MLP The construction is as follows: ; This design transforms the implicit representation task from simple "coordinate mapping" to "coordinate mapping based on image context." In this case, the MLP no longer memorizes the deformation of specific image pairs, but learns how to represent local anatomical texture differences (as determined by...). (Provide) to infer the spatial displacement of that location.

[0046] (2) In the selection of the network architecture of MLP, the traditional linear rectified activation function (ReLU) was abandoned. This is because the second derivative of ReLU is zero, resulting in a deformation field that lacks smooth curvature information in space, making it difficult to meet the topological requirements of differential homeomorphism transformation for medical image registration. Based on the exploration and verification of the characteristics of implicit neural representation, MLP adopted Sinusoidal Representation Networks (SIREN) as the core fitter. SIREN uses a periodic sine function as the core activation function. It is assumed that the SIREN network... The input of the layer is The weight matrix is , bias is Then the first Layer output The calculation formula is: ; in, This is a frequency scaling factor used to control the frequency response range of the activation function, ensuring that the spatial information stored in the weights can be effectively transmitted.

[0047] Furthermore, to ensure the training stability of deep SIREN networks, the SIREN network weights employ a special initialization distribution, which is used for input dimensions of... The initial network weights of the MLP layer Sampling from a uniform distribution: ;in, The internationally accepted standard mathematical symbol for representing a uniform distribution in probability theory.

[0048] By stacking multiple layers of such sinusoidal activation layers, the SIREN network eventually regresses the 3D displacement vector at that location via a linear output layer. This SIREN-based design enables the output deformation field to not only numerically fit the target displacement, but also maintain good smoothness and continuity at the derivative level (such as the Jacobian determinant), making it suitable for describing large-amplitude non-rigid deformations generated by lung breathing.

[0049] Step 2: Based on the floating image at a low resolution of 1 / 4 and the initial deformation field, a deformed image at a high resolution of 1 / 2 is obtained through STN (Spatial Transformer Network).

[0050] Step 3: Based on the floating image at a low resolution scale of 1 / 4 and the initial deformation field, the deformed image at a high resolution scale of 1 / 2, and the reference image at a high resolution scale of 1 / 2, a registration field (φ) with full resolution is obtained through fine registration based on bidirectional feature interaction, thus achieving accurate alignment between the fixed image and the moving image.

[0051] In this stage, two Swin Transformer branches run in parallel, processing image features at 1 / 2 high resolution and 1 / 4 low resolution respectively. To achieve information exchange between scales, a bidirectional feedback mechanism is designed: the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through a Convolutional Block Attention (CBAM) module. This design allows the low-resolution branch to utilize high-level semantic information when recovering features. Finally, the residual deformation field generated in the fine registration stage is multi-scale composited with the initial deformation field in the coarse registration stage to output the final registered deformation field. .

[0052] Step 301: For the 1 / 2 high-resolution deformed image and the 1 / 2 high-resolution reference image, a high-resolution displacement field is obtained through a U-shaped layered encoding-decoding network.

[0053] While implicit neural representations excel at handling continuous deformations, the quality of their input features directly determines the accuracy of the final registration. To overcome the limitations of traditional convolutional neural networks, such as limited receptive fields and difficulty in capturing long-range anatomical dependencies, a U-shaped hierarchical encoder-decoder network based on the Swing Transformer was designed, such as... Figure 3 As shown, the U-shaped hierarchical encoder-decoder network adopts a symmetrical U-shaped architecture and realizes the mapping from semantic understanding to pixel-level deformation regression through hierarchical feature extraction and reconstruction.

[0054] (1) The input medical image is first processed into blocks by the Patch Partition module. Assume the size of the input feature map is... (Where H, W, and L represent the resolution of the feature map in the three spatial dimensions, and C is the number of feature channels), the patch partitioning module divides it into non-overlapping... The local patches are then flattened and mapped to arbitrary dimensions using linear embedding layers and normalization. (Set to 96 in this experiment), this process converts image data into serialized token input, which not only reduces the resolution of subsequent calculations ( It also effectively preserves local texture information.

[0055] (2) Based on the serialized token input, the high-resolution bottleneck layer semantic features are obtained through the Transformer encoder. .

[0056] The Swin Transformer block is the core extraction unit of the Transformer encoder. In order to solve the problem that the computational complexity of the standard Vision Transformer increases quadratically with the image resolution, the Swin Transformer block introduces a window-based multi-head self-attention (W-MSA) mechanism and a shifted-window-based multi-head self-attention (SW-MSA) mechanism.

[0057] The calculation process is as follows in two consecutive Swin Transformer blocks: (A) W-MSA (Window-based Self-Attention): First, the input features are divided into several sizes. The local 3D (three-dimensional) window allows self-attention computation to be performed independently only within each window, thus reducing computational complexity from... Reduced to a value linear with the total number of image voxels. For features within the window The attention formula is defined as: ; in, , and These are query, key, and value matrices, respectively. For the feature dimension; specifically, the formula introduces a relative position bias. Compared to absolute position encoding, relative position offset can better preserve the translation invariance of an image, which is crucial for capturing relative displacement in registration tasks. (B) SW-MSA (Walking Window Self-Attention): Using only W-MSA will result in a lack of information interaction between windows. Therefore, in the next block, the window's division position will cyclically move towards the lower right corner. Individual features, this movement mechanism introduces cross-window connections, enabling features to interact globally across boundaries.

[0058] like Figure 4 As shown, the mathematical expression for two consecutive blocks is:

[0059] in, Representation layer normalization. It is a two-layer multilayer perceptron containing the GELU (Gaussian error linear unit) activation function; SW-MSA represents moving window self-attention, and W-MSA represents window-based self-attention; Represents the index of the current network layer; Indicates the first The output features of the layer are also the first layer. Input for each block (or layer); and They represent the first time. Layer and first In the layer, after layer normalization ( The intermediate feature representation is obtained after calculation by the corresponding self-attention module MSA (W-MSA or SW-MSA); and They represent the first Layer and first The final output features of the layers are derived from intermediate features through normalization and multilayer perceptron (MLP). After processing, the residual is added to the intermediate features to obtain the result.

[0060] In the Transformer encoder path, as the network depth increases, the model utilizes PatchMerging layers to construct a hierarchical feature pyramid (such as...). Figure 3 shown (hierarchy), by neighboring Patch concatenation is used to achieve downsampling, which halves the feature map resolution while doubling the number of channels. This allows for the gradual extraction of more abstract, high-level semantic features.

[0061] (3) Based on the high-resolution bottleneck layer semantic features, the decoder obtains the high-resolution displacement field through upsampling.

[0062] Step 302: For the floating image and initial deformation field at a low resolution scale of 1 / 4, after processing by the MSFG module, the low-resolution displacement field is obtained through a U-shaped hierarchical encoding-decoding network.

[0063] In the deepest layer (Bottleneck) of the Transformer encoder in the U-shaped hierarchical encoder-decoder network of the low-resolution branch, a Convolutional Block Attention Module (CBAM) is introduced to further enhance the ability to represent key anatomical structures and prepare for cross-scale interaction. Figure 3 As shown, the CBAM module adaptively suppresses background noise and enhances saliency features through a concatenated channel attention and spatial attention mechanism, and fuses them with the original features using residual connections. Subsequently, the decoder path forms a symmetrical structure with the Transformer encoder through a pixel shuffle operation, gradually halving the number of feature channels and increasing the resolution to restore the original size. During this process, skip connections are used to concatenate the shallow texture details of the corresponding layer of the encoder with the deep global semantics of the decoder, ultimately effectively compensating for the spatial information loss caused by downsampling and outputting a low-resolution displacement field. .

[0064] In this embodiment, to overcome the shortcomings of traditional cascaded registration frameworks where networks at different scales are independent and lack semantic interaction, an inter-scale feature feedback mechanism based on Convolutional Block Attention (CBAM) is designed. Traditional coarse-to-fine strategies often only pass low-resolution flow fields as initial values, ignoring the guiding role of rich anatomical topological information (such as lung lobe boundaries and vascular topology) in high-resolution images for low-level feature matching. To compensate for this deficiency, a top-down feedback path is constructed using a U-shaped hierarchical encoder-decoder network in the low-resolution branch, incorporating the bottleneck features of the deepest layer of the Transformer encoder in the high-resolution branch (1 / 2 scale). Extracted and injected as prior knowledge into the decoding process of the low-resolution branch (1 / 4 scale), such as... Figure 5 As shown, it includes: (1) In the decoder of the low-resolution branch, the high-level semantic features that have been upsampled and aligned are first... With current decoding features By performing channel concatenation (Concat), a joint feature tensor containing multi-scale information is constructed. (Its length, width, and height are c, w, and h, respectively): ; To filter out key clues useful for registration from redundant joint features, a CBAM module was introduced. To refine this process, the first step is to identify "what features are important" through the channel attention submodule.

[0065] (2) The CBAM module performs global average pooling (AvgPool) and max pooling (MaxPool) in the spatial dimension to aggregate the global spatial context. Then, the two aggregated vectors are feature mapped through a shared multilayer perceptron, and the outputs are summed element by element and activated by the Sigmoid function to generate a channel attention weight map. By using this weight map to weight the input features, it is possible to adaptively enhance discriminative anatomical texture channels while suppressing irrelevant background noise channels. ; ; in, This indicates multiplication by channel. This represents the Sigmoid activation function.

[0066] (3) Channel attention features after channel refinement Further processing through the spatial attention submodule is used to identify "where the important deformation regions are".

[0067] The spatial attention submodule focuses on features along the channel dimension. Average pooling and max pooling compression are performed to generate a two-dimensional descriptor that reflects spatial saliency; subsequently, using... Convolutional layers ( Spatial context aggregation is performed on the concatenated descriptors to generate a spatial attention map. Finally, residual connections are used to fuse the feedback information, which has undergone dual attention filtering, with the original low-resolution features. This design ensures that the low-resolution branch can explicitly utilize high-level semantic information to correct local mismatches when recovering spatial details, thereby significantly improving the robustness of the registration model at complex anatomical structures. ; ; Through the above mechanism, the CBAM module acts as a "filter" and "amplifier" for inter-scale information. It suppresses irrelevant background information in high-resolution feedback features while enhancing semantic features related to the current anatomical structure matching, thus ensuring the robustness and accuracy of the bidirectional multi-scale registration framework.

[0068] Step 303: Based on the high-resolution displacement field and the low-resolution displacement field, a registration field (φ) with full resolution is obtained, so as to achieve accurate alignment between the floating image and the reference image.

[0069] An image registration model is constructed using a continuous-domain coarse registration network based on implicit neural representation, a spatial transformation network, and a fine registration network based on bidirectional feature interaction. To significantly improve the alignment accuracy of the image registration model for complex lung anatomical structures (such as fine blood vessels, bronchial walls, and lobar boundaries) while ensuring the smoothness and physical plausibility of the registration field, a hybrid loss function with multiple constraints is built. The total loss function is... It is composed of a weighted combination of its parts, as defined below: ; in, , and These are image similarity loss, smoothing regularization loss, and Jacobian determinant constraint, respectively. This introduces anatomical structure-aware constraints based on the Sobel operator and SSIM (structural similarity) for generalization tasks.

[0070] To address the low resolution (2.5mm) and anisotropy of lung 4D-CT data in the Z-axis direction, a vectorized slice-level edge extraction module was designed to avoid inter-slice artifacts caused by directly applying 3D operators. Specifically: Input 3D volume data Remodeled into a stacked form of two-dimensional slices Subsequently, using Sobel convolution kernel and Parallel computation of the horizontal and vertical gradients for each slice: ; ; ; ; Where B represents the batch size, C is the number of channels, D is the number of slice layers of the 3D graphic on the z-axis, and H and W are the spatial resolution of a single 2D slice on the X and Y axes, respectively. Represents the two-dimensional spatial pixel coordinates on the slice , and These represent the horizontal and vertical spatial gradients of the input slice at pixel p, respectively.

[0071] To obtain robust edge feature maps The L1 norm sum of the gradients is calculated, and slice-level dynamic normalization is introduced to eliminate CT value distribution differences. For the th A slice, its edge map The calculation is as follows: ; ; in, This indicates taking the absolute value (L1 norm). Here, the sum of absolute values ​​is used to approximate the total gradient magnitude. Compared to calculating the sum of squares and then taking the square root (L2 norm), the L1 norm is more efficient and preserves the edges of tiny blood vessels in medical images better. and These represent the gradient maps of the k-th slice in the horizontal (x-axis) and vertical (y-axis) directions, respectively. Indicates the first The raw edge map of each slice; and These represent the minimum and maximum gradient values ​​in the original edge feature map of the slice, respectively. This represents a very small smoothing constant. It is introduced to prevent the denominator from being zero and to ensure the stability of numerical calculations.

[0072] like Figure 7 As shown, after the original CT image (T00 and T50 phases) is processed by the vectorized slice-level edge extraction module, the complex texture of the lung is transformed into a high-contrast anatomical edge map. It can be clearly observed that both the outer contour of the lung lobe and the fine vascular tree structure inside the lung parenchyma are significantly enhanced after edge extraction. This feature transformation effectively filters out grayscale noise caused by artifacts or density fluctuations, enabling the model to focus on topological structural information with clear biological significance.

[0073] After extracting the edge features of the reference image and the deformed floating image, the structural similarity index (SSIM) is introduced as a metric to construct the structural consistency loss. Unlike traditional mean square error (MSE), which only focuses on pixel-level numerical differences, and unlike MSE, which only focuses on pixel errors, SSIM considers brightness (… ), contrast ( ) and structure ( The comprehensive evaluation is conducted across three dimensions, which aligns closely with the topological consistency requirements of anatomical structures, and applies to the two edge map windows. and Its SSIM definition is: ; ; in, , and These represent the local mean, variance, and covariance, respectively. , , respectively, are the mean values ​​of the local windows corresponding to images x and y; , are the standard deviations of the local windows corresponding to images x and y, respectively; Let be the covariance of the corresponding local windows of the two images; To avoid the instability of calculations caused by the denominator approaching zero; These represent the weights of brightness, contrast, and structure in the overall evaluation (in practical applications, these are usually the default values). It is usually determined by the dynamic range of the image.

[0074] The final anatomical structure perception loss is defined as the dissimilarity between edge maps: ; Where E represents the edge feature map extraction operation, Represents a fixed image. This indicates a moving image.

[0075] By minimizing this loss, the image registration model is forced to align during training. Figure 7 The high-frequency anatomical boundaries shown in the figure mean that even in lung parenchyma regions lacking significant gray-level gradients, the image registration model can infer the correct deformation field based on edge cues from blood vessels and bronchi, thereby significantly improving the robustness and alignment accuracy of the generalization model in complex texture regions.

[0076] Experimental evaluation and analysis.

[0077] (1) Data set and preprocessing.

[0078] In terms of dataset selection, to comprehensively evaluate the performance of the generalized registration network in different clinical scenarios, this experiment used three internationally recognized publicly available 4D-CT datasets of the lungs. Among them, TheCancer Imaging Archive (TCIA) 4D-Lung database served as the core data source, containing complete respiratory cycle images (10 phases per case) of 82 patients with non-small cell lung cancer. This dataset has a large sample size and covers diverse tumor locations and respiratory patterns, making it suitable for training the generalization ability of deep networks. In addition, the DIR-Lab (Deformable Image Registration Laboratory Dataset) and POPI (Public Dataset for Lung Image Registration) datasets were continued, utilizing their high-precision landmark annotations to quantitatively evaluate registration errors.

[0079] In terms of data preprocessing, given the heterogeneity of the original images in terms of acquisition equipment and scanning parameters, this chapter implements a unified standardized processing workflow. First, the voxel spacing of all data is resampled to... mm³. Specifically, the original slice thickness of 2.5 mm was retained in the Z-axis direction instead of being forcibly interpolated to 1.0 mm, aiming to minimize the staircase effect and artifacts introduced by oversampling, thereby ensuring the fidelity of the anatomical structure. Subsequently, center cropping was performed based on the centroid of the lung mask to uniformly fix the image spatial dimensions to [missing value]. This is done to accommodate the model's requirement for integer divisibility of the input size. Finally, the grayscale values ​​of all images are truncated to... HU (Henness unit) and normalized to The interval is used to accelerate network convergence.

[0080] In terms of dataset partitioning and experimental setup, a hybrid training and cross-center validation strategy was adopted. The training set integrates the anatomical diversity of TCIA with the high-quality textures and large-amplitude motion features of the intermediate phases of DIR-Lab to cover a wide range of deformation patterns. To specifically monitor the model's convergence under large-amplitude deformation scenarios, this experiment reserved the maximum end-inspiratory and maximum end-expiratory (T00-T50) image pairs with the largest anatomical displacements in the DIR-Lab dataset to monitor the model's fitting ability under extreme motion scenarios within the domain. In addition, six images from the POPI dataset were used as a completely independent test set to evaluate the model's zero-shot inference ability and generalization robustness on unseen cross-center data.

[0081] (2) Implementation details and evaluation indicators.

[0082] The bidirectional multi-scale generalization registration network is built on PyTorch (an open-source framework for machine learning and deep learning), and all experiments were performed on a hardware platform. Regarding the specific parameter settings of the network structure, the in-scale conditional implicit neural representation (INR) branch uses a multilayer perceptron with three hidden layers, each containing 128 neurons, and a frequency scaling factor is selected. The sine function is used as the activation function to ensure a keen perception of high-frequency spatial details; in the feature feedback branch between scales, the basic embedding dimension of the Swin Transformer backbone is set to 96, and the depths of the four stages of the encoder are respectively... The corresponding number of multi-head attention heads are respectively Meanwhile, to adapt to the anisotropic resolution of 3D lung images, the size of the moving window was specifically set to... .

[0083] Regarding the training strategy, the network employs an end-to-end optimization approach, using the Adam (Adaptive Moment Estimation) optimizer with AMSGrad (an improved version of Adaptive Moment Estimation) enabled to ensure the stability of gradient updates. The initial learning rate is set to... The learning rate is dynamically adjusted using a polynomial decay strategy with an exponent of 0.9 to promote model convergence to the optimal solution. Due to the high memory usage of 3D images, the training batch size is set to 1, the maximum number of iterations is 100, and an early stopping mechanism is introduced, that is, training is terminated early when the validation set metrics do not show improvement within 10 consecutive epochs.

[0084] To maintain the continuity of the experimental evaluation framework in this paper and to ensure that the generalization model proposed in this embodiment can be fairly compared with instance-level optimization methods, this experiment continues to use the two core metrics established above: target registration error and the ratio of negative Jacobian determinant. The target registration error (TRE) is calculated based on expert markers in the DIR-Lab and POPI datasets, using the Euclidean distance between the reference point and the corresponding deformed point to quantify the alignment accuracy of the anatomical structure. The ratio of negative Jacobian determinant measures the topological fidelity and smoothness of the deformation field by statistically analyzing the proportion of voxels with a Jacobian determinant value less than or equal to 0, ensuring that the generated deformation conforms to the biomechanical continuity constraints of lung tissue.

[0085] (3) Comparison with advanced methods.

[0086] DIR-Lab dataset.

[0087] To thoroughly evaluate the local alignment accuracy of the registration algorithm when handling complex lung deformations, Table 1 details the target registration error (TRE) distribution of the proposed method and current mainstream algorithms (MIDIR (Medical Image Deformable Registration Network), VoxelMorph (Voxel Morphal Registration Network), ViT-V-Net (Vision Transformer Network for Volume Medical Image Registration), TransMorph (Transformer-based Morphal Registration Network), and UTSRMorph (Unified Medical Image Registration Network fusing Transformer and Super-resolution)) on all 10 cases of the DIR-Lab dataset. Based on the statistical results of the reproduction experiments, as shown in Table 2, the proposed method demonstrates a significant advantage in overall performance, achieving the lowest average error among all compared methods. Specifically, the average target registration error (TRE) of this method reaches... mm. Compared to the unsupervised benchmark VoxelMorph, which was reproduced under the same experimental conditions. Compared to TransMorph (mm), the method in this embodiment improves accuracy by approximately 24%, significantly overcoming the bottleneck of pure convolutional networks in feature alignment. Furthermore, compared to the leading Transformer-type methods in recent years, the method in this embodiment also demonstrates strong competitiveness, with an average accuracy superior to TransMorph (mm). mm) and the latest UTSRMorph ( In the case-by-case analysis in Table 1, the method of this embodiment achieved the best TRE values ​​in most cases, including cases 1 to 6, as well as cases 9 and 10, which fully demonstrates the generalization stability of the model under different respiratory amplitudes and anatomical textures.

[0088] Further focusing on challenging cases of large deformations, the superior robustness of this method in handling intense respiratory movements becomes clearly apparent. Cases 6 through 10 in the DIR-Lab dataset contain large initial displacements, especially Case 8, where the initial average error between the maximum inspiratory and expiratory phases reaches as high as 14.99 mm, making it a recognized high-difficulty registration case. In Case 8, the registration errors of the reproduced VoxelMorph and MIDIR models stalled at 2.30 mm and 3.47 mm, respectively, indicating that a single-scale network struggles to capture such large non-rigid deformations. In contrast, the method in this embodiment successfully reduced the TRE of Case 8 to 1.70 mm, outperforming the state-of-the-art method TransMorph (1.79 mm).

[0089] Table 2. TRE of several deep learning-based registration methods determined on DIR-Lab data

[0090] The results validate the key role of the implicit neural representation coarse registration module introduced in the first stage of this network. It successfully captures the global deformation field in the continuous domain, providing a good initial alignment foundation for subsequent fine registration and avoiding the model from getting trapped in local minima to a certain extent.

[0091] In medical image registration, besides pursuing pixel-level alignment accuracy, it is equally crucial that the generated deformation field conforms to the physical properties of biological tissues. Folding phenomena in the deformation field, i.e., the tearing or overlapping of voxels in space, not only violate the incompressibility of human tissues but also lead to serious biases in subsequent clinical analysis. Therefore, this chapter statistically analyzes the ratio of negative values ​​of the Jacobian determinant (…). The topological fidelity of the deformation field is quantified using a method described in this embodiment. Table 3 shows in detail the distribution of the negative Jacobian ratios of the method in this embodiment and the reproduced mainstream deep learning models in various cases of DIR-Lab.

[0092] Table 3. Comparison of folding rates of different methods on the Dir-Lab dataset

[0093] The statistical data in Table 3 clearly shows that the method proposed in this embodiment exhibits a significant order-of-magnitude advantage in topology preservation. While current advanced Transformer-based methods have improved accuracy, they often come at the cost of sacrificing the smoothness of the deformation field. For example, the average negative Jacobian ratios of TransMorph and UTSRMorph are 0.6035% and 0.5646%, respectively, and VoxelMorph also reaches 0.3618%. In contrast, the average negative Jacobian ratio of the method in this embodiment is only 0.0090% (i.e., ...). This means that the deformation field folding rate generated by the method in this embodiment is only 1 / 40 to 1 / 60 of that of the comparative method, almost achieving an ideal differential homeomorphic transformation. This advantage is particularly prominent in Case 8, where the deformation is most severe. Due to the large respiratory movements of the lungs, the folding rates of TransMorph and UTSRMorph in this case surged to 1.3359% and 1.1893%, respectively, indicating that the network produced severe non-physical distortions when handling large deformations. In contrast, the folding rate of this method was stably controlled at an extremely low level of 0.0171% in Case 8, demonstrating strong robustness. This superior topological property is mainly due to the implicit neural representation introduced in the first stage of this network. Unlike the interpolation operation performed on discrete grids by traditional convolutional networks, INR models the deformation field as a continuously differentiable function, mathematically guaranteeing the smoothness and continuity of the spatial mapping. Furthermore, combined with the anatomical structure-aware constraints designed in this embodiment, the gradient changes of the network in complex texture regions are further standardized, ensuring that the registration results meet the continuity requirements of lung biomechanics while maintaining high accuracy.

[0094] POPI dataset.

[0095] Table 4. TRE of several deep learning-based registration methods determined on POPI data

[0096] To verify the model's generalization ability on unseen data, the model trained only on the TCIA and DIR-Lab datasets was directly applied to the cross-center POPI dataset for testing. Tables 4 and 5 list the target registration error and the ratio of negative Jacobian determinant for each algorithm on the POPI dataset. From the statistical results in Table 4, the average TRE of the method in this embodiment is 1.84±1.35mm, which is better than the benchmark models VoxelMorph (2.51±1.69mm) and ViT-V-Net (2.75±2.00mm), and slightly better than the state-of-the-art method TransMorph (1.92±1.54mm). Although the latest UTSRMorph slightly leads in average accuracy (1.76±1.38mm), further observation of Case 2 (initial TRE of 14.04mm) with the largest initial error reveals that UTSRMorph's registration error is 3.13mm, while the method in this embodiment reduces it to 2.93mm. This indicates that when dealing with challenging samples such as those with large deformations, the method in this embodiment has stronger robustness due to the fitting advantage of implicit neural representations to continuous deformation fields.

[0097] Furthermore, the physical plausibility of the deformation field is equally important when evaluating registration quality. According to the Jacobi negative ratio data in Table 5, while UTSRMorph achieved a low TRE in some small-to-medium deformation cases, its average folding rate reached 0.6824%, especially in Case 2 with large deformation, where the folding rate was as high as 1.0924%, indicating significant non-physical overlap in the deformation field. In contrast, the method in this embodiment, while maintaining high-precision registration in Case 2, controlled the folding rate at 0.1690%, with an average folding rate of only 0.0806% (approximately 1 / 8 of UTSRMorph). These results demonstrate that the method proposed in this embodiment does not simply pursue the limits of numerical accuracy, but rather achieves a better balance between registration accuracy and topological fidelity, generating a deformation field that better conforms to biomechanical characteristics.

[0098] Table 5. Comparison of folding rates of different methods on the POPI dataset

[0099] Finally, regarding model complexity, the proposed bidirectional multi-scale network has 507.72M parameters, a high model capacity that brings a significant computational burden. This structural complexity is primarily driven by the architectural requirements for handling large lung deformations: specifically, the deployment of multi-scale gated fusion modules, the dense attention matrix in the Swing Transformer backbone for capturing long-range dependencies, and the increased MLP network depth required for accurate high-frequency fitting in the continuous domain all contribute to the expansion of the parameter space. However, thanks to the parallel computing efficiency of modern high-performance GPUs, the increase in model size has not resulted in unacceptable inference latency. The average time for a single forward inference is 0.3352 seconds. Despite the large computational cost of the model, this speed remains within the sub-second range required for real-time clinical interaction, thus ensuring its feasibility for practical deployment in time-sensitive radiotherapy workflows.

[0100] Example 2 The image registration system combining implicit neural representation and bidirectional multi-scale provided in this embodiment includes: The high-resolution module is configured to: acquire multiple floating images and reference images at different low-resolution scales; obtain the initial deformation field through a continuous domain coarse registration network based on implicit neural representation; obtain a high-resolution deformed image through a spatial transformation network; combine the high-resolution reference image with a U-shaped hierarchical encoder-decoder network with a high-resolution branch to obtain a high-resolution displacement field; wherein, the continuous domain coarse registration network based on implicit neural representation includes: performing multi-scale gated fusion on the input floating image and reference image to obtain fusion features, and concatenating them with a Fourier-encoded spatial coordinate grid to construct a vector containing position and texture information; and then using a fully connected network based on a sinusoidal activation function to regress the initial deformation field; The low-resolution module is configured to: based on the floating image and initial deformation field at a low-resolution scale, after multi-scale gating fusion processing, obtain the low-resolution displacement field through a U-shaped hierarchical encoder-decoder network of the low-resolution branch; wherein, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through convolutional block attention. The registration module is configured to obtain a registration field based on high-resolution and low-resolution displacement fields, thereby aligning the floating image with the reference image.

[0101] Furthermore, after the U-shaped hierarchical encoder-decoder network of the high-resolution branch processes the input image into blocks, it obtains high-resolution bottleneck layer semantic features through the Transformer encoder, and then uses the decoder to upsample to obtain a high-resolution displacement field.

[0102] Furthermore, an image registration model is constructed based on a continuous domain coarse registration network, a spatial transformation network, a U-shaped hierarchical encoder-decoder network with a high-resolution branch, and a U-shaped hierarchical encoder-decoder network with a low-resolution branch, all based on implicit neural representation. The loss function used for training the image registration model is a weighted sum of image similarity loss, smoothing regularization loss, Jacobi determinant constraint, and anatomical structure-aware constraint. The anatomical structure-aware constraint adopts the introduced Sobel operator and structural similarity.

[0103] Furthermore, the multi-scale gating fusion includes: for the reference image and the floating image, a dual-branch feature extraction architecture is used to capture local and global information in parallel, and multi-scale feature representations are generated through channel concatenation and convolution aggregation; after obtaining the multi-scale feature representations of the reference image and the floating image, a differential-aware gating fusion mechanism is introduced to obtain fused features.

[0104] Furthermore, the Transformer encoder includes multiple hierarchical visual transformers connected in sequence, and the hierarchical visual transformers introduce a window-based self-attention mechanism and a moving window self-attention mechanism.

[0105] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method combining implicit neural representation and bidirectional multi-scale image registration, characterized in that: include: Multiple floating images and reference images at different low-resolution scales are acquired. After obtaining the initial deformation field through a continuous domain coarse registration network based on implicit neural representation, a high-resolution deformed image is obtained through a spatial transformation network. Combined with the high-resolution reference image, a high-resolution displacement field is obtained through a U-shaped hierarchical encoder-decoder network with a high-resolution branch. The continuous domain coarse registration network based on implicit neural representation includes: multi-scale gated fusion of the input floating image and reference image to obtain fusion features, which are then concatenated with a Fourier-encoded spatial coordinate grid to construct a vector containing position and texture information. Finally, the initial deformation field is regressed using a fully connected network based on a sinusoidal activation function. Based on the floating image and initial deformation field at a low resolution scale, after multi-scale gating fusion processing, a low-resolution displacement field is obtained through a U-shaped hierarchical encoder-decoder network with a low-resolution branch; among them, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through convolutional block attention. Based on the high-resolution and low-resolution displacement fields, the registration field is obtained, and the alignment of the floating image with the reference image is achieved. The multi-scale gated fusion includes: for the reference image and the floating image, a dual-branch feature extraction architecture is used to capture local and global information in parallel, and multi-scale feature representations are generated by channel concatenation and convolution aggregation; after obtaining the multi-scale feature representations of the reference image and the floating image, a differential-aware gated fusion mechanism is introduced to obtain the fused features.

2. The image registration method combining implicit neural representation and bidirectional multi-scale as described in claim 1, characterized in that, An image registration model is constructed based on a continuous domain coarse registration network, a spatial transformation network, a U-shaped hierarchical encoder-decoder network with a high-resolution branch, and a U-shaped hierarchical encoder-decoder network with a low-resolution branch. The loss function used for training the image registration model is a weighted sum of image similarity loss, smoothing regularization loss, Jacobian determinant constraint, and anatomical structure-aware constraint. The anatomical structure-aware constraint adopts the introduced Sobel operator and structural similarity.

3. The image registration method combining implicit neural representation and bidirectional multi-scale as described in claim 1, characterized in that, The U-shaped hierarchical encoder-decoder network of the high-resolution branch processes the input image into blocks, obtains high-resolution bottleneck layer semantic features through the Transformer encoder, and then uses the decoder to upsample and obtain a high-resolution displacement field.

4. The image registration method combining implicit neural representation and bidirectional multi-scale as described in claim 3, characterized in that, The Transformer encoder includes multiple hierarchical visual transformers connected in sequence, and the hierarchical visual transformers introduce window-based self-attention mechanisms and moving window self-attention mechanisms.

5. A system combining implicit neural representation and bidirectional multi-scale image registration, characterized in that: include: The high-resolution module is configured to: acquire multiple floating images and reference images at different low-resolution scales; obtain the initial deformation field through a continuous domain coarse registration network based on implicit neural representation; obtain a high-resolution deformed image through a spatial transformation network; combine the high-resolution reference image with a U-shaped hierarchical encoder-decoder network with a high-resolution branch to obtain a high-resolution displacement field; wherein, the continuous domain coarse registration network based on implicit neural representation includes: performing multi-scale gated fusion on the input floating image and reference image to obtain fusion features, and concatenating them with a Fourier-encoded spatial coordinate grid to construct a vector containing position and texture information; and then using a fully connected network based on a sinusoidal activation function to regress the initial deformation field; The low-resolution module is configured to: based on the floating image and initial deformation field at a low-resolution scale, after multi-scale gating fusion processing, obtain the low-resolution displacement field through a U-shaped hierarchical encoder-decoder network of the low-resolution branch; wherein, the bottleneck layer semantic features extracted by the high-resolution branch are injected into the decoding process of the low-resolution branch through convolutional block attention. The registration module is configured to obtain a registration field based on high-resolution and low-resolution displacement fields, thereby aligning the floating image with the reference image. The multi-scale gated fusion includes: for the reference image and the floating image, a dual-branch feature extraction architecture is used to capture local and global information in parallel, and multi-scale feature representations are generated by channel concatenation and convolution aggregation; after obtaining the multi-scale feature representations of the reference image and the floating image, a differential-aware gated fusion mechanism is introduced to obtain the fused features.

6. The image registration system combining implicit neural representation and bidirectional multi-scale as described in claim 5, characterized in that, An image registration model is constructed based on a continuous domain coarse registration network, a spatial transformation network, a U-shaped hierarchical encoder-decoder network with a high-resolution branch, and a U-shaped hierarchical encoder-decoder network with a low-resolution branch. The loss function used for training the image registration model is a weighted sum of image similarity loss, smoothing regularization loss, Jacobian determinant constraint, and anatomical structure-aware constraint. The anatomical structure-aware constraint adopts the introduced Sobel operator and structural similarity.

7. The image registration system combining implicit neural representation and bidirectional multi-scale as described in claim 5, characterized in that, The U-shaped hierarchical encoder-decoder network of the high-resolution branch processes the input image into blocks, obtains high-resolution bottleneck layer semantic features through the Transformer encoder, and then uses the decoder to upsample and obtain a high-resolution displacement field.

8. The image registration system combining implicit neural representation and bidirectional multi-scale as described in claim 7, characterized in that, The Transformer encoder includes multiple hierarchical visual transformers connected in sequence, and the hierarchical visual transformers introduce window-based self-attention mechanisms and moving window self-attention mechanisms.

Images