Self-adaptive medical image sequence registration method and system based on space-time cooperative coding

Abstract

The invention belongs to the technical field of medical image sequence registration, and provides a self-adaptive medical image sequence registration method and system based on space-time collaborative coding, and the method comprises the steps: obtaining 4D image sequence data, defining each element in the 4D image sequence data as a 3D image corresponding to a corresponding moment, registering the 4D image sequence data by using a pre-trained deep neural network model to obtain a registration result; the deep neural network model comprises a feed-forward encoder and a reconstruction decoder which both adopt a multi-stage hierarchical design, and the feed-forward encoder fuses time sequence dependence and space context information in an image sequence; the reconstruction decoder performs step-by-step up-sampling and accurate reconstruction on the features output by the encoder, and constructs a feature compensation path through jump connection between layers so as to fuse shallow spatial information and deep semantic features. According to the invention, the accuracy of registration is improved.

Description

Technical Field

[0001] This invention belongs to the field of medical image sequence registration technology, specifically relating to an adaptive medical image sequence registration method and system based on spatiotemporal co-coding. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Medical image registration plays a crucial role in the field of medical image analysis. By accurately aligning image data acquired from different time sequences or multiple imaging modalities, registration technology enables the effective integration of multi-source information, providing a solid foundation for clinical diagnosis, treatment planning, and subsequent image processing. In the development of registration technology, research on medical image sequences containing temporal dynamic information has gradually become an important direction. Image sequence registration focuses on processing these time-dependent medical images, particularly demonstrating unique advantages in capturing the temporal features of deformation and pathological changes caused by organ movement. It is widely used in numerous scenarios such as dose planning in radiotherapy, precise positioning in surgical navigation, monitoring dynamic changes in patient anatomy, and 3D reconstruction and region segmentation of medical images.

[0004] The field of medical image registration has attracted widespread attention from researchers, leading to the development of diverse registration techniques. Traditional medical image registration methods primarily rely on two strategies: feature matching and grayscale comparison. Feature matching methods first extract representative feature points or regions from the image, then compare the corresponding feature relationships between the moving and stationary images to achieve registration. Grayscale comparison methods directly construct similarity measures in the grayscale or density domain and achieve image alignment through iterative optimization of transformation parameters. However, these traditional methods often face limitations when adapting to complex anatomical structures. This is mainly because the feature extraction and similarity measures they rely on struggle to effectively capture highly nonlinear and diverse tissue deformations, and they are also sensitive to noise and image quality variations, resulting in weak generalization ability and difficulty adapting to the diversity of different patients and multimodal data.

[0005] In recent years, with the rapid development of deep learning technology, neural network-based medical registration methods have become a research hotspot. These methods typically utilize neural networks to directly predict transformation parameters between images. Through a large-scale data-driven training process, the network can automatically extract rich and multi-layered image features and flexibly integrate latent structural information from the images. Thanks to this, deep learning has propelled the rapid development of medical image registration technology, resulting in numerous novel network architectures and optimization strategies.

[0006] The complex and large deformations encountered by organs such as the lungs in clinical practice present a significant challenge in maintaining the stability and accuracy of registration, prompting researchers to propose various innovative strategies. Current mainstream approaches can be broadly categorized into two types: one focuses on multi-scale and hierarchical registration, employing a progressively finer resolution process to first address large-scale coarse deformations and then gradually refine local details, thus mitigating the optimization difficulties and convergence instability caused by large deformations. The other approach focuses on the temporal information of medical image sequences (such as 4D CT scans), aiming to capture the dynamic deformation characteristics of organs over time. This is achieved by introducing temporal modeling modules (such as long short-term memory (LSTM) and temporal convolution) to enhance consistency and continuity in the temporal dimension. While these studies introduce diverse innovative concepts, most still follow the approach of breaking down 4D registration into several independent 3D sub-tasks. The comprehensive mining and utilization of temporal information needs further development, necessitating the construction of more systematic temporal modeling schemes to address complex and large-deformation scenarios. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes an adaptive medical image sequence registration method and system based on spatiotemporal co-coding. By achieving the co-fusion of spatial and temporal features at different network layers, this invention effectively avoids the problems of feature attenuation at each layer and insufficient spatiotemporal interaction expression caused by the separation of spatiotemporal information in previous methods. This improves the model's ability to express and match complex dynamic deformations, thereby increasing registration accuracy and efficiency.

[0008] According to some embodiments, the present invention adopts the following technical solution: An adaptive medical image sequence registration method based on spatiotemporal co-coding includes the following steps: Acquire 4D image sequence data, and define each element in the 4D image sequence data as the corresponding 3D image at the corresponding time point; A pre-trained deep neural network model is used to register 4D image sequence data to obtain the registration results; The deep neural network model includes a feedforward encoder and a reconstruction decoder. The feedforward encoder adopts a multi-level hierarchical design. Each level includes a Patch Embedding module and a CM-BiLSTM unit. The Patch Embedding module uses stride convolution to implicitly divide the image into blocks, generate feature representations, and simultaneously perform channel expansion and spatial downsampling. The generated features are fed into the corresponding level's CM-BiLSTM unit for modeling. Each level has at least two CM-BiLSTM units stacked together to fuse temporal dependencies and spatial context information in the image sequence. The reconstruction decoder adopts a multi-level hierarchical design. Each level includes a transposed convolutional module and a CNN unit, which are used to progressively upsample and accurately reconstruct the features output by the encoder. Furthermore, feature compensation pathways are constructed between layers through skip connections to fuse shallow spatial information and deep semantic features.

[0009] As an alternative implementation, the feature channel dimensions of the output of each layer of the feedforward encoder increase sequentially, namely {C, 2C, 4C, 8C}, where C is an integer value; Alternatively, the number of CM-BiLSTM units stacked at each level of the reconstruction decoder is {2, 2, 4, 2}, respectively.

[0010] As an alternative implementation, for each time step t, the CM-BiLSTM unit first concatenates the current input feature Xt with the previous hidden state Ht-1, and then feeds it into the CM block for multi-frequency feature extraction. The CM block contains two paths. The high-frequency path adopts a CNN series structure, focusing on high-frequency features including edges and local textures. The low-frequency path first performs average pooling to reduce computation, and then uses the Vision Mamba module, which has linear time complexity and strong temporal memory, to model long-range dependencies, focusing on low-frequency components including overall shape and slow changes. Finally, it is upsampled back to the original scale. The outputs of the two paths are fused to obtain an enhanced representation, which is then fed into the gating unit of the bidirectional LSTM and combined with the cell state Ct-1 of the previous time step to generate a new hidden state Ht and cell state Ct.

[0011] As a further defined implementation, the CM-BiLSTM unit adopts a bidirectional strategy, in which the two directions share the same CM block at each time t. The forward and backward hidden states obtained by bidirectional updates are concatenated, and a unified representation at that time is obtained through a linear mapping layer to drive subsequent registration.

[0012] As a further defined implementation, during the fusion of the two path outputs, a monotonically decreasing prior is set in the depth dimension, and sample adaptive soft routing is performed. In the s-th stage, the high-frequency and low-frequency paths output features Yh and Yl in parallel. Based on global statistical calculation, a scalar gate αs∈[0, 1] representing the contribution ratio of the high-frequency path is calculated to obtain the fused output Y=αsYh+(1-αs)Yl. The monotonically decreasing prior is initialized to reflect the hierarchical preference of shallow layers emphasizing high frequencies and deep layers emphasizing low frequencies. After training begins, the scalar gate is adaptively adjusted around the prior, allowing data-driven deviations while maintaining the hierarchical trend, thereby achieving progressive modeling from local details to the global structure.

[0013] As an alternative implementation, during the training process of the deep neural network model, a multi-constraint loss function is used as the loss function for training. The multi-constraint loss function includes image similarity loss and deformation field regularization loss. The image similarity loss downsamples the input image at several scale factors and calculates the structural similarity index value at each scale. The structural similarity index values ​​at multiple scales are weighted and summed to obtain the overall multi-scale loss. The deformation field regularization loss includes smoothness loss and anti-folding loss. The smoothness loss is represented by the sum of squares of the deformation field gradient, and the anti-folding loss is achieved by restricting the Jacobian determinant of the deformation field.

[0014] As an alternative implementation, during the registration of 4D image sequence data using a pre-trained deep neural network model, a local temporal registration strategy is adopted, which treats adjacent image pairs in the sequence as interrelated local time segments in order to capture minute deformations and structural displacements within short time intervals.

[0015] As an alternative implementation, during the registration of 4D image sequence data using a pre-trained deep neural network model, a cross-temporal registration strategy is adopted to select a certain image in the sequence as a reference image, and the reference image is directly registered with images at multiple other time points to construct a global deformation model across time periods.

[0016] As an alternative implementation method, in the process of registering 4D image sequence data using a pre-trained deep neural network model, a full temporal registration strategy is adopted. From the perspective of the entire time axis, the sequence images are globally registered and optimized in a collaborative manner. First, a local temporal registration strategy is used to obtain the local deformation field between adjacent time frames. Then, a cross-temporal registration strategy is combined to construct a global deformation model that spans a longer time interval. The two types of deformation fields are integrated and unified through composite operations.

[0017] An adaptive medical image sequence registration system based on spatiotemporal co-coding includes: The data acquisition module is configured to acquire 4D image sequence data, and each element in the 4D image sequence data is defined as the 3D image corresponding to the corresponding time. The registration module is configured to register 4D image sequence data using a pre-trained deep neural network model to obtain the registration result; The deep neural network model includes a feedforward encoder and a reconstruction decoder. The feedforward encoder adopts a multi-level hierarchical design. Each level includes a Patch Embedding module and a CM-BiLSTM unit. The Patch Embedding module uses stride convolution to implicitly divide the image into blocks, generate feature representations, and simultaneously perform channel expansion and spatial downsampling. The generated features are fed into the corresponding level's CM-BiLSTM unit for modeling. Each level has at least two CM-BiLSTM units stacked together to fuse temporal dependencies and spatial context information in the image sequence. The reconstruction decoder adopts a multi-level hierarchical design. Each level includes a transposed convolutional module and a CNN unit, which are used to progressively upsample and accurately reconstruct the features output by the encoder. Furthermore, feature compensation pathways are constructed between layers through skip connections to fuse shallow spatial information and deep semantic features.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention addresses the problems of feature decay at different layers and insufficient spatiotemporal interaction representation caused by the separation of multi-scale and temporal information extraction in existing methods. It proposes an innovative multi-frequency-temporal feature joint extraction unit, CNN-Vision Mamba with Bidirectional LSTM (CM-BiLSTM). This unit deeply integrates the parallel multi-scale feature extraction paradigm with long-range temporal modeling mechanisms, achieving collaborative spatiotemporal representation at each layer, thereby improving the overall joint spatiotemporal representation capability and effectively suppressing the aforementioned problems.

[0019] This invention designs an Adaptive Frequency Routing (AFR) method that uses data-driven gating to perform soft routing and dynamic fusion between high-frequency and low-frequency features, enabling frequency focus to adapt to the input.

[0020] This invention proposes an Adaptive Weighted Multi-Scale Structural Similarity Index Measure (AM-SSIM) as a similarity loss, which enables the structural similarity assessment at different scales to be dynamically optimized as the training process progresses.

[0021] This invention proposes and combines multiple sequence registration strategies to fully capture and utilize dynamic deformation information in 4D image sequences, thereby significantly improving the model's adaptability to complex spatiotemporal deformations.

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0024] Figure 1 This is a flowchart illustrating an image sequence registration method according to one embodiment; Figure 2 This is a schematic diagram of the registration network architecture of CML-UNet in one embodiment; Figure 3 This is a schematic diagram of a CM-BiLSTM unit structure according to one embodiment; Figure 4 This is a schematic diagram of an AFR integrated in a CML-UNet encoder according to one embodiment; Figure 5 This is a schematic diagram of the AM-SSIM calculation process in one embodiment; Figure 6 A schematic diagram of an LTR embodiment; Figure 7 A schematic diagram of a CTR embodiment; Figure 8 A schematic diagram of a GTR embodiment; Figure 9 This is an example of a DIR-Lab database image and corresponding points, where (a) is T00; (b) is T50. Figure 10 This is a scatter plot of registration results under three sequence registration strategies in one embodiment, where (a) is case 8 of the DIR-Lab database; (b) is case 2 of the POPI database; Figure 11 This is a visualization comparison of folded regions in DIR-Lab database case 8 under three sequence registration strategies, where (a) represents LTR; (b) represents CTR; and (c) represents GTR. Figure 12 This is a heatmap showing the differences in registration results in an ablation study of one embodiment. Figure 13 This is a Fourier spectrum of different paths output in one embodiment, where (a) is the high-frequency path, (b) is the low-frequency path, and (c) is the fused path; Figure 14 This is a comparison chart of the results of different similarity loss experiments in one embodiment, where (a) is the DIR-Lab database; (b) is the POPI database; Figure 15This is a loss variation during the optimization process of one embodiment, where (a) is case 8 of the DIR-Lab database; and (b) is case 2 of the POPI database. Figure 16 This is a schematic diagram of deformation field visualization of one embodiment, where (a) is case 8 of the DIR-Lab database; and (b) is case 2 of the POPI database. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0027] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0028] Where there is no conflict, the embodiments and features described in this application may be combined with each other.

[0029] Example 1 An adaptive medical image sequence registration method based on spatiotemporal co-coding, such as Figure 1 As shown, it includes the following steps: First, the input 4D image sequence data is defined as XN={x1, x2, …, xn}, where each xi is the 3D image corresponding to a specific time. When N=2, this problem is the common pairwise image registration; while when N>2, the problem is image sequence registration. The deep neural network module CML-UNet in this method consists of a CM-BiLSTM encoder and a CNN decoder. The core objective of image sequence registration is to find a set of deformation field sequences DN={d1, d2, …, dn}, where each deformation field di contains displacement information in the x, y, and z directions. Finally, the original image sequence XN is warped through a Spatial Transformer Network (STN) to obtain the deformed image sequence XW.

[0030] The overall registration network architecture of CML-UNet is shown in the diagram below. Figure 2As shown, a symmetrical encoder-decoder topology is adopted. The input image sequence is denoted as XN, where N represents the sequence length. The tensor dimension of each frame is D×H×W×1, so the tensor dimension of the entire input sequence is N×D×H×W×1. Here, D, H, and W represent the dimensions of the image in three dimensions (depth, height, and width), respectively.

[0031] The feedforward encoder consists of a Patch Embedding module and CM-BiLSTM units, employing a four-level hierarchical design. First, the Patch Embedding module implicitly divides the image into blocks using stride convolutions in each layer, generating compact feature representations while simultaneously performing channel expansion and spatial downsampling. The feature channel dimensions output by each layer are {C, 2C, 4C, 8C}, where C is 32. Subsequently, the features are fed into each level of the CM-BiLSTM units for modeling. Multiple CM-BiLSTM units are stacked in each level, with stacking numbers of {2, 2, 4, 2}. The CM-BiLSTM units fully integrate temporal dependencies and spatial contextual information from the image sequence while maintaining the feature channels unchanged. Through the synergistic effect of Patch Embedding and CM-BiLSTM units, the encoder as a whole achieves a hierarchical representation of multi-scale semantic features, providing a rich feature foundation for subsequent image reconstruction.

[0032] The reconstruction decoder employs a combination of CNN and ConvTranspose to progressively upsample and accurately reconstruct the features output by the encoder. Simultaneously, skip connections between corresponding layers construct feature compensation pathways, which helps fuse shallow spatial information with deep semantic features, improving reconstruction accuracy. After processing by CML-UNet, the input image sequence XN is mapped to a deformation field sequence DN, with a tensor dimension of N×D×H×W×3. Each deformation field has a tensor dimension of D×H×W×3, and the last dimension represents the displacement components in three directions (x, y, z), used for subsequent image deformation operations.

[0033] CM-BiLSTM, as the core unit in the registration network for joint extraction of multi-frequency temporal features, has the following structure: Figure 3As shown. For each time step t, the current input feature Xt is first concatenated with the hidden state Ht-1 from the previous time step, and then fed into a CNN-Vision Mamba (CM) block for multi-frequency feature extraction. The CM block contains two paths: the high-frequency path uses CNN series structures such as depthwise convolution and dynamic convolution to focus on high-frequency features such as edges and local textures; the low-frequency path first performs average pooling to reduce computation, then uses Vision Mamba (VMamba), which has linear time complexity and strong temporal memory, to model long-range dependencies, focusing on low-frequency components such as overall shape and slow changes, and finally upsamples back to the original scale. The outputs of the two paths are fused to obtain an enhanced representation, which is then fed into the gated unit of a bidirectional LSTM (BiLSTM) and combined with the cell state Ct-1 from the previous time step to generate new Ht and Ct.

[0034] Temporal modeling employs a strict bidirectional strategy, with both directions sharing the same CM block at each time t. Subsequently, the forward and backward hidden states obtained from the bidirectional updates are concatenated, and a unified representation for that time moment is obtained through a linear mapping layer, driving subsequent registration. This overall design enhances multi-scale representation and temporal dependency modeling while maintaining low computational cost, enabling joint encoding of information from preceding and following phases at all levels, adapting to local minute movements and slow global deformations in medical images.

[0035] Figure 3 The CM-BiLSTM cell in (a) is marked with a red box, indicating the CM block. This corresponds to the CM module shown in (b), which consists of a high-frequency path (yellow) and a low-frequency path (green). The VMamba block is marked with a red box (corresponding to...). Figure 3 (c)).

[0036] As observed in visual neuroscience, the visual cortex exhibits a hierarchical preference for frequency components: earlier regions are more sensitive to high-frequency details, while later regions better integrate low-frequency information. Inspired by this, this embodiment proposes an adaptive frequency routing (AFR) mechanism, the schematic of which is shown below. Figure 4As shown, a monotonically decreasing prior is set in the depth dimension, and sample adaptive soft routing is performed. In the s-th stage, high-frequency and low-frequency paths output features Yh and Yl in parallel. Subsequently, based on global statistical calculation of scalar gating αs∈[0, 1] (representing the contribution ratio of the high-frequency path), the fused output Y=αsYh+(1-αs)Yl is obtained. It is initialized with a monotonically decreasing prior {α1, α2, α3, α4}={7 / 8, 5 / 8, 3 / 8, 1 / 8} to reflect the hierarchical preference of "shallow layers focusing on high frequencies and deep layers emphasizing low frequencies". After training begins, αs is mildly adaptively adjusted around this prior, allowing data-driven deviations while maintaining the hierarchical trend, thereby achieving progressive modeling from local details to global structure. Figure 4 In the image, high-frequency components in each layer are represented in yellow, and low-frequency components in green. It is recommended to view the image in color for the best visual experience.

[0037] To achieve high-precision spatial alignment in medical image sequence registration, this embodiment constructs a multi-constraint loss function system, specifically including image similarity loss L. sim And deformation field regularization loss Lreg. Their specific descriptions are as follows.

[0038] The total loss Ltotal is defined as the sum of the image similarity loss Lsim and the deformation field regularization loss Lreg, aiming to constrain the smoothness and rationality of the deformation field while ensuring registration accuracy through joint optimization. Lsim uses the proposed Adaptive Weighted Multi-Scale Structural Similarity Index Measure (AM-SSIM), denoted as LAM-SSIM. Lreg includes both the loss Lsmo and the anti-folding loss Lant, balancing the smoothness of the deformation field during registration and preventing irreversible folding. Specifically, the smoothness loss Lsmo is represented by the sum of squares of the deformation field gradients, while the anti-folding loss Lant is implemented by constraining the Jacobian determinant of the deformation field. Ltotal is defined as follows:

[0039] Here, α, β, and γ are the weighting coefficients of the three types of loss terms, used to balance the contribution of different losses to the overall loss.

[0040] In medical image sequence registration tasks, accurately measuring the similarity between a fixed image sequence XN and a deformed image sequence XW is crucial for improving registration performance. The Structural Similarity Index Measure (SSIM), as an effective image similarity metric, comprehensively considers information such as image contrast and structure. Its definition is as follows:

[0041] in, and This represents the mean. and It represents the standard deviation. This represents the covariance. C1 and C2 are constants used for stability.

[0042] However, single-scale SSIM often struggles to comprehensively capture multi-scale structural information when processing medical images with large deformation backgrounds, and it also fails to adaptively balance the importance of different scales. Therefore, this embodiment proposes an AM-SSIM, which aims to dynamically optimize the contribution of features at different scales by combining a multi-scale strategy with an adaptive weighting mechanism, in order to better adapt to the multi-scale structural information in medical images. The overall computation process of AM-SSIM is as follows: Figure 5 As shown.

[0043] In practical calculations, to capture multi-scale features in the image, the input image is first downsampled at several scale factors {s1, s2, s3}, and the SSIM value is calculated at each scale. The resulting multi-scale SSIM values ​​are then weighted and summed in subsequent steps to obtain the overall multi-scale loss. Furthermore, to improve the adaptability to complex medical images, AM-SSIM introduces an adaptive weighting mechanism. In this embodiment, the scale weights {w1, w2, w3} are set as learnable parameters and dynamically optimized through backpropagation of the global registration loss. In other words, the model automatically adjusts the contribution of each scale loss to the final loss during training based on actual registration requirements. Therefore, the overall loss of AM-SSIM can be expressed as a weighted sum of the SSIM values ​​at each scale, i.e.

[0044] Among them, w i Let N be the learnable weights corresponding to the i-th scale, and N be the total number of scales.

[0045] Finally, this embodiment uses the proposed AM-SSIM as the similarity loss L. sim Its definition is as follows:

[0046] Where α is the weighting adjustment coefficient.

[0047] In traditional image registration tasks, the focus is on finding the optimal deformation field between two images to achieve spatial alignment. However, when the task is extended to continuous medical image sequences, simple image-pair registration strategies are no longer sufficient. At this point, not only spatial deformation relationships need to be considered, but also the dynamic changes over time must be incorporated. This makes the definition space of the registration problem more complex and also provides the possibility for more flexible and multi-level image sequence registration strategies. To address the complexity of the aforementioned sequence registration task, this embodiment proposes three registration strategies for medical image sequences: Local-Temporal Registration Strategy (LTR), Cross-Temporal Registration Strategy (CTR), and Global-Temporal Registration Strategy (GTR). These strategies, from different temporal relational perspectives, more flexibly capture and utilize the temporal dynamic features in medical image sequences.

[0048] LTR such Figure 6 As shown, this strategy treats adjacent image pairs in a sequence (such as x1 and x2, x2 and x3, etc.) as interconnected local time segments to capture minute deformations and structural displacements within short time intervals. By fully utilizing the high temporal correlation between adjacent frames, this strategy optimizes the registration accuracy of image sequences at a local scale.

[0049] CTR such as Figure 7 As shown, focusing on a larger time span, a specific image in the sequence (e.g., x1) is selected as the reference image, and this reference image is directly registered with images at multiple other time points (e.g., x2, x3, etc.) to construct a global deformation model spanning multiple time periods. Compared to LTR, this strategy is more suitable for analyzing long-term dynamic deformation trends in the sequence and can effectively reduce the cumulative error in the registration process.

[0050] The GTR strategy takes a holistic approach, performing global registration and optimization of image sequences in a collaborative manner. For example... Figure 8As shown, MI represents operations on the image, and MDF represents operations on the deformation field. GTR first uses LTR to obtain the local deformation field between adjacent time frames, and then combines it with CTR to construct a global deformation model spanning a longer time interval. Finally, these two types of deformation fields are integrated into the same framework through composite operations. The specific deformation field composite operations are shown below. Through the above comprehensive processing, GTR effectively alleviates the sensitivity of the registration results to deviations at specific times or in local areas, ensuring that the registration effect of the sequence images achieves a better balance at the overall level.

[0051]

[0052] Where dxy represents the deformation field from the x-th image to the y-th image. This indicates that spatial coordinates are mapped and transformed using STN. Compose() represents the composite operation of the deformation field.

[0053] This embodiment validates the proposed method on two publicly available 4D CT databases. The first database is DIR-Lab, which contains respiratory time-series 4D CT data of 10 patients, labeled as case1 to case10. Each case consists of 10 phases of 3D CT images. At the T00 and T50 phases, 300 pairs of corresponding points, annotated by a physician, are provided as labels. Figure 9 Example images and their corresponding points for the T00 and T50 phases from case 1 are shown. The second database is POPI, which covers respiratory time-series 4D CT data from 6 patients, labeled case 1 to case 6, each containing 3D CT images of 10 phases. The first 3 cases provide 100 sets of corresponding points for all phases, while the latter 3 cases provide approximately 100 sets of corresponding points only for the T00 and T50 phases.

[0054] In the preprocessing stage, spatial cropping based on corresponding point labels is first implemented to focus the region of interest in the lungs and reduce computational cost. Subsequently, a bi-percentile truncation method using the 1st and 99th percentiles as truncation thresholds is used for intensity normalization to suppress anomalous voxel values ​​caused by differences in imaging conditions. Finally, linear normalization is performed on the images to standardize the global intensity to the dimensionless range [0,1] to ensure intensity consistency across different images.

[0055] The medical image registration network used in the experiment was implemented using the PyTorch framework. During training, the Adam algorithm was employed to optimize the network parameters, with an initial learning rate set to 1e-3. The model introduced Automatic Mixed Precision (AMP), utilizing NVIDIA Apex tools to achieve mixed-precision training of FP32 and FP16, thereby improving training efficiency. Both training and inference were performed on a single NVIDIA A100 GPU node. To ensure the reliability of the experimental results, all reported performance metrics are the average of five independent experiments.

[0056] Training employs a sequential iterative method based on one-time learning. To prevent getting trapped in local optima and reduce computational burden, an early stopping mechanism is introduced. Specifically, by calculating the standard deviation σ of the similarity loss over the most recent Nstop iterations, training is terminated early when it falls below a set threshold σstop and reaches the minimum number of iterations Niter. Nstop, σstop, and Niter are set to 100, 0.0007, and 200, respectively.

[0057] In the loss function configuration, the scaling factors {s1, s2, s3} of AM-SSIM are set to {1.0, 0.5, 0.25} by default, and the corresponding initial learnable weight coefficients {w1, w2, w3} are set to {0.7, 0.2, 0.1}, respectively. Furthermore, in the total loss Ltotal, the weight adjustment coefficients α, β, and γ are set to 1, 0.001, and 0.01, respectively.

[0058] Two evaluation metrics are used to measure the effectiveness of the proposed registration method: Target Registration Error (TRE) and the proportion of negative Jacobian determinants. Specifically, TRE measures the alignment of images by calculating the coordinate differences of corresponding points in the two registered images and taking their average Euclidean distance. This metric is reported as mean ± standard deviation. In ideal registration, TRE equals 0; therefore, a smaller value indicates better registration. On the other hand, the proportion of negative Jacobian determinants measures the smoothness of the deformation field. When the Jacobian determinant at a voxel is less than 0, it means that there is local folding in that region. Therefore, a lower proportion indicates a smoother deformation field.

[0059] Based on the CML-UNet method proposed in this embodiment, three sequence registration strategies—LTR, CTR, and GTR—were employed, and their performance on two 4D CT databases was evaluated. The results are shown in Table 1. First, the accuracy of the three sequence registration strategies was analyzed based on the TRE index. Overall, all three registration strategies achieved significant accuracy improvements compared to before registration. CTR performed best; in the DIR-Lab database, the average TRE for 10 cases decreased significantly from 8.46±5.48 mm before registration to 1.09±0.73 mm. For the POPI database, the average TRE for 6 cases decreased significantly from 8.12±4.77 mm before registration to 0.81±0.56 mm. This result can be attributed to the direct and strong constraint between the reference image and the target image in CTR, which provides a more reliable registration guarantee. In contrast, LTR only applies indirect constraints through adjacent images, lacking a direct connection between the reference image and the target image, resulting in the worst registration results. GTR, on the other hand, partially introduces the strong constraints of CTR on the basis of LTR. Therefore, it performs better than LTR alone in long-range image sequence registration tasks, but is slightly inferior to CTR alone.

[0060] Table 1 compares the performance of the algorithm across three sequence registration strategies. "↓" indicates a lower value indicates better performance, and bold indicates the optimal result.

[0061] In the DIR-Lab database, case 8 exhibited an initial TRE of 14.99±9.00 mm before registration. In the POPI database, case 2 also showed an initial TRE of 14.04±7.20 mm before registration. These are considered typical examples of large deformations in their respective databases and therefore warrant close attention. Figure 10 The following are scatter plots of the registration results under the three registration strategies. Observing the scatter plots, it can be seen that CTR still achieved the best registration accuracy in these two cases, consistent with the overall trend described above.

[0062] To assess the smoothness of the registration results, we evaluated the folding phenomenon by analyzing the proportion of negative values ​​in the Jacobian determinant. LTR had the smallest proportion of folded regions, at only 0.01% and 0.00% in the two databases, respectively; GTR was next, at 0.39% and 0.31%, respectively; and CTR had the highest proportion of folded regions, reaching 0.61% and 0.45%, respectively. Figure 11This section shows examples of folded region distribution in case 8 of the DIR-Lab database under three sequence registration strategies. The red areas represent folded regions in the deformed images. Folded regions are almost invisible in LTR, the folding range increases slightly in GTR, and CTR produces a larger area of ​​folding. It should be noted that, to most clearly demonstrate the folding phenomenon, a lung slice near the diaphragm region with significant motion and obvious folding was specifically chosen as an example. In the main lung region, folded regions are less abundant in all three strategies.

[0063] The aforementioned folding differences primarily stem from the characteristics of the sequence registration strategies themselves. LTR employs progressive registration, limiting image differences to a small range through stepwise inference between adjacent frames. This strategy maintains high smoothness of the deformation field when large deformations are decomposed into multiple smaller deformations. This becomes significant when nonlinear organs with large deformations, such as the lungs, have stringent smoothness requirements. CTR, on the other hand, requires inference across images with longer time dimensions, directly inferring from images with large deformations, thus increasing the probability of folding. This places higher demands on the network's fitting ability. GTR combines the advantages of both strategies within the same framework, balancing the processing needs of both large and small deformations, thus achieving a smoothness performance between the two.

[0064] Considering both registration accuracy and deformation smoothness, although CTR has the highest proportion of folded regions, this value is still within a reasonable range. Especially considering its significant improvement in registration accuracy, this trade-off is acceptable. Therefore, CTR is considered the superior sequence registration strategy in this embodiment, and subsequent experiments were all conducted based on CTR.

[0065] This embodiment conducted a series of detailed ablation experiments on an experimental database to comprehensively evaluate the method's performance and analyze the contribution and robustness of each module and its configuration to the model's effectiveness. The quantitative results of the ablation experiments are shown in Table 2. The results show that the most significant performance decrease occurs when the CM-BiLSTM unit is removed and replaced with a regular CNN, indicating that this unit plays a crucial role in capturing temporal dependencies while jointly extracting high-frequency and low-frequency features. Even though the BiLSTM structure is preserved after removing the CM module, replacing it with a regular CNN forces the model to focus only on local detail modeling, neglecting global structural information, thus leading to performance degradation. Removing only the BiLSTM module prevents the model from effectively capturing temporal relationships between image sequences, also affecting registration results. Furthermore, removing the AFR fixes the ratio of high-frequency and low-frequency feature channels in each coding level of the model, limiting the flexibility and adaptability of feature extraction. After removing AM-SSIM, we replaced it with Normalized Cross-Correlation (NCC), a commonly used metric in medical image registration. However, this metric cannot effectively assess structural similarity across multiple scales, and the registration results are also affected. Overall, all ablation models exhibited varying degrees of performance degradation. In contrast, the complete model consistently performed best on quantitative metrics, validating the necessity of each component and the effectiveness of the ablation experiments.

[0066] Table 2. Quantitative results under different ablation configurations (w / o indicates removal of the corresponding component). Bold indicates the optimal result.

[0067] In addition, Figure 12 The paper presents heatmaps showing the differences in registration quality under different ablation configurations for Case 8, which visually reflects the registration quality of each configuration. Red indicates positive error (over-displacement), and blue indicates negative error (under-displacement); lighter colors indicate more ideal registration results. Visually, the complete model reduces residual error better than other ablation models, which corroborates the quantitative results and further demonstrates that the collaborative work of each component is crucial for achieving high-quality registration. Figure 12 In the diagram, positive errors are marked in red, and negative errors are marked in blue. The lighter the color, the more ideal the registration effect.

[0068] To further verify the effectiveness of CM-BiLSTM in extracting features at different frequency components, we compared the outputs of the high-frequency path (Yh1+Yh2) using Fourier spectrum analysis. Figure 3 b), the output of the low-frequency path (Yl, Figure 3 b) and the output of the final fusion path (Y, Figure 3b). For visualization purposes, we selected case 8 from the DIR-Lab database, focusing on the third level of the encoding path that continuously passes through CM-BiLSTM the most times. The corresponding spectrogram is shown below. Figure 13 As shown in the figure, the high-frequency path has a significant advantage in capturing high-frequency details, while the low-frequency path is better at acquiring low-frequency global information. After fusion, the output achieves complementarity between high and low-frequency information, thereby improving the overall feature representation effect.

[0069] To evaluate the effectiveness of AFR, we set three four-stage frequency priors for the scalar routing gate αs, denoted as {α1, α2, α3, α4}: K1={7 / 8, 5 / 8, 3 / 8, 1 / 8}, K2={4 / 8, 4 / 8, 4 / 8, 4 / 8}, and K3={1 / 8, 3 / 8, 5 / 8, 7 / 8}. During training, αs is moderately adaptively adjusted around the selected priors, allowing data-driven deviations while preserving the hierarchical trend in depth. K1 reflects a bias that emphasizes high frequencies in the shallow stages and low frequencies in the deep stages. As shown in Table 3, K1 achieves the best results, indicating that this frequency prior provides a more informative starting point for the task.

[0070] Table 3. Comparison of AFR results under different stage-by-stage αs frequencies prior. The best results are marked in bold.

[0071]

[0072] To address the registration requirements of medical images with large deformations, this embodiment selects several commonly used similarity loss functions in the field of medical imaging for comparative experiments. To ensure fairness, a controlled variable method is used to integrate different similarity losses into the same sequence registration framework. The relevant results are shown in Table 4. The results show that our proposed AM-SSIM achieves the best registration effect, achieving average TREs of 1.09±0.73mm and 0.81±0.56mm on the two databases, respectively. Figure 14 The results provide a visual comparison of quantitative results from two databases. It can be seen that AM-SSIM shows a significant advantage for cases with large deformation characteristics. Other loss functions often exhibit significant fitting difficulties and are accompanied by large numerical fluctuations. Notably, in some cases with small deformations, Local CrossCorrelation (LCC) achieves the best results, mainly due to its high sensitivity to subtle local structural differences. However, its performance decreases as the initial degree of deformation increases.

[0073] Table 4. Quantitative results of experiments with different similarity losses. Bold text indicates optimal results.

[0074] Further analysis of the optimized performance of AM-SSIM on two representative cases, and the changes in loss during the training process are as follows: Figure 15 As shown in the figure, the loss values ​​for the three scales {s1, s2, s3} = {1.0, 0.5, 0.25}, and the overall loss after adaptive weighting, are illustrated. With each training iteration, the loss at all scales shows a stable decreasing trend, indicating that the model effectively improves image similarity at different scales. The overall decrease in the total weighted loss, shown by the green curve, verifies the effectiveness of the adaptive weighting mechanism in fusing multi-scale features. This result demonstrates that AM-SSIM can dynamically coordinate the optimization process of features at each scale, enhancing the fidelity of local details while ensuring global structural consistency, thereby significantly improving the overall performance and robustness of medical image registration.

[0075] Tables 5 and 6 present the comparison results of the proposed method with state-of-the-art methods on two publicly available 4D CT databases (DIR-Lab and POPI). It can be seen that even when faced with cases exhibiting the most severe initial deformation—cases 6, 7, and 8 in DIR-Lab, and case 2 in the POPI database—the proposed method still achieves highly competitive registration results, fully demonstrating its robustness and accuracy advantages in scenarios with large-scale nonlinear deformation.

[0076] This performance improvement is attributed to several innovations in model architecture, loss function, and registration strategy. First, by introducing the multi-frequency-temporal feature extraction unit CM-BiLSTM, the model can simultaneously capture high-frequency local details and low-frequency global trends, and accurately model dynamic deformations using temporal gating. Second, AFR enables dynamic balancing of high- and low-frequency features at different levels, thereby enhancing adaptability to deformations at different scales. Furthermore, AM-SSIM specifically strengthens alignment at key scales during training. Finally, considering the temporal dynamics involved in medical image sequence registration tasks, this embodiment further designs three registration strategies with different temporal correlation modeling capabilities: Local Temporal Registration (LTR), Cross-Temporal Registration (CTR), and Global Temporal Registration (GTR), to flexibly address diverse temporal deformation features. The synergistic effect of these innovations enables the proposed method to accurately recover large-scale displacements and preserve minute structural details in complex clinical respiratory motion contexts, thus demonstrating excellent registration results in various cases.

[0077] Table 5. Comparison of registration performance between the proposed method and the state-of-the-art (SOTA) method on the DIR-Lab database. (Bold indicates the optimal result.)

[0078] Table 6. Comparison of registration performance between the proposed method and the state-of-the-art (SOTA) method on the POPI database. (Bold indicates the optimal result.)

[0079] Figure 16 The deformation field visualization results on the DIR-Lab database (case 8) and the POPI database (case 2) are presented. Displacement vectors (red-purple bars representing different displacement magnitudes) are overlaid on the central slice, with the arrows pointing in the actual deformation direction. It can be seen that the deformation field distribution in case 8 is relatively uniform and continuous, mainly concentrated in the lung lobe edge region. Case 2, on the other hand, exhibits a larger and more complex vector distribution, reflecting more significant chest wall movement. Overall, both sets of results maintain the smoothness and reasonableness of the deformation field, without obvious folding or discontinuity, verifying the robustness of the proposed method when processing image sequences with different respiratory amplitudes.

[0080] In summary, 4D CT sequence registration is crucial for clinical evaluation and radiotherapy planning. Existing methods are limited by the large-scale non-rigid deformations caused by respiratory motion, cross-scale temporal variations, and artifacts. They still have shortcomings in long-term temporal dependency modeling, cross-scale adaptation, and topology preservation, making it difficult to simultaneously guarantee accuracy and reversibility in scenarios with strong deformation. To address these gaps, this embodiment proposes a general approach of achieving spatiotemporal collaborative representation at each layer of the network. Specifically, this embodiment proposes CML-UNet, which performs spatiotemporal collaborative representation at each layer of the network to address the challenge of large-scale and complex spatiotemporal deformations in 4D CT sequences. Furthermore, CM-BiLSTM, AFR, and AM-SSIM are used to jointly improve registration robustness and accuracy. In addition, LTR, CTR, and GTR sequence registration strategies are combined to fully exploit the dynamic features of the sequences. To ensure consistent interpretation of results, registration effectiveness is evaluated using the proportion of negative values ​​in TRE and Jacobi determinant, which respectively measure alignment accuracy and deformation reversibility and smoothness.

[0081] Tables 5 and 6 compare the proposed method with state-of-the-art methods on two publicly available 4D CT datasets, DIR-Lab and POPI, demonstrating its continued competitiveness. A comparison of two mainstream methods for organs with large deformations, such as the lungs, reveals two approaches: First, multi-scale and hierarchical registration, represented by UHMR-Net, MultiResUNet, pyramid CRNet, and HPRN, which mitigate optimization instability under large deformations through coarse-to-fine alignment. Second, sequence-oriented temporal modeling, typically using C-LSTMNet, Lung-CRNet, and periodic one-time learning frameworks, relying on LSTM or GRU and periodic modeling to enhance temporal consistency. Both approaches have their advantages; however, many studies still decompose the 4D registration task into several independent 3D sub-tasks, resulting in insufficient spatiotemporal coupling. We model the 4D CT registration task as a whole as a sequence registration problem and uniformly integrate multi-scale and temporal information across all network layers. Therefore, we achieved optimal or near-optimal target registration errors in most cases, while simultaneously satisfying reversibility and smoothness constraints. Even in extreme cases with significantly increased initial displacement and deformation gradients, the method can output a stable and physically feasible registration field. These results demonstrate that the method exhibits good adaptability and robustness to large-amplitude, strongly nonlinear deformations, and has the potential for application in complex clinical scenarios.

[0082] Overall, the results show a clear trade-off between "strong constraints for accuracy and weak constraints for deformation reversibility." CTR, with direct temporal constraints, consistently delivers the lowest TRE on both databases, but also results in a higher folding rate. LTR, which progressively advances between adjacent frames, has very few folds, but its accuracy is limited due to a lack of strong coupling with the baseline image. GTR falls somewhere in between. This pattern remains consistent even in extreme cases of large deformation (such as DIR-Lab case 8 and POPI case 2), indicating that the strength of temporal coupling is the dominant factor influencing the performance-topology trade-off. Therefore, choosing CTR as the default strategy in this paper is a reasonable choice. At the mechanistic level, ablation experiments show that the performance improvement is not driven by a single component, but by the combined effect of intra-layer spatiotemporal coordination and cross-layer frequency preference. Removing CM-BiLSTM results in the largest degradation, indicating that intra-layer coupling of "parallel multi-scale + long-range temporal" is key to capturing strong nonlinear respiratory motion. Retaining only BiLSTM or only CM weakens the joint characterization of global-local and temporal dependencies. Fixing the ratio of high-frequency and low-frequency channels in each layer also limits the cross-scale adaptability, further confirming the contribution of AFR to the stable optimization path.

[0083] From the perspective of similarity metrics, AM-SSIM exhibits better convergence and robustness under large deformations compared to traditional single-scale or local metrics (such as NCC, LCC, etc.). LCC may occasionally show an advantage under small deformations, but it rapidly decays with increasing initial displacement. This aligns with the mechanism of AM-SSIM's multi-scale, learnable weights adaptively emphasizing key scale consistency during training. It preserves local details while maintaining the global structure, thus simultaneously improving accuracy and stability on two benchmark databases. In summary, the results demonstrate that synchronous spatiotemporal co-representation across network layers, using adjustable frequency preferences and adaptive multi-scale similarity as optimization anchors, can simultaneously improve alignment accuracy and the physical feasibility of the deformation field in strongly deformed sequences. When the task objective leans towards minimizing error, CTR is preferable, while LTR or GTR can be introduced to reduce folding risk when topology preservation is emphasized, thus providing an interpretable strategy—a model matching paradigm—for different clinical scenarios.

[0084] Several limitations and areas for improvement must be acknowledged. First, this paper primarily validates the method on monomodal 4D CT; cross-modal scenes still require the introduction of contrast-invariant representations at the feature layers or the integration of generative priors for domain alignment. Second, the computational and memory demands of long sequences and high-resolution voxel grids remain a significant bottleneck; incremental optimization, sparsification, and memory playback warrant further exploration. At the application level, joint optimization with downstream tasks such as segmentation and dosimetry can be explored, along with prospective evaluations and uncertainty quantification based on clinical endpoints, thereby propelling the method towards a clinically usable and interpretable sequence registration system.

[0085] In summary, this embodiment designs a key component, CM-BiLSTM, which deeply integrates parallel multi-scale feature extraction and cross-phase bidirectional long-range temporal modeling mechanisms to achieve efficient representation of spatiotemporal dynamic deformation features. Furthermore, the accompanying AFR adaptively reweights features across frequency bands during the encoding stage, further enhancing the collaborative characterization and dynamic balance of multi-frequency information, giving the network greater flexibility. The further proposed AM-SSIM significantly enhances the model's robust matching ability to complex deformation fields through a multi-scale weight adaptive strategy. Finally, the one-time learning method is organically combined with different sequence registration strategies to reasonably alleviate the constraint of data scarcity. Empirical studies show that this method achieves excellent registration results on two 4D CT benchmark databases, demonstrating its technical advantages in medical image sequence registration tasks. Overall, this method provides an effective and innovative technical paradigm for the clinical application of medical image registration.

[0086] Example 2 An adaptive medical image sequence registration system based on spatiotemporal co-coding includes: The data acquisition module is configured to acquire 4D image sequence data, and each element in the 4D image sequence data is defined as the 3D image corresponding to the corresponding time. The registration module is configured to register 4D image sequence data using a pre-trained deep neural network model to obtain the registration result; The deep neural network model includes a feedforward encoder and a reconstruction decoder. The feedforward encoder adopts a multi-level hierarchical design. Each level includes a Patch Embedding module and a CM-BiLSTM unit. The Patch Embedding module uses stride convolution to implicitly divide the image into blocks, generate feature representations, and simultaneously perform channel expansion and spatial downsampling. The generated features are fed into the corresponding level's CM-BiLSTM unit for modeling. Each level has at least two CM-BiLSTM units stacked together to fuse temporal dependencies and spatial context information in the image sequence. The reconstruction decoder adopts a multi-level hierarchical design. Each level includes a transposed convolutional module and a CNN unit, which are used to progressively upsample and accurately reconstruct the features output by the encoder. Furthermore, feature compensation pathways are constructed between layers through skip connections to fuse shallow spatial information and deep semantic features.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An adaptive medical image sequence registration method based on spatiotemporal co-coding, characterized in that, Includes the following steps: Acquire 4D image sequence data, and define each element in the 4D image sequence data as the corresponding 3D image at the corresponding time point; A pre-trained deep neural network model is used to register 4D image sequence data to obtain the registration results; The deep neural network model includes a feedforward encoder and a reconstruction decoder. The feedforward encoder adopts a multi-level hierarchical design. Each level includes a Patch Embedding module and a CM-BiLSTM unit. The Patch Embedding module uses stride convolution to implicitly divide the image into blocks, generate feature representations, and simultaneously perform channel expansion and spatial downsampling. The generated features are fed into the corresponding level's CM-BiLSTM unit for modeling. Each level has at least two CM-BiLSTM units stacked together to fuse temporal dependencies and spatial context information in the image sequence. The reconstruction decoder adopts a multi-level hierarchical design. Each level includes a transposed convolutional module and a CNN unit, which are used to progressively upsample and accurately reconstruct the features output by the encoder. Furthermore, feature compensation pathways are constructed between layers through skip connections to fuse shallow spatial information and deep semantic features.

2. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 1, characterized in that, The feature channel dimensions of the output of each layer of the feedforward encoder increase sequentially, namely {C, 2C, 4C, 8C}, where C is an integer value; Alternatively, the number of CM-BiLSTM units stacked at each level of the reconstruction decoder is {2, 2, 4, 2}, respectively.

3. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 1, characterized in that, For each time t, the CM-BiLSTM unit first concatenates the current input feature Xt with the hidden state Ht-1 of the previous time step, and then feeds it into the CM block for multi-frequency feature extraction. The CM block contains two paths. The high-frequency path adopts a CNN series structure, focusing on high-frequency features including edges and local textures. The low-frequency path first performs average pooling to reduce the amount of computation, and then uses the Vision Mamba module with linear time complexity and strong temporal memory to model long-range dependencies, focusing on low-frequency components including overall shape and slow changes. Finally, it is upsampled back to the original scale. The outputs of the two paths are fused to obtain an enhanced representation, which is then fed into the gating unit of the bidirectional LSTM. Combined with the cell state Ct-1 of the previous time step, a new hidden state Ht and cell state Ct are generated.

4. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 3, characterized in that, The CM-BiLSTM unit adopts a bidirectional strategy, with both directions sharing the same CM block at each time t. The forward and backward hidden states obtained by bidirectional updates are concatenated, and a unified representation at that time is obtained through a linear mapping layer to drive subsequent registration.

5. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 3, characterized in that, During the fusion of the two path outputs, a monotonically decreasing prior is set in the depth dimension, and sample adaptive soft routing is performed. In the s-th stage, the high-frequency and low-frequency paths output features Yh and Yl in parallel. Based on the global statistical calculation, a scalar gate αs∈[0, 1] representing the contribution ratio of the high-frequency path is obtained, resulting in the fused output Y=αsYh+(1-αs)Yl. The monotonically decreasing prior is initialized to reflect the hierarchical preference of shallow layers emphasizing high frequencies and deep layers emphasizing low frequencies. After training begins, the scalar gate is adaptively adjusted around the prior, allowing data-driven deviations while maintaining the hierarchical trend, thereby achieving progressive modeling from local details to the global structure.

6. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 1, characterized in that, During the training process of the deep neural network model, a multi-constraint loss function is used as the loss function for training. The multi-constraint loss function includes image similarity loss and deformation field regularization loss. The image similarity loss downsamples the input image at several scale factors and calculates the structural similarity index value at each scale. The structural similarity index values ​​at multiple scales are weighted and summed to obtain the overall multi-scale loss. The deformation field regularization loss includes smoothness loss and anti-folding loss. The smoothness loss is represented by the sum of squares of the deformation field gradient, and the anti-folding loss is achieved by restricting the Jacobian determinant of the deformation field.

7. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 1, characterized in that, In the process of registering 4D image sequence data using a pre-trained deep neural network model, a local temporal registration strategy is adopted, which treats adjacent image pairs in the sequence as interrelated local time segments in order to capture small deformations and structural displacements within short time intervals.

8. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 1, characterized in that, In the process of registering 4D image sequence data using a pre-trained deep neural network model, a cross-temporal registration strategy is adopted to select a certain image in the sequence as a reference image, and directly register the reference image with images at multiple other time points to construct a global deformation model across time periods.

9. The adaptive medical image sequence registration method based on spatiotemporal co-coding as described in claim 1, characterized in that, In the process of registering 4D image sequence data using a pre-trained deep neural network model, a full temporal registration strategy is adopted. From the perspective of the entire time axis, the sequence images are globally registered and optimized in a collaborative manner. First, a local temporal registration strategy is used to obtain the local deformation field between adjacent time frames. Then, a cross-temporal registration strategy is combined to construct a global deformation model that spans a longer time interval. The two types of deformation fields are integrated and unified through composite operations.

10. An adaptive medical image sequence registration system based on spatiotemporal co-coding, characterized in that, include: The data acquisition module is configured to acquire 4D image sequence data, and each element in the 4D image sequence data is defined as the 3D image corresponding to the corresponding time. The registration module is configured to register 4D image sequence data using a pre-trained deep neural network model to obtain the registration result; The deep neural network model includes a feedforward encoder and a reconstruction decoder. The feedforward encoder adopts a multi-level hierarchical design. Each level includes a Patch Embedding module and a CM-BiLSTM unit. The Patch Embedding module uses stride convolution to implicitly divide the image into blocks, generate feature representations, and simultaneously perform channel expansion and spatial downsampling. The generated features are fed into the corresponding level's CM-BiLSTM unit for modeling. Each level has at least two CM-BiLSTM units stacked together to fuse temporal dependencies and spatial context information in the image sequence. The reconstruction decoder adopts a multi-level hierarchical design. Each level includes a transposed convolutional module and a CNN unit, which are used to progressively upsample and accurately reconstruct the features output by the encoder. Furthermore, feature compensation pathways are constructed between layers through skip connections to fuse shallow spatial information and deep semantic features.

Images