Three-dimensional medical image registration method based on semantic difference modeling and double-branch Transform

Abstract

The invention discloses a three-dimensional medical image registration method based on semantic difference modeling and a double-branch Transform, and relates to the field of medical image processing. According to the method, local structure keeping, global semantic modeling and boundary semantic guiding mechanisms are combined, the problem that the receptive field of a traditional CNN structure is insufficient is solved, the defect of Transform in the aspect of boundary modeling is overcome, and the method is a high-performance three-dimensional medical image registration scheme suitable for various clinical complex registration scenes; according to the network disclosed by the invention, a double-flow feature extraction mechanism and a Transform structure are introduced, so that local structure modeling and global semantic perception are effectively considered; meanwhile, the semantic diffusion fusion strategy enables the decoding process to be more accurate and robust, the decoding process is particularly superior in the aspects of complex organ deformation and boundary alignment, and a powerful end-to-end solution is provided for a three-dimensional medical image registration task.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and specifically to a three-dimensional medical image registration method based on semantic difference modeling and a dual-branch Transformer. Background Art

[0002] Deformable image registration is crucial in medical imaging. It aligns fixed and moving images by finding a nonlinear mapping that minimizes the difference between them. This process establishes spatial correspondences through a continuous deformation field, which can be modeled in various ways, giving rise to numerous methods. Traditional non-rigid registration methods are often based on variational optimization frameworks, such as the Demons algorithm, the B-spline Free-Form Deformation (FFD) model, and the SyN (Symmetric Normalization) method. These methods optimize the displacement field parameters in a stepwise iteration by constructing similarity metrics and deformation field regularization terms. While some success has been achieved in scenarios with small deformations, their high computational complexity and sensitivity to initial conditions make them difficult to scale to high-resolution 3D data. Furthermore, traditional methods lack robustness and generalization when dealing with complex clinical images, such as those with distorted anatomical structures and blurred boundaries.

[0003] In recent years, deep learning techniques, especially convolutional neural networks (CNNs), have significantly promoted the development of registration. The representative work, VoxelMorph, models registration as a deformation field prediction problem and learns the spatial mapping relationship between images through an end-to-end convolutional network, achieving breakthroughs in both efficiency and accuracy. Subsequent works such as LapIRN and AttentionReg introduced hierarchical modeling and attention mechanisms to further improve deformation estimation capabilities. However, the CNN structure is limited by the local receptive field and has difficulty capturing long-range cross-regional semantic dependencies. In addition, the convolution operation is insufficient to model complex anatomical topology, and performance bottlenecks still exist when aligning large deformations or non-rigid structures.

[0004] To enhance the model's global perception capabilities, Transformers have been gradually introduced into medical image analysis tasks in recent years. TransUNet and Swin-Unet have achieved promising results in segmentation tasks. Their self-attention mechanism can model dependencies between arbitrary locations in the image, making them suitable for registration scenarios with significant structural differences. Some studies, such as ViT-VoxelMorph, have made preliminary attempts to incorporate the ViT architecture into registration networks, demonstrating the advantages of global modeling. The Swin Transformer captures global spatial relationships through an encoder and combines it with a CNN decoder to generate a high-resolution deformation field. However, this feature interaction mechanism still relies on a simple concatenation or addition strategy, resulting in inefficient coupling of local details with global semantics. Furthermore, existing decoders often rely on unidirectional feature upsampling, ignoring anatomical boundary information embedded in shallow encoder features, which can easily lead to non-physiological distortions in complex deformation scenarios. To this end, recent work has used multiple cascades (each estimating a small decomposition of the final deformation) or a coarse-to-fine pyramid of multi-scale displacements within a network to gradually estimate large and complex deformations. While cascade methods improve registration accuracy by combining small deformations to estimate large deformations, their computational cost increases exponentially with the number of cascades. On the other hand, pyramid methods estimate multi-scale deformations and combine them sequentially into the final deformation. For example, LapIRN utilizes a Laplacian pyramid network to capture large deformations by composing three streams at different scales. However, the optimal performance of LapIRN requires complex iterative training at different scales, which negatively affects its training efficiency. PRNet++ adopts a two-stream pyramid network to perform coarse-to-fine registration through sequential warping on multi-scale feature maps, while the adaptation of the local 3D correlation layer greatly increases memory usage and computational cost.

[0005] In addition, the way shallow structures and deep semantics are integrated during the decoding stage of the registration network also has a significant impact on the final accuracy. Although the traditional skip connection mechanism can transmit shallow information to high-level layers, it lacks a sophisticated guidance mechanism and is prone to causing information redundancy and feature drift. To this end, methods such as attention fusion and guided residual fusion have emerged in recent years to improve the efficiency and directionality of feature interactions. However, these methods still generally lack the ability to model the "semantic difference" itself, that is, to actively identify and emphasize areas of structural change in the image, which limits the registration performance under anatomical boundaries and large deformation scenarios.

[0006] In actual clinical scenarios, registration accuracy depends not only on deformation modeling capabilities but also on ensuring the continuity and boundary consistency of anatomical structures during the alignment process. Therefore, effectively integrating the global modeling advantages of the Transformer with the structural perception capabilities of CNN has become a key research topic. Furthermore, feature fusion strategies in the decoding phase are crucial for the information exchange between shallow structures and deep semantics. Improper fusion can easily lead to semantic drift or loss of structural information, compromising the final registration quality. Therefore, a new solution to these problems is needed. Summary of the Invention

[0007] The purpose of the present invention is to provide a three-dimensional medical image registration method based on semantic difference modeling and dual-branch Transformer to solve the technical problems in the background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a 3D medical image registration method based on semantic difference modeling and a dual-branch Transformer, comprising at least the following steps:

[0009] S1: Design the network framework;

[0010] S2: A dual-stream feature extraction module is established in the encoder part of the network framework. The dual-stream feature extraction module, namely the DFEB module, is modeled in parallel with CNN and Swin Transformer to achieve collaborative perception of local structural details and global semantic dependencies;

[0011] S3: Introducing a semantic diffusion fusion module (SDFM) in the decoding stage. The SDFM module constructs a learnable semantic difference map and boundary enhancement strategy, enabling shallow features to achieve structural perception enhancement under the guidance of deep semantics.

[0012] S4: Design the loss function. By optimizing the loss function, the network can fully utilize large-scale training of unlabeled images.

[0013] Furthermore, the network framework includes a multi-scale encoder and a decoder guided by semantic differences;

[0014] Let I f and I m Represent fixed images and moving images respectively. The goal is to generate a three-dimensional deformation field φ, and I f and I m The mapped spatial positions are aligned to achieve accurate registration;

[0015] In the encoding stage, the input fixed image I f With moving images I mFirst, it is spliced into a tensor of size H×W×D×2 in the channel dimension and passes through four levels of feature extraction modules in sequence;

[0016] Each level consists of a Patch Embedding layer and a DFE-former Block. The Patch Embedding layer first divides the image into overlapping patches with a stride of 4, thereby reducing the spatial dimension to aisle;

[0017] Subsequently, the DFE-former Block is used to model the features through a dual-branch structure, focusing on coarse-grained and fine-grained information respectively, thereby achieving complementary expression of local structure and global semantics. As the network deepens, the spatial dimension is reduced layer by layer with a stride of 2, and the number of channels is doubled synchronously. The final output is a size of Deep semantic features of ,realize the gradual capture of multi-scale context information;

[0018] In the decoding stage, four symmetrical ConvUp3D Block modules are used for feature upsampling to gradually restore the spatial resolution. In order to fully fuse the shallow features of the encoder and the deep semantic information of the decoder, a semantic diffusion fusion module is introduced. The SDFM module performs the encoder output F in each decoding layer. i With the current decoding feature G i Deep fusion allows semantic information to be effectively diffused into shallow structures, thereby improving the deformation modeling capability in the decoding stage, especially in boundary areas and scenes with large deformations.

[0019] Finally, the highest resolution features output by the decoder are fed into multiple 3D convolutional layers to regress the final 3D deformation field. The deformation field describes the displacement of the voxel along each axis in three-dimensional space;

[0020] Then, the deformation field φ is applied to the moving image I through a differentiable spatial transformation network m , get the registered image

[0021] Furthermore, the dual-stream feature extraction module includes a local detail modeling branch and a global semantic modeling branch, wherein the local detail modeling branch and the global semantic modeling branch are constructed based on CNN and Swin Transformer respectively, and the local detail modeling branch and the global semantic modeling branch are surface information extraction branches and deep information extraction branches respectively. The surface information is coarse-grained, i.e., rough features, and the deep information is fine-grained, i.e., used to model the relationship between pixels and learn global information;

[0022] The surface information extraction branch consists of two serially connected three-dimensional convolutional layers and a ReLU activation function, forming the basic Conv3D Block structure:

[0023] F conv =Conv3D 3×3×3 (ReLU(Conv3D 3×3×3 (x)))

[0024] In the deep information extraction branch, the Swin Transformer is introduced as the backbone structure to mine long-range dependencies between pixels and perform multi-head self-attention modeling within local windows. The Swin Transformer architecture is improved to adapt to the characteristics of medical images, dividing features into multiple sub-windows. The window shift strategy is used to enhance cross-window information interaction capabilities. The core attention mechanism is as follows:

[0025]

[0026] This formula describes how to calculate the attention score and weight the features by query Q, key K and value V. In this process, Softmax is used to calculate the attention weight of each element. represents the scaled dot product operation, and B is the bias term.

[0027] Furthermore, the SDFM module aims to fuse the low-level features F output by each layer of the encoder i and the deep semantic features G generated by the decoder i ,Through semantic guidance, the modeling capability of structural boundaries is enhanced, and the overall registration accuracy is improved;

[0028] The SDFM module uses the feature map output by the encoder and high semantic features of the decoder As input, a feature enhancement mechanism based on the idea of nonlinear diffusion is introduced;

[0029] First, a semantic difference map is constructed to depict the gradient changes between deep features, which is used to guide shallow features to focus on structural difference areas. The semantic difference map is defined as:

[0030]

[0031] in, and G p Represents voxel point p and its neighborhood respectively; δ(p) represents the 3×3×3 neighborhood centered on voxel point p, and the semantic difference map reflects the G i The degree of change in the spatial neighborhood, thus acting as a regulator, guiding F i Local difference features in ;

[0032] A learnable boundary convolution operator is designed in the SDFM module. The boundary convolution operator learns by fixed weights at the center and adaptive weights at non-center locations to better adapt to anatomical changes in different directions and scales. On this basis, the SDFM module adopts the following nonlinear diffusion update strategy to obtain boundary enhancement features:

[0033]

[0034] Where: h(·) is a set of convolution mapping functions, which is used to perform nonlinear modulation on semantic differences and enhance the feature response of important areas; and Respectively represent the position p at time t e and the eigenvalues at position p;

[0035] Finally, the updated boundary enhancement features are fused with the original features to obtain the fused feature representation:

[0036]

[0037] Among them, λ and ν are weighted coefficients that control the fusion ratio of original and guided features.

[0038] Furthermore, the S5 at least includes the following steps:

[0039] The predicted deformation field It is input into the differentiable spatial transformer network, which transforms the moving image I m Resample and generate the registered image

[0040] During the training process, the loss function consists of two main parts: image similarity loss and deformation smoothness loss:

[0041] The image similarity loss is used to measure the image registration With fixed image I f The degree of similarity between the two images is determined by the normalized correlation coefficient as the similarity measurement function.

[0042] The deformation smoothness loss achieves smoothness constraint on deformation by minimizing the first-order gradient of the displacement vector in three-dimensional space;

[0043] In summary, the total loss function is expressed as the weighted sum of image similarity loss and deformation smoothness loss.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. This invention combines local structure preservation, global semantic modeling, and boundary semantic guidance mechanisms. It not only overcomes the problem of insufficient receptive field of traditional CNN structures, but also makes up for the shortcomings of Transformer in boundary modeling. It is a high-performance 3D medical image registration solution suitable for a variety of complex clinical registration scenarios.

[0046] 2. The network of the present invention effectively balances local structure modeling and global semantic perception by introducing a dual-stream feature extraction mechanism and a Transformer structure. At the same time, the semantic diffusion fusion strategy makes the decoding process more accurate and robust, especially in complex organ deformation and boundary alignment. The present invention provides a powerful end-to-end solution for 3D medical image registration tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A schematic diagram of the present invention as a whole;

[0049] Figure 2 This is the structure diagram of the semantic diffusion fusion module SDFM of the present invention;

[0050] Figure 3 This is a flow chart of the unsupervised training framework of the present invention. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0052] A 3D medical image registration method based on semantic difference modeling and a dual-branch Transformer includes at least the following steps:

[0053] S1: Design the network framework;

[0054] S2: A dual-stream feature extraction module is established in the encoder part of the network framework. The dual-stream feature extraction module, also known as the DFEB module, is modeled in parallel with CNN and Swin Transformer to achieve collaborative perception of local structural details and global semantic dependencies.

[0055] S3: Introducing the semantic diffusion fusion module (SDFM) during the decoding phase. The SDFM module constructs a learnable semantic difference map and boundary enhancement strategy, enabling shallow features to achieve structural enhancement under the guidance of deep semantics, effectively alleviating boundary mismatch and anatomical structure distortion.

[0056] S4: Design the loss function. Through the optimization of the loss function, the network can make full use of large-scale training on unlabeled images, avoiding the subjective errors and data bottleneck problems caused by manual labels, while maintaining strong generalization ability and clinical adaptability.

[0057] See also Figure 1 ,The network framework includes a multi-scale encoder and a decoder guided by semantic differences;

[0058] Let I f and I m Represent fixed images and moving images respectively. The goal is to generate a three-dimensional deformation field φ, and I f and I m The mapped spatial positions are aligned to achieve accurate registration;

[0059] In the encoding stage, the input fixed image I f With moving images I m First, it is spliced into a tensor of size H×W×D×2 in the channel dimension and passes through four levels of feature extraction modules in sequence;

[0060] Each level consists of a Patch Embedding layer and a DFE-former Block. The Patch Embedding layer first divides the image into overlapping patches with a stride of 4, thereby reducing the spatial dimension to aisle;

[0061] Subsequently, the DFE-former Block is used to model the features through a dual-branch structure, focusing on coarse-grained and fine-grained information respectively, thereby achieving complementary expression of local structure and global semantics. As the network deepens, the spatial dimension is reduced layer by layer with a stride of 2, and the number of channels is doubled synchronously. The final output is a size of Deep semantic features of ,realize the gradual capture of multi-scale context information;

[0062] In the decoding stage, four symmetrical ConvUp3D Block modules are used for feature upsampling to gradually restore the spatial resolution. In order to fully integrate the shallow features of the encoder and the deep semantic information of the decoder, a semantic diffusion fusion module is introduced. The SDFM module performs the encoder output F in each decoding layer. i With the current decoding feature Gi Deep fusion allows semantic information to be effectively diffused into shallow structures, thereby improving the deformation modeling capability in the decoding stage, especially in boundary areas and scenes with large deformations.

[0063] Finally, the highest resolution features output by the decoder are fed into multiple 3D convolutional layers to regress the final 3D deformation field. The deformation field describes the displacement of the voxel along each axis in three-dimensional space;

[0064] Then, the deformation field φ is applied to the moving image I through a differentiable spatial transformation network m , get the registered image

[0065] See also Figure 2 The dual-stream feature extraction module includes a local detail modeling branch and a global semantic modeling branch. The local detail modeling branch and the global semantic modeling branch are built based on CNN and SwinTransformer respectively. The local detail modeling branch and the global semantic modeling branch are surface information extraction branches and deep information extraction branches respectively. The surface information is coarse-grained, that is, coarse features, and the deep information is fine-grained, that is, used to model the relationship between pixels and learn global information;

[0066] The surface information extraction branch consists of two serially connected three-dimensional convolutional layers and a ReLU activation function, forming the basic Conv3DBlock structure:

[0067] F conv =Conv3D 3×3×3 (ReLU(Conv3D 3×3×3 (x)))

[0068] The Conv3DBlock architecture focuses on extracting shallow texture information and edge details from images, making it suitable for modeling position-sensitive areas such as deformable boundaries and organ contours. Furthermore, to enhance the expressiveness of low-level features, the present invention uses residual connections to fuse input features with the convolution results, enhancing the representation of shallow-level information.

[0069] In the deep information extraction branch, SwinTransformer is introduced as the backbone structure to mine the long-distance dependency between pixels and perform multi-head self-attention modeling within the local window. Although Transformer shows the advantage of pixel-level correlation modeling in two-dimensional natural images, it faces two challenges when directly applied to three-dimensional medical volume data: one is the destruction of spatial topology. Serializing three-dimensional voxels will destroy the spatial continuity of the anatomical structure; the other is the explosive growth of computational complexity. Global self-attention calculation for full-resolution features requires O ((HWD)2)The complexity far exceeds the GPU memory capacity. Therefore, by improving the Swin Transformer architecture to adapt to the characteristics of medical images, the features are divided into multiple sub-windows, and the cross-window information interaction capability is improved through the window shift strategy (such as SM-SWA in the figure). The core attention mechanism is as follows:

[0070]

[0071] This formula describes how to calculate the attention score and weight the features by query Q, key K and value V. In this process, Softmax is used to calculate the attention weight of each element. represents the scaled dot product operation, and B is the bias term;

[0072] Due to the fundamental differences in the modeling methods of CNN and Transformer, direct addition may lead to feature confusion. Therefore, this paper adopts a splicing + convolution fusion strategy. After splicing the output features of the two branches, a weighted fusion is performed through 1×1×1 convolution, guiding the model to automatically learn the weighted contributions of different branches.

[0073] The SDFM module aims to fuse the low-level features F output by each layer of the encoder i and the deep semantic features G generated by the decoder i ,Through semantic guidance, the modeling capability of structural boundaries is enhanced, and the overall registration accuracy is improved;

[0074] The SDFM module uses the feature map output by the encoder and high semantic features of the decoder As input, a feature enhancement mechanism based on the idea of nonlinear diffusion is introduced;

[0075] First, a semantic difference map is constructed to depict the gradient changes between deep features, which is used to guide shallow features to focus on structural difference areas. The semantic difference map is defined as:

[0076]

[0077] in, and G p Represents voxel point p and its neighborhood respectively; δ(p) represents the 3×3×3 neighborhood centered on voxel point p, and the semantic difference map reflects the G i The degree of change in the spatial neighborhood, thus acting as a regulator, guiding F i Local difference features in ;

[0078] To address the limited extraction capabilities of traditional edge operators in medical images, a learnable boundary convolution operator is designed in the SDFM module. The boundary convolution operator learns by combining fixed weights at the center and adaptive weights at non-center locations to better adapt to anatomical changes in different directions and scales. Based on this, the SDFM module uses the following nonlinear diffusion update strategy to obtain boundary enhancement features:

[0079]

[0080] Where: h(·) is a set of convolution mapping functions, which is used to perform nonlinear modulation on semantic differences and enhance the feature response of important areas; and Respectively represent the position p at time t e and the eigenvalues at position p;

[0081] Finally, the updated boundary enhancement features are fused with the original features to obtain the fused feature representation:

[0082]

[0083] Among them, λ and ν are weighted coefficients that control the fusion ratio of original and guided features.

[0084] The SDFM module can realize multi-scale feature fusion under semantic guidance.

[0085] Through the SDFM module, shallow features can obtain stronger semantic perception capabilities while retaining structural details, thereby significantly improving the fineness and stability of the deformation field in the boundary area. It is particularly suitable for processing medical image registration tasks with severe deformation and blurred boundaries.

[0086] S5 includes at least the following steps:

[0087] The predicted deformation field It is input into the differentiable spatial transformer network, which transforms the moving image I m Resample and generate the registered image

[0088] During the training process, the loss function consists of two main parts: image similarity loss and deformation smoothness loss:

[0089] Image Similarity Loss, used to measure the alignment of images With fixed image I fConsidering the grayscale consistency characteristics in medical image registration, the present invention adopts the Normalized Cross-Correlation (NCC) as the similarity measurement function, which is expressed as:

[0090]

[0091] This term encourages the network to learn a deformation field that maximizes the local similarity between two images.

[0092] Deformation Smoothness Loss: To ensure the spatial continuity and physical feasibility of the predicted deformation field, a regularization term for the displacement field gradient is introduced. This loss implements the smoothness constraint of the deformation by minimizing the first-order gradient of the displacement vector in three-dimensional space. It is defined as:

[0093]

[0094] Where Ω represents all voxels in the image domain, represents the first-order derivative of the deformation field in three dimensions.

[0095] In summary, the total loss function is expressed as the weighted sum of image similarity loss and deformation smoothness loss.

[0096]

[0097] The hyperparameter λ controls the contribution of the deformation smoothing term to the total loss. In this paper, we experimentally set it to a constant value to balance image matching accuracy with the regularity of the deformation field. In this unsupervised framework, the network can fully utilize large-scale training on unlabeled images, avoiding the subjective errors and data bottlenecks caused by manual labeling while maintaining strong generalization and clinical adaptability.

[0098] The specific unsupervised training framework flow chart of the present invention is shown in the attached figure. Figure 3 As shown in Figure 2, the present invention used a 3D brain MRI dataset (IXI). The preprocessed IXI dataset consists of 570 MRI scans (160×192×224) from healthy subjects. For the training, validation, and test sets, the present invention followed the same protocol using partitions 400, 58, and 112, respectively. Using the atlas generated in

[15] , atlas-based brain registration was performed on each scan.

[0099] Specifically, the model in the present invention is implemented based on PyTorch and trained for 1000 epochs using the Adam optimizer. The initial learning rate is set to 1e-4, and a cosine decay strategy is adopted during training. The input image size is 160×192×160, the batch size is set to 1, and no registration labels are used during training. The loss function used in unsupervised training includes: image similarity loss: normalized cross-correlation coefficient (NCC); deformation regularization term: first-order gradient smoothing term; the total loss is the weighted sum of the two, where the regularization term coefficient λ is set to 0.1. All experiments are performed on a single NVIDIA A100 40GB GPU.

[0100] During the implementation process, the system first receives a three-dimensional registered image pair, including a fixed image and a moving image, and splices the two in the channel dimension to form an input tensor. After the input image undergoes preprocessing operations such as normalization, it is input into a multi-level encoder structure. The encoder consists of four layers of feature extraction units, each layer of which contains a patch embedding module (Patch Embedding) and a dual-flow feature extraction block (Dual-Flow Feature Extraction Block, DFEB). The Patch Embedding module divides the image into overlapping patches, reducing the spatial dimension while improving the semantic modeling capability. The DFEB module is an important structural innovation of the present invention. It uses parallel branches constructed by CNN and Swin Transformer to model local texture details and long-range semantic dependency information in the image respectively. The features extracted by the two branches are spliced and fused through a 1×1×1 convolution to achieve collaborative perception of structural features and semantic features. As the network layer deepens, the spatial resolution of the feature map decreases layer by layer, and the number of channels gradually increases to obtain richer contextual semantics.

[0101] After encoding, features are fed into the decoder for multi-level upsampling. The decoder architecture is symmetrical to the encoder. To achieve high-quality feature recovery and structure preservation, the decoder incorporates a Semantic-Diffused Fusion Module (SDFM) at each level to deeply fuse the high-level semantic features of the current decoder layer with the shallow structural features of the corresponding encoder layer. The SDFM module first generates a semantic difference map from the decoded features to characterize the structural differences in regions of deep semantic variation. This difference map then guides the nonlinear enhancement of shallow encoder features in boundary regions. The module employs a learnable boundary convolution kernel to adapt to anatomical structures of varying scales and orientations, and employs a diffusion update strategy to enforce boundary reinforcement on shallow features. The fused features are then weighted to integrate the original semantic features with the guided features, resulting in a more boundary-aware decoded representation, effectively improving the model's representation and fitting capabilities in complex structural regions.

[0102] Finally, the high-resolution features output by the decoder are fed into several 3D convolutional layers to regress a 3D deformation field, which describes the displacement vector of the voxel in space. The deformation field is then fed into a differentiable Spatial Transformer Network (STN) to resample the moving image and generate a registered image aligned with the fixed image.

[0103] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A 3D medical image registration method based on semantic difference modeling and a dual-branch Transformer, characterized by: At least the following steps are included: S1: Design the network framework; S2: A dual-stream feature extraction module is established in the encoder part of the network framework. The dual-stream feature extraction module, namely the DFEB module, is modeled in parallel with CNN and Swin Transformer to achieve collaborative perception of local structural details and global semantic dependencies; S3: Introducing a semantic diffusion fusion module (SDFM) in the decoding stage. The SDFM module constructs a learnable semantic difference map and boundary enhancement strategy, enabling shallow features to achieve structural perception enhancement under the guidance of deep semantics. S4: Design the loss function. Through the optimization of the loss function, the network can fully utilize the large-scale training of unlabeled images.

2. The 3D medical image registration method based on semantic differential modeling and dual-branch Transformer according to claim 1, characterized in that: The network framework includes a multi-scale encoder and a decoder guided by semantic differences; Let I f and I m Represent fixed images and moving images respectively. The goal is to generate a three-dimensional deformation field φ, and I f and I m The mapped spatial positions are aligned to achieve accurate registration; In the encoding stage, the input fixed image I f With moving images I m First, it is spliced into a tensor of size H×W×D×2 in the channel dimension and passes through four levels of feature extraction modules in sequence; Each level consists of a Patch Embedding layer and a DFE-former Block. The Patch Embedding layer first divides the image into overlapping patches with a stride of 4, thereby reducing the spatial dimension to aisle; Subsequently, the DFE-former Block is used to model the features through a dual-branch structure, focusing on coarse-grained and fine-grained information respectively, thereby achieving complementary expression of local structure and global semantics. As the network deepens, the spatial dimension is reduced layer by layer with a stride of 2, and the number of channels is doubled synchronously. The final output is a size of Deep semantic features of ,realize the gradual capture of multi-scale context information; In the decoding stage, four symmetrical ConvUp3D Block modules are used for feature upsampling to gradually restore the spatial resolution. In order to fully fuse the shallow features of the encoder and the deep semantic information of the decoder, a semantic diffusion fusion module is introduced. The SDFM module performs the encoder output F in each decoding layer. i With the current decoding feature G i Deep fusion allows semantic information to be effectively diffused into shallow structures, thereby improving the deformation modeling capability in the decoding stage, especially in boundary areas and scenes with large deformations. Finally, the highest resolution features output by the decoder are fed into multiple 3D convolutional layers to regress the final 3D deformation field. The deformation field describes the displacement of the voxel along each axis in three-dimensional space; Then, the deformation field φ is applied to the moving image I through a differentiable spatial transformation network m , get the registered image 3. The 3D medical image registration method based on semantic differential modeling and dual-branch Transformer according to claim 2, characterized in that: The dual-stream feature extraction module includes a local detail modeling branch and a global semantic modeling branch, which are respectively constructed based on CNN and Swin Transformer. The local detail modeling branch and the global semantic modeling branch are respectively the surface information extraction branch and the deep information extraction branch. The surface information is coarse-grained, i.e., rough features, and the deep information is fine-grained, i.e., used to model the relationship between pixels and learn global information; The surface information extraction branch consists of two serially connected three-dimensional convolutional layers and a ReLU activation function, forming the basic Conv3D Block structure: F conv =Conv3D 3×3×3 (ReLU(Conv3D 3×3×3 (x))) In the deep information extraction branch, the Swin Transformer is introduced as the backbone structure to mine long-range dependencies between pixels and perform multi-head self-attention modeling within local windows. The Swin Transformer architecture is improved to adapt to the characteristics of medical images, dividing features into multiple sub-windows. The window shift strategy is used to enhance cross-window information interaction capabilities. The core attention mechanism is as follows: This formula describes how to calculate the attention score and weight the features by query Q, key K and value V. In this process, Softmax is used to calculate the attention weight of each element. represents the scaled dot product operation, and B is the bias term.

4. The 3D medical image registration method based on semantic differential modeling and dual-branch Transformer according to claim 2, characterized in that: The SDFM module aims to fuse the low-level features F output by each layer of the encoder i and the deep semantic features G generated by the decoder i ,Through semantic guidance, the modeling capability of structural boundaries is enhanced, and the overall registration accuracy is improved; The SDFM module uses the feature map output by the encoder and high semantic features of the decoder As input, a feature enhancement mechanism based on the idea of nonlinear diffusion is introduced; First, a semantic difference map is constructed to depict the gradient changes between deep features, which is used to guide shallow features to focus on structural difference areas. The semantic difference map is defined as: in, and G p Represents voxel point p and its neighborhood respectively; δ(p) represents the 3×3×3 neighborhood centered on voxel point p, and the semantic difference map reflects the G i The degree of change in the spatial neighborhood, thus acting as a regulator, guiding F i Local difference features in ; A learnable boundary convolution operator is designed in the SDFM module. The boundary convolution operator learns by fixed weights at the center and adaptive weights at non-center locations to better adapt to anatomical changes in different directions and scales. On this basis, the SDFM module adopts the following nonlinear diffusion update strategy to obtain boundary enhancement features: Where: h(·) is a set of convolution mapping functions, which is used to perform nonlinear modulation on semantic differences and enhance the feature response of important areas; and Respectively represent the position p at time t e and the eigenvalues at position p; Finally, the updated boundary enhancement features are fused with the original features to obtain the fused feature representation: Among them, λ and ν are weighted coefficients that control the fusion ratio of original and guided features.

5. The 3D medical image registration method based on semantic differential modeling and dual-branch Transformer according to claim 2, characterized in that: The S5 at least includes the following steps: The predicted deformation field It is input into the differentiable spatial transformer network, which transforms the moving image I m Resample and generate the registered image During the training process, the loss function consists of two main parts: image similarity loss and deformation smoothness loss: The image similarity loss is used to measure the image registration With fixed image I f The degree of similarity between them is measured using the normalized mutual correlation coefficient as the similarity measurement function; The deformation smoothness loss achieves smoothness constraint on deformation by minimizing the first-order gradient of the displacement vector in three-dimensional space; In summary, the total loss function is expressed as the weighted sum of image similarity loss and deformation smoothness loss.

Images