Image registration method and system based on discontinuity assumption and segmentation-driven approach
By segmenting the image into sub-regions and performing local registration, and combining this with segmentation mask guidance, a discontinuous deformation field is generated, which solves the problem of decreased registration accuracy in complex scenes in traditional methods and achieves high-precision image registration.
Patent Information
- Application Number
- CN202511273290.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-08
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2045-09-08
AI Technical Summary
Traditional deformable image registration methods suffer from decreased registration accuracy in complex anatomical scenes, especially due to excessive constraints caused by the global smoothing regularization assumption, making it difficult to capture the different motion characteristics of different organs and tissues.
The image is segmented into multiple sub-regions, each sub-region is independently registered, and a complete deformation field is generated by integrating the deformation fields of the sub-region pairs. With the guidance of the segmentation mask, the discontinuity of the deformation field is preserved, and a backbone registration network is used for feature extraction and deformation field prediction.
It improves registration accuracy and anatomical rationality, breaks through the global smoothness constraint, and achieves image registration that is more consistent with real biomechanical characteristics.
Smart Images
Figure CN120747175B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically relating to an image registration method and system based on the discontinuity assumption and segmentation-driven approach. Background Technology
[0002] Deformable image registration is a key technology in the field of medical imaging, widely used in tasks such as surgical planning, surgical navigation, disease diagnosis, and motion analysis. Its core lies in constructing a dense nonlinear correspondence between moving and stationary images, i.e., a deformation field.
[0003] Traditional deformable image registration methods typically employ multi-step optimization strategies, achieving registration by minimizing an error function comprised of a similarity metric and smoothness regularization. The similarity metric quantifies the difference between the distorted moving image and the stationary image, while smoothness regularization enforces the smoothness of the deformable field to reduce the impact of irregular motion. However, the introduction of smoothness regularization can lead to oversimplification of the deformable model, thereby reducing registration accuracy. Furthermore, while traditional methods can achieve high-precision registration, their computational efficiency is low, making them unsuitable for applications requiring rapid registration.
[0004] Currently, deep learning-based registration methods are widely used to achieve efficient and near real-time registration, such as unsupervised registration networks. However, the trend of improving registration accuracy by increasing the complexity of registration models has stabilized in recent years. The main limitation of this approach lies in its commonly used smoothing regularization assumption, namely, that the deformation field is globally smooth (Assumption A1). This assumption performs well in simple tasks, but in complex anatomical scenarios, overly constrained regularization can lead to a decrease in registration accuracy. For example, global smoothing regularization struggles to capture the different motion characteristics of different organs and tissues, thus limiting registration performance. To improve registration accuracy, registration techniques that maintain discontinuity are needed, meaning the deformation field is locally smooth but globally discontinuous (Assumption A2). However, achieving this assumption is quite difficult without structural information. Summary of the Invention
[0005] This invention provides an image registration method and system based on the discontinuity assumption and segmentation-driven approach. It divides the original fixed and moving images into multiple sub-regions and constructs sub-region pairs for registration. A complete deformation field is generated by integrating the sub-deformation fields of multiple sub-region pairs to construct a medical image registration model. This method simplifies registration requirements by decomposing complex image registration into multiple simple sub-region registration tasks. Furthermore, by combining the guidance of segmentation masks for each sub-region, it preserves the discontinuity of the deformation field, thereby improving the upper limit of the registration method's accuracy.
[0006] An image registration method based on discontinuity assumptions and segmentation-driven methods includes:
[0007] Acquire stationary and moving images, and segment the stationary and moving images into multiple sub-region pairs;
[0008] A fixed image and a moving image with sub-region pairs are selected and stitched together to form an input image. A backbone registration network is used to predict and generate sub-deformation fields for each sub-region pair. The sub-deformation fields of each sub-region pair are then integrated to generate a complete deformation field. A medical image registration model is constructed and optimized. The backbone registration network includes an encoder module, a field estimation module, and a deformation field combination module. The encoder module extracts input image features, the field estimation module performs input image feature transformation and low-resolution displacement field prediction, and the deformation field combination module transforms and integrates the features to generate sub-deformation fields.
[0009] The original moving image is applied to the optimized medical image registration model to generate a registered distorted image, thus completing the registration between the moving image and the fixed image.
[0010] By dividing the original fixed and moving images into multiple sub-regions and constructing sub-region pairs, registration based on these sub-region pairs is performed. A complete deformation field is generated by integrating the sub-deformation fields of multiple sub-region pairs to construct a medical image registration model. This model simplifies the registration requirements by decomposing complex image registration into multiple simple sub-region registration tasks. Furthermore, by combining the guidance of each sub-region segmentation mask, the discontinuity of the deformation field is preserved, thereby improving the upper limit of the registration method's accuracy.
[0011] Furthermore, the step of acquiring fixed and moving images, and segmenting the fixed and moving images into multiple sub-region pairs, includes:
[0012] Acquire training images; the training images include both stationary and moving images;
[0013] A pre-trained SAM segmentation model is used to segment the training images into anatomical structures to obtain the semantic mask of the training images;
[0014] The training images are unfolded along the slice direction and then a two-dimensional image sequence is synthesized using frame stitching technology.
[0015] The two-dimensional image sequence is input into the SAM segmentation model, and the two-dimensional image segmentation result is obtained automatically.
[0016] Based on the two-dimensional image segmentation results, the image is back-projected and reconstructed into three-dimensional sub-regions, and each three-dimensional sub-region has a corresponding semantic mask, so that the training image is segmented into multiple sub-regions.
[0017] Region mapping technology is used to align corresponding sub-regions of fixed and moving images to form multiple sub-region pairs.
[0018] By segmenting the original fixed and moving images into multiple sub-regions and aligning the corresponding local sub-regions individually, it is easier to capture the best smooth registration within each sub-region, which in turn benefits the inherent motion continuity within the local sub-regions. At the same time, the SAM segmentation model can automatically identify and segment the precise boundary mask of each sub-region and its corresponding anatomical semantic label, achieving smooth deformation field within the sub-region and allowing discontinuous motion between regions, thus conforming to real biomechanical characteristics and improving registration accuracy and anatomical rationality.
[0019] Furthermore, in the backbone registration network,
[0020] The encoder module consists of three convolutional blocks and two trilinear downsampling layers, used to extract image features; the convolutional block includes a 3×3×3 convolutional layer, a batch normalization layer, and an activation layer.
[0021] The field estimation module includes a Hadamard transform layer, three convolutional blocks, and a convolutional layer, which are used to process and predict low-resolution displacement fields based on extracted image features.
[0022] The deformation field combination module includes two trilinear upsampling layers, a differential transformation layer, and a spatial transformation layer, which are used to generate a deformation field based on deformation estimation of the displacement field.
[0023] Furthermore, the process of selecting fixed and moving images with sub-region pair relationships to stitch together to form an input image, using a backbone registration network to predict and generate sub-deformation fields for each sub-region pair, integrating the sub-deformation fields of each sub-region pair to generate a complete deformation field, and constructing and optimizing a medical image registration model includes:
[0024] Obtain the sub-region pair relationships between stationary and moving images;
[0025] A set of sub-region pairs with anatomical consistency is randomly selected as training objects and input into the backbone registration network for training to predict and generate the sub-deformation fields of the corresponding sub-region pairs.
[0026] Based on all sub-regions of the fixed and moving images, they are respectively input into the backbone registration network to predict and generate the sub-deformation fields of each sub-region pair;
[0027] Based on the sub-deformation fields of each sub-region pair, deformation field synthesis technology is used to integrate them to generate a complete deformation field and construct a medical image registration model.
[0028] By combining multiple loss functions, the medical image registration model is optimized.
[0029] By integrating and merging the predicted smooth sub-deformation fields of sub-regions into a complete deformation field, it is possible to ensure that the final deformation maintains both local smoothness and global discontinuity; thus, while adhering to the registration principle of maintaining discontinuity, it can effectively adapt to different motion patterns between different regions.
[0030] Furthermore, the random selection of a set of anatomically consistent sub-region pairs as training objects, inputting them into the backbone registration network for training, and predicting and generating the sub-deformation fields of the corresponding sub-region pairs includes:
[0031] Based on sub-region pairs of moving and stationary images, a set of sub-region pairs with anatomical consistency is randomly selected, and the pairs are stitched together along the channel dimension to serve as the input image.
[0032] Based on the input image, it is input into the backbone registration network to extract image features and output a sub-deformation field. The process of extracting image features and outputting a sub-deformation field includes inputting the input image into the encoder module of the backbone registration network, extracting downsampled features of different scales using different trilinear downsampling layers, and passing the downsampled features of different scales to the trilinear upsampling layers of the corresponding scales in the deformation field combination module to extract upsampled features of different scales, and fusing them to generate image features and outputting a sub-deformation field.
[0033] By employing a weakly supervised training strategy based on random region registration, sub-region pairs with anatomical consistency are randomly selected from the original image for training. This approach is applicable to medical image registration scenarios with local deformation differences, effectively improving registration accuracy and maintaining the topological consistency of anatomical structures.
[0034] Furthermore, the method of jointly optimizing the medical image registration model using multiple loss functions includes:
[0035] Voxel loss is calculated and obtained based on the voxel loss function.
[0036] The mask loss is calculated based on the mask loss function.
[0037] The regularization loss is calculated based on the regularization loss function.
[0038] By combining voxel loss, mask loss, and regularization loss, a hybrid loss is constructed to optimize the medical image registration model.
[0039] Furthermore, the mask loss function adopts the EDT loss function; the mask loss is calculated using the EDT loss function, including:
[0040] Based on the semantic masks of the original fixed image and the moving image, they are converted into fixed EDT masks and moving EDT masks, respectively.
[0041] Based on the fixed EDT mask, and combined with the deformation function of each sub-region from the moving image to the fixed image, a composite moving EDT mask is formed.
[0042] By combining the overall moving EDT mask, the fixed EDT mask, and the deformation functions corresponding to each sub-region, the overall EDT loss function of the region is calculated.
[0043] A system based on the discontinuity assumption and segmentation-driven image registration method includes:
[0044] The image acquisition and preprocessing module is used to acquire fixed and moving images and segment them into multiple sub-region pairs.
[0045] The model training and optimization module is used to select fixed and moving images with sub-region pairs to stitch together to form an input image. It uses a backbone registration network to predict and generate sub-deformation fields for each sub-region pair, and integrates these sub-deformation fields to generate a complete deformation field, thus constructing and optimizing the medical image registration model. The backbone registration network includes an encoder module, a field estimation module, and a deformation field combination module. The encoder module extracts input image features, the field estimation module performs input image feature transformation and low-resolution displacement field prediction, and the deformation field combination module transforms and integrates these features to generate sub-deformation fields.
[0046] The image registration module is used to apply the original moving image to the optimized medical image registration model, generate the registered distorted image, and complete the registration of the moving image and the fixed image.
[0047] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described above.
[0048] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.
[0049] The beneficial effects of this invention are as follows:
[0050] This invention divides the original fixed and moving images into multiple sub-regions and constructs sub-region pairs for registration. It then generates a complete deformation field by integrating the deformation fields of multiple sub-region pairs to construct a medical image registration model. This simplifies registration requirements by decomposing complex image registration into multiple simple sub-region registration tasks. Furthermore, by incorporating sub-region segmentation masks as guidance, it preserves the discontinuity of the deformation field, thereby improving the upper limit of registration accuracy. It overcomes the limitations of traditional global smoothing constraints by introducing semantic segmentation information to guide sub-region-level registration, achieving smooth deformation fields within sub-regions while allowing discontinuous movement between regions. This better reflects real biomechanical characteristics and significantly improves registration accuracy and anatomical plausibility. Attached Figure Description
[0051] Figure 1 This is a flowchart of the present invention;
[0052] Figure 2 A schematic diagram of the backbone registration network;
[0053] Figure 3 A schematic diagram of the encoder module in the backbone registration network;
[0054] Figure 4 A schematic diagram of the field estimation module for the backbone registration network;
[0055] Figure 5 A schematic diagram of the deformation field combination module in the backbone registration network;
[0056] Figure 6 A schematic diagram illustrating the training process for generating sub-deformation fields using a medical image registration model;
[0057] Figure 7 A schematic diagram of the test process for generating a deformation field using a medical image registration model;
[0058] Figure 8 This is a schematic diagram of the system structure of the present invention;
[0059] Figure 9 This is a schematic diagram of the structure of a computer device. Detailed Implementation
[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0061] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.
[0062] In addition, specific details are provided in the following description to facilitate a thorough understanding of the examples, and those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0063] Example 1
[0064] Figure 1 This paper presents an image registration method based on the discontinuity assumption and segmentation-driven approach. It divides the original stationary and moving images into multiple sub-regions and constructs sub-region pairs for registration. A complete deformation field is generated by integrating the sub-deformation fields of multiple sub-region pairs to construct a medical image registration model. This method simplifies registration requirements by decomposing complex image registration into multiple simple sub-region registration tasks. Furthermore, by incorporating the guidance of sub-region segmentation masks, it preserves the discontinuity of the deformation field, thereby improving the upper limit of the registration method's accuracy. The specific steps include the following:
[0065] S1: Acquire stationary and moving images, and segment the stationary and moving images into multiple sub-region pairs;
[0066] S11: Obtain training images; training images include stationary and moving images;
[0067] S12: A pre-trained SAM segmentation model is used to segment the anatomical structure of the fixed image and the moving image respectively, and the semantic mask of the fixed image and the semantic mask of the moving image are obtained.
[0068] S13: Unfold the fixed image and the moving image along the slice direction respectively, and use frame stitching technology to synthesize a two-dimensional image sequence to obtain a two-dimensional image sequence of the fixed image and a two-dimensional image sequence of the moving image, while preserving the spatial correlation of the slices;
[0069] S14: Input the two-dimensional image sequence of the fixed image and the two-dimensional image sequence of the moving image into the SAM segmentation model respectively, and automatically obtain the two-dimensional image segmentation results of the fixed image and the two-dimensional image segmentation results of the moving image.
[0070] S15: Based on the two-dimensional image segmentation results of the fixed image and the two-dimensional image segmentation results of the moving image, they are back-projected and reconstructed into three-dimensional sub-regions, and each three-dimensional sub-region has a corresponding semantic mask, so that the fixed image and the moving image are segmented into multiple sub-regions respectively.
[0071] S16: Use region mapping technology to align corresponding sub-regions of fixed and moving images to form multiple sub-region pairs.
[0072] S2: Select fixed and moving images with sub-region pair relationships and stitch them together to form the input image. Use a backbone registration network to predict and generate the sub-deformation field of each sub-region pair, and integrate the sub-deformation fields of each sub-region pair to generate a complete deformation field. Construct and optimize the medical image registration model.
[0073] S21: Obtain the sub-region pair relationship between the fixed image and the moving image;
[0074] S22: Randomly select a set of sub-region pairs with anatomical consistency as training objects, input them into the backbone registration network for training, predict and generate the sub-deformation field of the corresponding sub-region pairs;
[0075] S221: Based on the sub-region pairs of moving and stationary images, a set of sub-region pairs with anatomical consistency is randomly selected, and the pairs are stitched together in the channel dimension as the input image;
[0076] S222: Based on the input image, input it into the backbone registration network, extract image features and output sub-deformation field;
[0077] The backbone registration network includes:
[0078] An encoder module consists of three convolutional blocks and two trilinear downsampling layers for extracting image features. The convolutional block includes a 3×3×3 convolutional layer, a batch normalization layer, and an activation layer, with the activation layer using the ReLU activation function. The two trilinear downsampling layers perform downsampling operations on the motion image and the training image at different scales, and generate downsampled features at the corresponding scales.
[0079] A field estimation module, comprising a Hadamard transform layer, three convolutional blocks, and a convolutional layer, is used to process extracted image features and predict low-resolution displacement fields.
[0080] A deformation field combination module includes two trilinear upsampling layers, a differential transformation layer, and a spatial transformation layer, used to generate a deformation field based on displacement field deformation estimation; the differential transformation layer includes displacement field scaling operation, squaring operation, and deformation function operation; the spatial transformation layer includes scaling operation.
[0081] In this embodiment, extracting image features and outputting a sub-deformation field includes: inputting the input image into the encoder module of the backbone registration network, extracting downsampled features of different scales using different trilinear downsampling layers; passing the downsampled features of different scales to the trilinear upsampling layers of the corresponding scales in the deformation field combination module, extracting upsampled features of different scales, fusing them to generate image features, and outputting a sub-deformation field.
[0082] The expression for image registration using sub-deformation fields is as follows:
[0083] ;
[0084] ;
[0085] In the formula, The deformation function representing the transformation from a moving image to a stationary image; This represents the displacement field, used to represent the movement of each voxel in the image; Represents the position coordinates in the image; Represents a fixed image; Represents a moving image;
[0086] Figure 2 The diagram shows the structure of the backbone registration network. Figures 3-5 The diagrams shown are schematic representations of the encoder module, field estimation module, and deformation field combination module in the backbone registration network.
[0087] in, , These represent the outputs of the convolutional blocks in the encoder modules for moving and stationary images, respectively. , These represent the outputs of the first trilinear downsampling layer in the encoder modules for moving and stationary images, respectively. , These represent the outputs of the second trilinear downsampling layer in the encoder modules for moving and stationary images, respectively. Indicates the encoder module number The output after being transformed by the Hadamard transform layer; This represents the deformation function corresponding to the second trilinear downsampling layer; This represents the deformation function corresponding to the first trilinear downsampling layer; This represents the deformation function corresponding to the convolutional block layer in the encoder; , These represent the moving image and the stationary image in the encoder module, respectively. One output; Indicates the first Each submodule corresponds to a displacement field; Indicates the first The shape variables corresponding to each submodule; Indicates the first Deformation functions for each submodule; Indicates the first The output after upsampling operation of each submodule.
[0088] S23: Based on all sub-regions of the fixed and moving images, input them into the backbone registration network to predict and generate the sub-deformation fields of each sub-region pair;
[0089] In this embodiment, the moving image and the stationary image are respectively divided into Each sub-region has a smooth sub-deformation field.
[0090] The motion mask of a moving image is then represented as:
[0091] ;
[0092] In the formula, A motion mask representing a moving image; This indicates the number of image labels, i.e., the number of sub-regions;
[0093] The fixed mask for the fixed image is:
[0094] ;
[0095] In the formula, A fixed mask representing a fixed image;
[0096] In this context, since the element-wise product of the fixed image and its fixed mask within a sub-region is equivalent to the product of the corresponding deformation function acting on the moving image and its motion mask within that sub-region, that is, during the training phase, the... The registration relationship within each sub-region can be represented as:
[0097] ;
[0098] In the formula, Indicates the first Fixed mask for fixed images in each sub-region; Indicates the first Motion mask of moving images in each sub-region; Indicates the first The deformation function from the moving image to the fixed image in each sub-region is used to align the sub-region of the moving image to the corresponding sub-region of the fixed image, thereby achieving local non-rigid registration; This indicates the total number of sub-regions.
[0099] S24: Based on the sub-deformation fields of each sub-region pair, the deformation field synthesis technology is used to integrate them to generate a complete deformation field and construct a medical image registration model;
[0100] Among them, deformation field synthesis technology is used to integrate the corresponding sub-deformation fields of each sub-region into the whole deformation field. That is, by performing mask weighted summation on the sub-deformation fields corresponding to each sub-region, differentiated deformation control of different anatomical structures is achieved, and a smooth, continuous and structurally consistent complete deformation field is synthesized.
[0101] The deformation field corresponding to the segmentation mask is represented as:
[0102] ;
[0103] In the formula, This represents the deformation function of the entire deformation field.
[0104] Based on this, by using the deformation function of the entire deformation field to perform spatial transformation on the moving image, a distorted image corresponding to the fixed image is obtained, thereby completing the non-rigid registration of medical images.
[0105] In this embodiment, Figure 6 The diagram shown illustrates the generation of the overall deformation field, resulting in an overall distorted image.
[0106] S25: Combine multiple loss functions to optimize the medical image registration model.
[0107] S251: Calculate and obtain voxel loss based on voxel loss function;
[0108] In this embodiment, voxel loss The loss function is calculated using the mean squared error (MSE) or normalized cross-correlation (NCC) function.
[0109] The MSE loss function quantifies the overall deviation in intensity distribution between the two images by calculating the squared mean of the differences between the voxel values in the fixed image and the corresponding voxel values in the predicted image. Its expression is:
[0110] ;
[0111] In the formula, Indicates the mean square error loss; The number representing the voxel. , Represents the total number of prime numbers; Represents the first in a fixed image Individual value; Indicates the first image in the predicted image Individual value;
[0112] The NCC loss function measures the structural similarity between two images by evaluating the ratio of the product of the local intensity patterns of the stationary and moving images to their respective standard deviations. Its expression is as follows:
[0113] ;
[0114] In the formula, Indicates normalized loss; Represents the mean of a fixed image; This represents the mean of the predicted image.
[0115] S252: Calculate and obtain the mask loss based on the mask loss function;
[0116] In this embodiment, the mask loss function is the EDT loss function; the mask loss is calculated using the EDT loss function, including:
[0117] S2521: Based on the semantic masks of the original fixed image and the moving image, convert them into fixed EDT masks and moving EDT masks respectively;
[0118] In this embodiment, during the training process, the input registration pairs only include foreground and background, which are converted into EDT masks. Specifically, pixels / voxels in the foreground are converted into their distances from the boundary, thereby assigning different weights to pixels / voxels and extending to pixels / voxels on the boundary, as shown below:
[0119] ;
[0120] In the formula, Represents the Euclidean distance transformation value of the target pixel or voxel; Indicates the boundary of the foreground. Represents pixels or voxels on the boundary; This represents the target pixel or voxel. In this embodiment, the Euclidean distance transformation value of the target pixel or voxel closer to the interior of the region is larger, the Euclidean distance transformation value of the target pixel or voxel closer to the boundary of the region is smaller, and the Euclidean distance transformation value of the target pixel or voxel at the boundary of the region is zero.
[0121] By converting the semantic masks of the original fixed and moving images into fixed EDT masks and moving EDT masks, the EDT transformation is achieved. This transforms the semantic mask into a distance map with a smooth gradient, thereby providing continuous and detailed geometric constraint information for subsequent deformations. This significantly enhances the sensitivity of the registration process to anatomical structure boundaries and the alignment accuracy.
[0122] S2522: Based on a fixed EDT mask, combined with the deformation function of each sub-region centering from a moving image to a fixed image, a composite moving EDT mask is formed.
[0123] S2523: By combining the overall moving EDT mask, the fixed EDT mask, and the deformation function corresponding to each sub-region, the EDT loss function of the overall region is calculated.
[0124] The mask loss function uses the EDT loss function, which evaluates the degree of agreement between the deformed moving EDT mask and the fixed EDT mask on the boundary distance distribution by comparing the difference between the two. Its expression is as follows:
[0125] ;
[0126] In the formula, This represents the EDT loss across all regions; Indicates the moving EDT mask; Indicates a fixed EDT mask; This represents the deformation function for each sub-region from a moving image to a stationary image; The operator representing function composition is used to combine the moving EDT masks of each sub-region into a whole moving EDT mask;
[0127] S253: Calculate and obtain the regularization loss based on the regularization loss function;
[0128] In this embodiment, regularization loss The loss function is calculated using L2 regularization, and its expression is:
[0129] ;
[0130] In the formula, This represents the regularization loss, used to enhance the smoothness of the predicted deformation field; This represents the displacement field generated by registration; Represents the displacement field The spatial gradient is used to penalize non-smooth fluctuations in the deformation field, thereby avoiding physically unreliable deformation or folding phenomena.
[0131] S254: Combine voxel loss, mask loss, and regularization loss to construct a hybrid loss and optimize the medical image registration model;
[0132] In this embodiment, the expression for the hybrid loss is:
[0133] ;
[0134] In the formula, Indicates mixed loss; This represents voxel loss, which includes MSE loss. Or NCC loss This is used to measure the consistency of image intensity; This represents the mask loss, used to assess the alignment of anatomical structure boundaries; , , These represent the hyperparameters for voxel loss, mask loss, and regularization loss, respectively, and are used to adjust the relative contributions of each loss to balance the trade-off between registration accuracy and deformation smoothness.
[0135] S3: Apply the original moving image to the optimized medical image registration model to generate a registered distorted image, thus completing the registration of the moving image and the fixed image.
[0136] In this embodiment, Figure 7 The diagram shows the qualitative effect of image registration using a medical image registration model. A qualitative comparison with existing registration models reveals that the medical image registration model used in this application is superior in terms of deformation field smoothness. The smoothness of the deformation field is one of the key indicators for evaluating image registration quality, signifying a better registration effect.
[0137] Example 2
[0138] Based on the same technological concept, such as Figure 8 As shown, this embodiment also provides a system based on the discontinuity assumption and segmentation-driven image registration method, including an image acquisition and preprocessing module, a model training and optimization module, and an image registration module.
[0139] Specifically, the image acquisition and preprocessing module is used to acquire fixed images and moving images, and to segment the fixed images and moving images into multiple sub-region pairs;
[0140] Specifically, the model training and optimization module is used to select fixed and moving images with sub-region pairs to stitch together to form the input image. It uses a backbone registration network to predict and generate sub-deformation fields for each sub-region pair, and integrates the sub-deformation fields of each sub-region pair to generate a complete deformation field, thus constructing and optimizing the medical image registration model. The backbone registration network includes an encoder module, a field estimation module, and a deformation field combination module. The encoder module extracts input image features, the field estimation module performs input image feature transformation and low-resolution displacement field prediction, and the deformation field combination module transforms and integrates the features to generate sub-deformation fields.
[0141] Specifically, the image registration module is used to apply the original moving image to the optimized medical image registration model to generate a registered distorted image, thus completing the registration of the moving image and the fixed image.
[0142] Example 3
[0143] Based on the same technical concept, embodiments of this application also provide a computer device, including a memory 1 and a processor 2, such as... Figure 9 As shown, the memory 1 stores a computer program, and the processor 2 executes the computer program to implement any of the methods described above.
[0144] The memory 1 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 1 can be an internal storage unit of an image registration system based on discontinuity assumptions and segmentation-driven methods, such as a hard disk. In other embodiments, the memory 1 can also be an external storage device of the image registration system based on discontinuity assumptions and segmentation-driven methods, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 1 can include both internal storage units and external storage devices for the image registration system based on discontinuity assumptions and segmentation-driven methods. The memory 1 can be used not only to store application software and various types of data installed in the image registration system based on discontinuity assumptions and segmentation-driven methods, such as the code of the image registration system program, but also to temporarily store data that has been output or will be output.
[0145] In some embodiments, processor 2 may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip, used to run program code stored in memory 1 or process data, such as executing an image registration system program based on discontinuity assumptions and segmentation drive.
[0146] The present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
[0147] The computer program product of the application page content refresh method provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.
[0148] The present invention also discloses a computer program that, when executed by a processor, implements any of the methods described in the foregoing embodiments. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.
[0149] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.
[0150] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.
[0151] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.
[0152] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0153] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0154] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0155] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.
[0156] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0157] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. An image registration method based on the discontinuity assumption and segmentation-driven approach, characterized in that, include: Acquire stationary and moving images, and segment the stationary and moving images into multiple sub-region pairs; A fixed image and a moving image with sub-region pairs are selected and stitched together to form an input image. A backbone registration network is used to predict and generate sub-deformation fields for each sub-region pair. The sub-deformation fields of each sub-region pair are then integrated to generate a complete deformation field. A medical image registration model is constructed and optimized. The backbone registration network includes an encoder module, a field estimation module, and a deformation field combination module. The encoder module extracts input image features, the field estimation module performs input image feature transformation and low-resolution displacement field prediction, and the deformation field combination module transforms and integrates the features to generate sub-deformation fields. The original moving image is applied to the optimized medical image registration model to generate a registered distorted image, thus completing the registration between the moving image and the fixed image. In the backbone registration network The encoder module consists of three convolutional blocks and two trilinear downsampling layers, used to extract image features; the convolutional block includes a 3×3×3 convolutional layer, a batch normalization layer, and an activation layer. The field estimation module includes a Hadamard transform layer, three convolutional blocks, and a convolutional layer, which are used to process and predict low-resolution displacement fields based on extracted image features. The deformation field combination module includes two trilinear upsampling layers, a differential transformation layer, and a spatial transformation layer, which are used to generate a deformation field based on deformation estimation of the displacement field. The process involves selecting fixed and moving images with sub-region pair relationships, stitching them together to form the input image, using a backbone registration network to predict and generate sub-deformation fields for each sub-region pair, integrating the sub-deformation fields of each sub-region pair to generate a complete deformation field, and constructing and optimizing a medical image registration model, including: Obtain the sub-region pair relationships between stationary and moving images; A set of sub-region pairs with anatomical consistency is randomly selected as training objects and input into the backbone registration network for training to predict and generate the sub-deformation fields of the corresponding sub-region pairs. Based on all sub-regions of the fixed and moving images, they are respectively input into the backbone registration network to predict and generate the sub-deformation fields of each sub-region pair; Based on the sub-deformation fields of each sub-region pair, deformation field synthesis technology is used to integrate them to generate a complete deformation field and construct a medical image registration model. By combining multiple loss functions, the medical image registration model is optimized.
2. The image registration method based on discontinuity assumption and segmentation-driven method according to claim 1, characterized in that, The process of acquiring fixed and moving images, and segmenting them into multiple sub-region pairs, includes: Acquire training images; the training images include both stationary and moving images; A pre-trained SAM segmentation model is used to segment the training images into anatomical structures to obtain the semantic mask of the training images; The training images are unfolded along the slice direction and then a two-dimensional image sequence is synthesized using frame stitching technology. The two-dimensional image sequence is input into the SAM segmentation model, and the two-dimensional image segmentation result is obtained automatically. Based on the two-dimensional image segmentation results, the image is back-projected and reconstructed into three-dimensional sub-regions, and each three-dimensional sub-region has a corresponding semantic mask, so that the training image is segmented into multiple sub-regions. Region mapping technology is used to align corresponding sub-regions of fixed and moving images to form multiple sub-region pairs.
3. The image registration method based on the discontinuity assumption and segmentation-driven approach according to claim 1, characterized in that, The process of randomly selecting a set of anatomically consistent sub-region pairs as training objects, inputting them into the backbone registration network for training, and predicting and generating the sub-deformation fields of the corresponding sub-region pairs includes: Based on sub-region pairs of moving and stationary images, a set of sub-region pairs with anatomical consistency is randomly selected, and the pairs are stitched together along the channel dimension to serve as the input image. Based on the input image, it is input into the backbone registration network to extract image features and output a sub-deformation field. The process of extracting image features and outputting a sub-deformation field includes inputting the input image into the encoder module of the backbone registration network, extracting downsampled features of different scales using different trilinear downsampling layers, and passing the downsampled features of different scales to the trilinear upsampling layers of the corresponding scales in the deformation field combination module to extract upsampled features of different scales, and fusing them to generate image features and outputting a sub-deformation field.
4. The image registration method based on discontinuity assumption and segmentation-driven method according to claim 3, characterized in that, The method of jointly using multiple loss functions to optimize the medical image registration model includes: Voxel loss is calculated and obtained based on the voxel loss function. The mask loss is calculated based on the mask loss function. The regularization loss is calculated based on the regularization loss function. By combining voxel loss, mask loss, and regularization loss, a hybrid loss is constructed to optimize the medical image registration model.
5. The image registration method based on discontinuity assumption and segmentation-driven method according to claim 4, characterized in that, The calculation of the mask loss based on the mask loss function includes: Based on the semantic masks of the original fixed image and the moving image, they are converted into fixed EDT masks and moving EDT masks, respectively. Based on the fixed EDT mask, and combined with the deformation function of each sub-region from the moving image to the fixed image, a composite moving EDT mask is formed. By combining the overall moving EDT mask, the fixed EDT mask, and the deformation functions corresponding to each sub-region, the overall EDT loss function of the region is calculated.
6. A system for the image registration method based on discontinuity assumption and segmentation-driven approach as described in claim 1, characterized in that, include: The image acquisition and preprocessing module is used to acquire fixed and moving images and segment them into multiple sub-region pairs. The model training and optimization module is used to select fixed and moving images with sub-region pairs to stitch together to form an input image. It uses a backbone registration network to predict and generate sub-deformation fields for each sub-region pair, and integrates these sub-deformation fields to generate a complete deformation field, thus constructing and optimizing the medical image registration model. The backbone registration network includes an encoder module, a field estimation module, and a deformation field combination module. The encoder module extracts input image features, the field estimation module performs input image feature transformation and low-resolution displacement field prediction, and the deformation field combination module transforms and integrates these features to generate sub-deformation fields. The image registration module is used to apply the original moving image to the optimized medical image registration model, generate the registered distorted image, and complete the registration of the moving image and the fixed image.
7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.