RigNet-based medical image rigid registration method
By using a RigNet-based rigid registration method for medical images, rigid registration of medical images is automatically completed, solving the problems of low efficiency and insufficient accuracy of traditional methods, and achieving efficient and accurate image alignment and information fusion.
Patent Information
- Application Number
- CN202411207215.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional rigid registration methods for medical images require manual intervention, which is inefficient and prone to errors.
A RigNet-based rigid registration method for medical images is adopted, which includes data preprocessing, RigNet model training, rigid transformation and resampling, and post-processing steps. It automatically completes the rigid registration of medical images and uses the RigNet model to generate displacement and rotation parameters for image alignment.
It improves registration efficiency and accuracy, and realizes automated rigid registration of medical images without manual intervention, with high generalization ability and good fusion effect.
Smart Images

Figure CN119006547A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image rigid registration method based on RigNet. Background Art
[0002] In medical diagnosis, medical images of different modalities (such as CT, MRI, PET, etc.) are usually used to obtain different aspects of information about the patient's body. However, medical images obtained by patients at different time points or with different imaging devices often have inconsistencies in spatial position. In order to make comprehensive use of this information, these images need to be aligned in space, that is, rigid registration. Traditional registration methods usually require manual intervention, are inefficient and prone to errors. Therefore, developing a medical image rigid registration method based on RigNet has important application value. Summary of the invention
[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a medical image rigid registration method based on RigNet, which does not require manual intervention and improves the registration efficiency and accuracy.
[0004] The present invention provides the following technical solution: a medical image rigid registration method based on RigNet, characterized in that it comprises the following steps:
[0005] S1, data preprocessing module: preprocess the input medical images, including bone material extraction, spatial random sampling, image contrast enhancement, cropping, and normalization operations;
[0006] S2, RigNet model training: The RigNet model is trained using paired medical image data. The training model can generate displacement and rotation parameters of floating images and fixed images.
[0007] S3, rigid transformation and resampling: Based on the output parameters of the trained RigidNet model, the preprocessed medical images are rigidly transformed and resampled. Rigid transformation refers to linear transformation of the input image based on the deformation information of the output grid to align it with the fixed image in spatial position; resampling refers to interpolation or resampling of the original image based on the transformed image coordinates to generate a registered image with the same resolution and size as the fixed image;
[0008] S4, post-processing module: post-processing the deformed medical images, including smoothing and denoising operations.
[0009] Preferably, during the bone material extraction process in S1, the data preprocessing module performs binarization processing on the medical image according to the HU value of the CT image, and uses corrosion operation according to the size of the outlier area to corrode the outlier area, retain the largest connected domain in the three-dimensional space, filter out small three-dimensional connected bodies, and only retain the main part of the bone. The cavity caused by the bone marrow is filled by a closing operation, the confidence area is increased, and the automatic extraction of the human bone mask is completed.
[0010] Preferably, the spatial random sampling of the data preprocessing module adopts random slicing in the Z-axis direction in the spatial coordinate system, and the starting position of the slice is used as the starting position of the training image. The fixed image and the floating image use the same starting position. From the starting position to the end position, the length is equal to the Z-axis direction scale of the actual training setting. Finally, the image is filled. If the end position of the spatial slice is out of bounds, a blank layer is filled upwards, and the pixel value of the blank layer adopts the HU value of air.
[0011] Preferably, in the image contrast enhancement process in S1, for CBCT / CT images, a range truncation method is used to limit the HU value to between a given minimum value and a given maximum value.
[0012] Preferably, the training process of the medical imaging data in S2 is as follows:
[0013] ① Data collection and preprocessing: Collect medical imaging data from different hospitals, different modalities and different anatomical parts, and preprocess the original medical imaging data through batch bone material extraction, contrast enhancement, cropping and filling, and normalization steps;
[0014] ② Multimodal data fusion: First, multimodal medical imaging data (such as CT, MRI, CBCT) is used for fusion processing, and the image information of different modalities is integrated through appropriate fusion algorithms to extract richer features and improve the model's ability to recognize the target structure;
[0015] ③. Model selection and optimization: According to the task requirements and data characteristics, RigNet based on convolutional neural network is selected as the training basis. During the training process, the model performance is optimized by adjusting the model's hyperparameters and optimization algorithms;
[0016] ④. Transfer learning and fine-tuning: Using the transfer learning method, the model pre-trained on a large dataset is initialized, and then fine-tuned on the target dataset to adapt to the needs of a specific task;
[0017] ⑤. Loss function and evaluation indicators: Select appropriate loss functions (such as Dice loss, NMI loss) and evaluation indicators (such as MI mutual information measure, recall rate, registration time), evaluate the performance of the model regularly, and adjust the training strategy according to the evaluation results to continuously optimize the model.
[0018] Preferably, the RigNet model includes the following modules:
[0019] Convolution module: contains 4 convolution layers, which are mainly used to extract the spatial features of the input data. By increasing the number of layers, more complex features are gradually extracted, while reducing the spatial dimension and increasing the nonlinear expression ability.
[0020] Fully connected module: contains three fully connected layers, which are mainly used to map the features extracted by the convolution layer to the final output, and gradually reduce the dimension and increase the nonlinear transformation by increasing the number of layers;
[0021] Downsampling module: Using the ResNet architecture, the input of each layer is the sum of the convolution and downsampling of the upper layer. This module is designed to improve the scalability of the model and enable the model to converge faster. The model training uses Dice loss and normalized mutual information loss similarity measurement loss.
[0022] Preferably, the training of the model uses Dice loss to evaluate the accuracy of the registration. In theory, the more aligned the two are, the higher the Dice is and the lower the DiceLoss is. The formula of DiceLoss is as follows: Where A represents a fixed image, B represents a floating image, |A∩B| represents the intersection of the fixed image and the floating image, and |A| and |B| represent the sizes of the fixed image and the floating image, respectively.
[0023] Preferably, the model is trained using normalized mutual information loss to measure the similarity between two images. NMI evaluates the amount of common information between images by comparing the joint histogram of the images with their respective edge histograms. It is highly robust to the registration of different image modalities (such as CT and MRI) because it measures the statistical dependence between images rather than direct pixel values. Its formula is as follows: Where I(A; B) represents the mutual information between image A and image B, and H(A) and H(B) represent the entropy of image A and image B respectively.
[0024] Compared with the prior art, the present invention has the following beneficial effects:
[0025] High degree of automation: Through the trained RigNet model, the system can automatically complete the rigid registration of medical images without manual intervention, greatly improving the efficiency and accuracy of registration.
[0026] Strong generalization ability: The RigNet model used in the present invention has strong generalization ability and can adapt to medical imaging data of different parts and different patients to achieve a wide range of registration applications.
[0027] Good fusion effect: Through rigid registration and image fusion technology, the present invention can fuse different medical imaging information together to provide doctors with more comprehensive and accurate diagnostic support.
[0028] Easy to implement and promote: The algorithms and technologies used in this invention are based on mature deep learning frameworks and image processing libraries, which are easy to implement and promote. At the same time, the system can be easily integrated into existing medical image processing software to provide doctors with more convenient and efficient services. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 This is a diagram of a fully automated rigid registration system for medical images of the present invention;
[0030] Figure 2 The flowchart of the rigid registration training of the bone mask image of the present invention;
[0031] Figure 3 This is the RigNet network model topology of the present invention. DETAILED DESCRIPTION
[0032] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of known functions and known components to avoid unnecessary confusion of the concepts of the present invention.
[0033] In a first aspect, the present invention provides a fully automated rigid registration system for medical images, the implementation steps of which are shown in FIG. Figure 1 , the system mainly includes the following steps:
[0034] Data preprocessing: Preprocess the input medical images, including bone material extraction, spatial random sampling, denoising, smoothing, normalization and other operations to improve image quality and consistency.
[0035] RigNet model training: The RigNet model is trained using unpaired medical imaging data. RigNet is an original neural network model with basic convolution and full connection structure, which can learn the position deviation from floating image to fixed image. Through training, the model can generate displacement and rotation parameters of floating image and fixed image, so as to achieve rigid alignment between different images.
[0036] Rigid transformation and resampling: Based on the output parameters of the trained RigidNet model, the preprocessed medical images are rigidly transformed and resampled. Rigid transformation refers to linear transformation of the input image based on the deformation information of the output grid so that it is aligned with the fixed image in spatial position. Resampling refers to interpolation or resampling of the original image based on the transformed image coordinates to generate a registered image with the same resolution and size as the fixed image.
[0037] Post-processing: Post-process the deformed medical images, including smoothing, denoising and other operations, to further improve the quality and consistency of the registered images.
[0038] In a second aspect, the present invention provides a method for extracting bone material, the implementation steps comprising:
[0039] (1) According to the HU value of the image, the medical image is binarized.
[0040] (2) According to the size of the outlier region, the corrosion operation is used to corrode the outlier region.
[0041] (3) Retain the largest connected domain in the three-dimensional space, filter out small three-dimensional connected bodies, and only retain the main part of the skeleton.
[0042] (4) Fill the voids caused by the bone marrow through closing operations to increase the confidence region.
[0043] Through the above steps, the system can automatically extract the human skeleton mask, so that the important features in the registration process can be highlighted.
[0044] In the process of image contrast enhancement, for CBCT / CT images, the range truncation method is used to limit the HU value between the given minimum and maximum values.
[0045] In a third aspect, the present invention provides a method for spatial random sampling, the implementation steps comprising:
[0046] (1) Spatial random sampling in the data preprocessing module uses random slicing in the Z-axis direction of the spatial coordinate system. Rigid registration is highly sensitive to the spatial position of the image. Random slicing position positioning. Take the minimum value of the number of layers of the fixed image and the floating image, and half of the minimum value as the maximum starting range of the slicing position. Randomly select a value within this range as the starting position of the slicing.
[0047] (2) Spatial slicing of the image, with the slice start position as the start position of the training image, and the fixed image and the floating image use the same start position. From the start position to the end position, its length is equal to the Z-axis scale of the actual training setting.
[0048] (3) Image filling processing: If the end point of the spatial slice is out of bounds, a blank layer is filled upwards, and the pixel value of the blank layer adopts the HU value of air.
[0049] Through the above steps, the system uses spatial random sampling to ensure data diversity. It has been verified that this method can enhance the generalization ability of the model during training.
[0050] In a fourth aspect, the present invention provides a training process for medical image data as follows:
[0051] ①. Data collection and preprocessing: Collect medical imaging data from different hospitals, different modalities and different anatomical parts, and preprocess the original medical imaging data through batch bone material extraction, contrast enhancement, cropping and filling, and normalization steps.
[0052] ② Multimodal data fusion: First, multimodal medical imaging data (such as CT, MRI, CBCT) is used for fusion processing, and the image information of different modalities is integrated through appropriate fusion algorithms to extract richer features and improve the model's ability to recognize the target structure.
[0053] ③. Model selection and optimization: According to the task requirements and data characteristics, RigNet based on convolutional neural network is selected as the training basis. During the training process, the model performance is optimized by adjusting the model's hyperparameters and optimization algorithms.
[0054] ④. Transfer learning and fine-tuning: Using the transfer learning method, the model pre-trained on a large dataset is initialized, and then fine-tuned on the target dataset to adapt to the needs of specific tasks.
[0055] ⑤. Loss function and evaluation indicators: Select appropriate loss functions (such as Dice loss, NMI loss) and evaluation indicators (such as MI mutual information measure, recall rate, registration time), evaluate the performance of the model regularly, and adjust the training strategy according to the evaluation results to continuously optimize the model.
[0056] In a fifth aspect, the present invention provides a RigNet network model for rigid registration of medical images (eg Figure 3As shown). The RigNet network model is characterized in that the RigNet model includes a convolution module, a fully connected module, and a downsampling module, wherein the convolution module includes 4 convolution layers, which are mainly used to extract the spatial features of the input data, and gradually extract more complex features by increasing the number of layers, while reducing the spatial dimension and increasing the nonlinear expression ability. The fully connected module includes three fully connected layers, which are mainly used to map the features extracted by the convolution layer to the final output, and gradually reduce the dimension and increase the nonlinear transformation by increasing the number of layers, thereby improving the regression effect. The downsampling module draws on the idea of ResNet, and the input of each layer is the sum of the convolution and downsampling of the upper layer. The design of this module is mainly to improve the scalability of the model and enable the model to converge faster. The model training adopts Dice loss and normalized similarity measurement loss to improve the registration accuracy.
[0057] The choice of the target loss function determines the result of network training. The target loss function of the entire network model includes the following two items: Dices loss and normalized mutual information loss.
[0058] The purpose of Dice loss is to align images in space. The overlap ratio of the fixed image and the floating image is calculated to determine whether the image is aligned in space. Dice is one of the most common indicators in the field of segmentation. The registration implementation of the present invention relies on the binary image of the image bone material mask, and the image differences caused by different modalities and different devices can be ignored. Therefore, the present invention can use Dice to evaluate the accuracy of registration. In theory, the more aligned the two are, the higher the Dice is, and the lower the DiceLoss is. The formula of DiceLoss is as follows:
[0059]
[0060] Where A represents a fixed image, B represents a floating image, |A∩B| represents the intersection of the fixed image and the floating image, and |A| and |B| represent the sizes of the fixed image and the floating image, respectively.
[0061] The standard mutual information loss function is a metric based on information theory that is used to measure the similarity between two images. NMI evaluates the amount of common information between images by comparing the joint histogram of the images with their respective edge histograms. It is highly robust to the registration of different image modalities (such as CT and MRI) because it measures the statistical dependence between images rather than direct pixel values. Its formula is as follows:
[0062]
[0063] Where I(A; B) represents the mutual information between image A and image B, and H(A) and H(B) represent the entropy of image A and image B respectively.
[0064] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.
Claims
1. A medical image rigid registration method based on RigNet, characterized in that: The following steps are involved: S1, data preprocessing module: preprocess the input medical images, including bone material extraction, spatial random sampling, image contrast enhancement, cropping, and normalization operations; S2, RigNet model training: The RigNet model is trained using paired medical image data. The training model can generate displacement and rotation parameters of floating images and fixed images. S3, rigid transformation and resampling: Based on the output parameters of the trained RigidNet model, the preprocessed medical images are rigidly transformed and resampled. Rigid transformation refers to linear transformation of the input image based on the deformation information of the output grid to align it with the fixed image in spatial position; resampling refers to interpolation or resampling of the original image based on the transformed image coordinates to generate a registered image with the same resolution and size as the fixed image; S4, post-processing module: post-processing the deformed medical images, including smoothing and denoising operations.
2. The medical image rigid registration method based on RigNet according to claim 1, characterized in that: During the bone material extraction process in S1, the data preprocessing module performs binarization processing on the medical image according to the HU value of the CT image, and uses corrosion operation according to the size of the outlier area to corrode the outlier area, retain the largest connected domain in the three-dimensional space, filter out small three-dimensional connected bodies, and only retain the main part of the bone. The cavity caused by the bone marrow is filled by closing operation, the confidence area is increased, and the automatic extraction of the human bone mask is completed.
3. The medical image rigid registration method based on RigNet according to claim 1, characterized in that: The spatial random sampling of the data preprocessing module adopts random slicing in the Z-axis direction in the spatial coordinate system. The starting position of the slice is used as the starting position of the training image. The fixed image and the floating image use the same starting position. From the starting position to the end position, its length is equal to the Z-axis direction scale of the actual training setting. Finally, the image is filled. If the end position of the spatial slice is out of bounds, a blank layer is filled upwards. The pixel value of the blank layer adopts the HU value of air.
4. The medical image rigid registration method based on RigNet according to claim 1, characterized in that: In the image contrast enhancement process in S1, for CBCT / CT images, the range truncation method is used to limit the HU value between the given minimum and maximum values.
5. The medical image rigid registration method based on RigNet according to claim 1, characterized in that: The training process of medical imaging data in S2 is as follows: ① Data collection and preprocessing: Collect medical imaging data from different hospitals, different modalities and different anatomical parts, and preprocess the original medical imaging data through batch bone material extraction, contrast enhancement, cropping and filling, and normalization steps; ② Multimodal data fusion: First, multimodal medical imaging data (such as CT, MRI, CBCT) is used for fusion processing, and the image information of different modalities is integrated through appropriate fusion algorithms to extract richer features and improve the model's ability to recognize the target structure; ③. Model selection and optimization: According to the task requirements and data characteristics, RigNet based on convolutional neural network is selected as the training basis. During the training process, the model performance is optimized by adjusting the model's hyperparameters and optimization algorithms; ④. Transfer learning and fine-tuning: Using the transfer learning method, the model pre-trained on a large dataset is initialized, and then fine-tuned on the target dataset to adapt to the needs of a specific task; ⑤. Loss function and evaluation indicators: Select appropriate loss functions (such as Dice loss, NMI loss) and evaluation indicators (such as MI mutual information measure, recall rate, registration time), evaluate the performance of the model regularly, and adjust the training strategy according to the evaluation results to continuously optimize the model.
6. The medical image rigid registration method based on RigNet according to claim 1, characterized in that: The RigNet model includes the following modules: Convolution module: contains 4 convolution layers, which are mainly used to extract the spatial features of the input data. By increasing the number of layers, more complex features are gradually extracted, while reducing the spatial dimension and increasing the nonlinear expression ability. Fully connected module: contains three fully connected layers, which are mainly used to map the features extracted by the convolution layer to the final output, and gradually reduce the dimension and increase the nonlinear transformation by increasing the number of layers; Downsampling module: Using the ResNet architecture, the input of each layer is the sum of the convolution and downsampling of the upper layer. This module is designed to improve the scalability of the model and enable the model to converge faster. The model training uses Dice loss and normalized mutual information loss similarity measurement loss.
7. The medical image rigid registration method based on RigNet according to claim 6, characterized in that: The training of the model uses Dice loss to evaluate the accuracy of the registration. In theory, the more aligned the two are, the higher the Dice and the lower the DiceLoss. The formula of DiceLoss is as follows: Where A represents a fixed image, B represents a floating image, |A∩B| represents the intersection of the fixed image and the floating image, and |A| and |B| represent the sizes of the fixed image and the floating image, respectively.
8. The medical image rigid registration method based on RigNet according to claim 6, characterized in that: The model training uses normalized mutual information loss to measure the similarity between two images. NMI evaluates the common information between images by comparing the joint histogram of the images with their respective edge histograms. It is highly robust to the registration of different image modalities (such as CT and MRI) because it measures the statistical dependence between images rather than direct pixel values. The formula is as follows: Where I(A; B) represents the mutual information between image A and image B, and H(A) and H(B) represent the entropy of image A and image B respectively.
Citation Information
Cited By
Multi-modal medical image elastic registration system based on CyclGan
CN119295300A
Medical image report automatic generation method and device based on artificial intelligence
CN120932859A
