A multi-modal head image registration method based on stable markers
By employing a dual anatomical landmark cascade strategy involving the ventricles and skull, the robustness and accuracy issues of multimodal head image registration in cross-modal scenarios were resolved. This resulted in highly automated and robust image alignment, improving the accuracy of clinical diagnosis and image analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUILIN UNIV OF ELECTRONIC TECH
- Filing Date
- 2026-03-10
- Publication Date
- 2026-05-29
AI Technical Summary
Existing multimodal head image registration methods lack robustness in cross-modal scenarios and struggle to achieve accurate alignment, especially in cases of complex head structures where rigid skulls and non-rigid brain tissues coexist. Furthermore, existing marker-based methods are cumbersome to operate and have poor repeatability.
A dual anatomical landmark cascade strategy involving the ventricles and skull was adopted. The cavum septum pellucidum and the frontal horn of the lateral ventricles in the head were used as first-level landmarks for local registration, while the frontal pole, occipital pole and longitudinal fissure of the skull were used as second-level landmarks for refined global registration. Quantitative evaluation was carried out by combining multiple accuracy assessment indicators.
It achieves highly automated, robust, and reliable multimodal head image registration, which can make full use of morphologically stable and high-contrast feature structures in cross-modal imaging to achieve coarse-to-fine spatial alignment, thereby improving the accuracy of clinical diagnosis and image analysis.
Smart Images
Figure CN122115520A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, specifically to a multimodal head image cascade registration method based on stable anatomical markers. Background Technology
[0002] Medical image registration is one of the core technologies in medical image analysis. Its goal is to align medical images from different imaging devices, at different times, or in different modalities into a unified coordinate system to achieve precise correspondence of anatomical structures. Multimodal image registration has significant application value in clinical diagnosis, surgical navigation, efficacy evaluation, and radiomics research, providing physicians with complementary anatomical and functional information and improving the accuracy of diagnostic and treatment decisions.
[0003] Head medical image registration is a crucial task in neuroimaging. Computed tomography (CT) has excellent bone tissue imaging capabilities, offering unique advantages in displaying skull structures, calcifications, and hemorrhages; magnetic resonance imaging (MRI) provides superior soft tissue contrast, clearly displaying brain parenchyma, the ventricular system, and lesions. In clinical practice, it is often necessary to fuse and analyze head images from multiple modalities, such as CT and MRI, to comprehensively utilize the imaging advantages of each modality.
[0004] However, multimodal head image registration faces numerous technical challenges. The grayscale features of images from different imaging modalities differ significantly, and traditional registration methods based on grayscale similarity (such as mutual information methods) lack robustness in cross-modal scenarios, easily getting trapped in local optima. Secondly, the head structure is complex, with rigid skull and non-rigid brain tissue coexisting; a single global rigid transformation cannot simultaneously guarantee the alignment accuracy of the skull and internal brain structures. Differences in patient position, scanning parameters, and imaging timing introduce additional spatial biases, further increasing the difficulty of registration.
[0005] Marker-based registration methods establish spatial correspondences by utilizing stable and identifiable feature points in anatomical structures, effectively overcoming registration difficulties caused by grayscale differences and exhibiting strong cross-modal adaptability and interpretability. However, existing marker-based registration methods often rely on manual point selection, which is cumbersome and has poor repeatability; at the same time, single-level marker registration struggles to balance fine local alignment with global consistency.
[0006] Therefore, there is an urgent need to develop a highly automated, robust, and accurate multimodal head image registration method that can fully utilize the morphologically stable and high-contrast features of the head anatomy in cross-modal imaging, and achieve coarse-to-fine spatial alignment through a hierarchical registration strategy, providing high-quality registration results for clinical diagnosis and image analysis. Summary of the Invention
[0007] To achieve the above objectives, this invention provides a multimodal head image registration method based on stable markers. It employs a dual anatomical marker cascade strategy involving the ventricles and skull to achieve accurate registration of multimodal head images, comprising the following steps:
[0008] S1. Obtain the multimodal head image dataset from local storage;
[0009] S2. Preprocess the data in the multimodal head image dataset;
[0010] S3. Use the cavum septum pellucidum and the frontal horn of the lateral ventricle as primary landmarks;
[0011] S4. Perform local registration based on the ventricle landmarks in the head, and apply the transformation matrix obtained from the local ventricle registration to the entire head image to achieve primary global registration;
[0012] S5. Use the frontal pole, occipital pole and longitudinal fissure of the brain in the skull as secondary landmarks;
[0013] S6. Perform local registration based on skull landmarks, and apply the transformation matrix obtained from the local skull registration to the entire head image to achieve refined global registration.
[0014] S7. Calculate the accuracy evaluation index of multimodal registration and quantitatively evaluate the registration results;
[0015] Furthermore, the specific method of step S1 is as follows:
[0016] S11: The acquired multimodal head image dataset contains multiple human head data, each containing multiple modal sequences such as CT, MRI T1, MRI T1C, and MRI T2, as well as delineated ventricles and skull labels;
[0017] S12: Each modal sequence exists in the folder as a DICOM sequence, and the tag is saved in .nii.gz format.
[0018] Furthermore, the specific method of step S2 is as follows:
[0019] S21: Based on the storage method of the multimodal head image dataset, create images and labels folders in sequence to store multimodal head image data and corresponding anatomical structure labels;
[0020] S22: Create a points folder under the labels folder to store the marker data. Each layer of the image is saved in .txt format.
[0021] S23: Create fixed, moving, and moved folders in sequence within each folder to store fixed images, floating images, and registered image data;
[0022] S24: Perform resampling, normalization, and format unification operations on the multimodal head image data to achieve data preprocessing;
[0023] Furthermore, the specific method for step S3 is as follows:
[0024] S31: The morphological changes of the head ventricular system are relatively small under different modalities and at the same time. Among them, the cavum septum pellucidum and the frontal horn of the lateral ventricle are stable anatomical structures within the ventricle and have clear imaging features in multiple modalities such as CT and MRI, making them suitable as markers for cross-modal registration. Landmarks for the cavum septum pellucidum are established as the apex of the cavum septum pellucidum (CSPA) and the base of the cavum septum pellucidum (CSPB); landmarks for the frontal horns of the lateral ventricles are established as the apex of the superior wall of the left ventricular frontal horn (LFH_SWA), the apex of the superior wall of the right ventricular frontal horn (RFH_SWA), the apex of the inferior wall of the left ventricular frontal horn (LFH_IWA), and the apex of the inferior wall of the right ventricular frontal horn (RFH_IWA).
[0025] S32: For the fixed and floating images in the registration process, the marker points are named separately to distinguish between the different images. In the fixed image, the six points are named F_CSPA, F_CSPB, F_LFH_SWA, F_RFH_SWA, F_LFH_IWA, and F_RFH_IWA; in the floating image, the six points are named M_CSPA, M_CSPB, M_LFH_SWA, M_RFH_SWA, M_LFH_IWA, and M_RFH_IWA.
[0026] Furthermore, the specific method for step S4 is as follows:
[0027] S41: On the pre-processed fixed and floating images, identify and extract the three-dimensional coordinates of all landmarks on the ventricles of the patient's head. These landmarks are clearly visible in both imaging modalities, and their relative positions are fixed in the patient coordinate system.
[0028] S42: After extraction, two sets of corresponding 3D point sets are obtained. The point set in the fixed image is F_P = {F_CSPA, F_CSPB, F_LFH_SWA, F_RFH_SWA, F_LFH_IWA, F_RFH_IWA}, and the corresponding point set in the floating image is M_P = {M_CSPA, M_CSPB, M_LFH_SWA, M_RFH_SWA, M_LFH_IWA, M_RFH_IWA}.
[0029] S43: Based on these two sets of corresponding marker points, calculate the optimal rigid body transformation, which consists of a rotation transformation matrix R and a translation matrix T. The goal of the calculation is to find a set of R and T such that when the transformation is applied to the point set of the floating image, the difference between it and the point set of the fixed image is minimized.
[0030] S44: The specific steps for calculating the translation matrix are as follows:
[0031] Calculate the centroid of a fixed set of image markers centroid of the floating image marker set To establish the center positions of two point sets in their respective spaces, the coordinate difference between these two centroids directly defines the translation component between the two spaces. The formula for calculating the centroid is as follows:
[0032]
[0033] In the formula, It is the centroid of the set of marker points. It is the number of marker points. These are the corresponding marker points.
[0034] The formula for calculating the translation component is as follows:
[0035]
[0036] In the formula, , , It is the translation component in a three-dimensional coordinate system. , , These are the coordinates of the centroid of a fixed image in different dimensions of a three-dimensional coordinate system. , , These are the coordinates of the centroid of the floating image in different dimensions of the three-dimensional coordinate system.
[0037] In order to perform unified matrix multiplication with subsequent rotation transformation matrices, this three-dimensional translation component needs to be... A translation matrix T in homogeneous coordinates is constructed. This is done by placing the vector into the first three elements of the fourth column of a 4x4 identity matrix, thus obtaining the final translation matrix. The formula for calculating the translation matrix is as follows:
[0038]
[0039] S45: Apply the calculated rigid body transformation matrix to the floating image. This process is first reflected in the marker points, that is, the set of marker points in the floating image. By performing a dot product operation with the transformation matrix, local registration of the marker point set can be achieved.
[0040] S46: Extend the spatial transformation relationship determined by this local registration to the entire 3D head image. The image registration algorithm will calculate the new coordinates of each voxel in the floating image in the fixed image space according to this transformation, and determine the pixel value at that position through the interpolation algorithm, thereby completing the alignment of the entire head image.
[0041] S47: After global alignment is completed, the nasopharyngeal carcinoma tumor regions in the floating image, which are drawn by doctors or segmented by algorithms, are also accurately mapped and located in the unified coordinate system of the fixed image, laying a spatial foundation for subsequent target area comparison or treatment plan overlay.
[0042] Furthermore, the specific method of step S5 is as follows:
[0043] S51: On the fixed and floating images that have been resampled and normalized, the three-dimensional coordinates of all landmarks on the patient's skull are identified and extracted. These landmarks are clearly visible in both imaging modalities, and their relative positions are fixed in the patient coordinate system.
[0044] S52: After extraction, two sets of corresponding 3D point sets are obtained. The point set in the fixed image is F_P = {F_FP, F_OP, O}, and the corresponding point set in the floating image is M_P = {M_FP, M_OP, P}.
[0045] Furthermore, the specific method of step S6 is as follows:
[0046] S61: After obtaining the skull landmark point sets of the fixed image and the floating image, construct the spatial correspondence between the two sets of points. This correspondence is based on the consistency of anatomical structure. Each landmark point in the skull has rigid properties in the patient coordinate system, that is, its relative position remains unchanged in different imaging modalities. By establishing a set of one-to-one mapped point pairs, the input constraints required for registration can be formed, providing a geometric basis for subsequent transformation parameter estimation.
[0047] S62: In the set of marker points in the fixed and floating images, points O and P are the midpoints of the lines F_FP / F_OP and M_FP / M_OP, respectively, and their calculation formulas are as follows:
[0048]
[0049]
[0050]
[0051] S63: By calculating a fixed set of image markers The center positions of two point sets in their respective spaces are determined by a point P and a floating image marker point set. The coordinate difference between these two points directly defines the translation component between the two spaces. The formula for calculating the translation component is as follows:
[0052]
[0053] S64: After obtaining the corresponding translation components, construct the corresponding translation transformation matrix using the components; the formula for calculating the translation transformation matrix is as follows:
[0054]
[0055] S65: Apply the transformation matrix to the floating image, first verify the local registration with the marker points, then map the transformation to the whole head volume voxel by voxel, and complete the overall alignment by interpolation and resampling;
[0056] S66: Nasopharyngeal carcinoma target areas manually or automatically delineated in floating images are projected onto a fixed image coordinate system at once, providing a unified spatial reference for subsequent dose stacking and contour comparison.
[0057] Furthermore, the specific method for step S7 is as follows:
[0058] S71: After registration with multiple anatomical markers, observe whether the various registration indicators are normal. The evaluation indicators include, but are not limited to, DSC and HD. 95 And IOU; the calculation formula for the evaluation index is as follows:
[0059]
[0060] In the formula, F represents the size of the marker in the fixed image, and M represents the size of the marker in the floating image.
[0061]
[0062] In the formula, sup represents the upper limit; inf represents the lower limit; F represents the volume of the marker in the fixed image; and M represents the volume of the marker in the floating image.
[0063]
[0064] In the formula, F represents the size of the marker in the fixed image, and M represents the size of the marker in the floating image.
[0065] S72: For images registered via the ventricles of the head, relevant evaluation indicators are calculated using the skull for verification; for images registered via the ventricles of the head and skull, relevant evaluation indicators are calculated using the ventricles of the head for verification. The registration effect is verified through cross-comparison, thereby designing the clinical target volume for radiotherapy. Attached Figure Description
[0066] Figure 1 The flowchart of the method of the present invention
[0067] Figure 2 This is a flowchart of the head ventricle registration process in this invention.
[0068] Figure 3 This is a flowchart of the skull registration process in this invention.
[0069] Figure 4 This is a diagram showing the overall registration effect in this invention.
[0070] Figure 5 This is a diagram illustrating the registration of the ventricles and skull in this invention. Detailed Implementation
[0071] The multimodal head image registration method based on stable markers described in this invention is implemented according to the following steps:
[0072] Step 1: Obtain a multimodal head image dataset, which includes multiple human head data and their corresponding ventricles and skull labels, stored in .nii.gz file format;
[0073] Step 2: Read in different modal images in DICOM sequence format, such as CT and MRI T2, and convert them to .nii.gz format for saving;
[0074] Step 3: Perform data preprocessing on images of different modalities, and create images and labels folders to store multimodal head image data and corresponding anatomical structure labels;
[0075] Step 4: Read the preprocessed image data of different modalities, as well as the corresponding head ventricles and skull labels and key point coordinates, and perform cascade registration after designing the relevant parameters;
[0076] Step 5: After primary registration of the ventricles and refined registration of the skull, the corresponding evaluation indicators and data are generated and placed in the corresponding result folder;
[0077] Step six: Based on the spatial alignment relationship after registration, the region of interest (such as lesion or anatomical structure) delineated in any modality can be mapped to other modalities to achieve cross-modal structural localization and fusion analysis;
[0078] Step 7: After registration is completed, the registration results and evaluation indicators can be organized into an analysis report according to a fixed template, which can be used to evaluate the registration quality and guide subsequent image analysis work.
[0079] Step 8: After registration is complete, the registration verification results of the skull and ventricles, deformation field information, and related coordinate information need to be organized into a Chinese text according to a fixed template. Then, add the sentence "Please analyze the following nasopharyngeal carcinoma tumor registration results and provide suggestions" at the beginning of the text to form a complete prompt. Next, you need to embed this complete prompt into a ChatGPT API request. The specific steps are: 1. First, ensure that the OpenAI Python library is installed, and then set the API key in the code; 2. Construct a request message containing your prompt. This message should be a dictionary list containing roles and content. Set the role to "user," and the content is the complete text you just assembled; then call the ChatGPT API to send this request. After the API returns a response, you need to extract the analysis and suggestion text generated by ChatGPT from the response result. Finally, set this text to a QTextEdit control and display the content by calling the QTextEdit's setText method to obtain the final tumor information reference report.
[0080] The examples above are merely illustrative of embodiments of the present invention and should not be construed as limiting the invention. Within the scope of the claims, any modifications, equivalent substitutions, and improvements that conform to the core spirit and principles of the present invention should be considered as included within the protection scope of the present invention.
Claims
1. A multimodal head image registration method based on stable markers, characterized in that, Includes the following steps: S1. Obtain the multimodal head image dataset from local storage; S2. Preprocess the data in the multimodal head image dataset; S3. Use the cavum septum pellucidum and the frontal horn of the lateral ventricle as primary landmarks; S4. Perform local registration based on the ventricle landmarks in the head, and apply the transformation matrix obtained from the local ventricle registration to the entire head image to achieve primary global registration; S5. Use the frontal pole, occipital pole and longitudinal fissure of the brain in the skull as secondary landmarks; S6. Perform local registration based on skull landmarks, and apply the transformation matrix obtained from the local skull registration to the entire head image to achieve refined global registration. S7. Calculate the accuracy evaluation index of multimodal registration and quantitatively evaluate the registration results.
2. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, In step S1, the obtained multimodal head image dataset contains multiple human head data sets. Each data set contains multiple modal sequences such as CT, MRI T1, MRI T1C, and MRI T2, as well as delineated ventricles and skull labels. Each modal sequence is stored in DICOM format, and the labels are saved in .nii.gz format.
3. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, Step S2 involves preprocessing the data in the multimodal head image dataset, including: S21. Create images and labels folders to store multimodal head image data and corresponding anatomical structure labels; S22. Create a folder named "points" under the "labels" folder to store the marker point data; S23. Create folders named fixed, moving, and moved to store fixed images, floating images, and registered image data; S24. Perform resampling, normalization, and format unification operations on the multimodal head image data.
4. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, In step S3, the cavum septum pellucidum and the frontal horn of the lateral ventricle are used as first-level landmarks, which include: S31. Vertex of the superior frontal horn of the left ventricle (LFH_SWA), vertex of the superior frontal horn of the right ventricle (RFH_SWA), vertex of the inferior frontal horn of the left ventricle (LFH_IWA), vertex of the inferior frontal horn of the right ventricle (RFH_IWA), vertex of the cavum septum pellucidum (CSPA), and basal point of the cavum septum pellucidum (CSPB); S32. For fixed and floating images, name the six points in the fixed image as: F_LFH_SWA, F_RFH_SWA, F_LFH_IWA, F_RFH_IWA, F_CSPA, F_CSPB; and name the six points in the floating image as: M_LFH_SWA, M_RFH_SWA, M_LFH_IWA, M_RFH_IWA, M_CSPA, M_CSPB.
5. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, In step S4, the ventricle landmarks of the fixed image and the floating image are calculated to obtain the corresponding translation matrix T and rotation transformation matrix R. The landmarks and transformation matrices are multiplied by a dot to achieve local registration. The global head image is aligned according to the corresponding coordinates of the landmarks after local registration to complete the primary global registration.
6. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, In step S5, the frontal pole, occipital pole, and longitudinal fissure of the brain in the skull are used as secondary landmarks, which include: S51. Frontal pole (FP), occipital pole (OP), and longitudinal fissure of the brain in the skull; S52. For fixed and floating images, name the three points in the fixed image as F_FP, F_OP and O; name the three points in the floating image as M_FP, M_OP and P.
7. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, In step S6, the skull landmarks of the fixed image and the floating image are calculated to obtain the corresponding translation matrix T and rotation transformation matrix R. The landmarks and transformation matrices are multiplied by a dot to achieve local registration. Based on the corresponding coordinate positions of the landmarks after local registration, the global head image is refined and aligned to complete the refined global registration.
8. The multimodal head image registration method based on stable markers according to claim 1, characterized in that, In step S7, after cascaded registration of multiple anatomical markers, the registration accuracy evaluation index is calculated. The evaluation index includes, but is not limited to, DSC (Dice Similarity Coefficient), HD95 (95th Percentile Hausdorff Distance), and IOU (Intersection over Union).