A multimodal image fusion algorithm for neurointerventional therapy

Abstract

The present invention proposes a multimodal image fusion algorithm for neurointerventional treatment, which specifically includes the following steps: S1, acquiring an MRI image, a left-side infrared thermal map, a right-side infrared thermal map, and a two-dimensional DSA image; S2, reconstructing a three-dimensional brain structure based on the MRI image; S3, performing a registration operation on the left-side infrared thermal map and the right-side infrared thermal map; S4, correcting the three-dimensional brain structure based on the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures; S5, screening the candidate three-dimensional brain structures based on the registered left-side infrared thermal map and the right-side infrared thermal map to determine the final three-dimensional brain structure. In the present invention, the intraoperative two-dimensional DSA image and the preoperative MRI image are fused with the assistance of the infrared thermal map to obtain an accurate three-dimensional brain structure, reflecting spatial information while accurately displaying vascular distribution.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to a multimodal image fusion algorithm applied to neurointerventional therapy. Background Art

[0002] Cerebrovascular diseases are characterized by high morbidity, disability, and mortality rates. To address this issue, neurointerventional therapy has emerged. Through the integrated application of multiple medical imaging information, neurointerventional therapy performs preoperative diagnosis, disease analysis, surgical path planning, intraoperative lesion location, real-time tracking of surgical instruments, and spatial adjustment of surgical instrument placement. It also uses catheters within cerebral vessels to achieve selective angiography, embolization, dilation, and other cerebrovascular disease treatments.

[0003] During neurointerventional treatment, limited imaging speed, radiation exposure, and other factors mean that only two-dimensional images can be used, preventing high-resolution three-dimensional scanning. Two-dimensional images have low resolution and cannot reflect spatial information. Doctors cannot make accurate judgments based on single-modality intraoperative images, which can affect treatment effectiveness. Summary of the Invention

[0004] To address the technical problem in the prior art that single-modality intraoperative images provide limited information, which affects the treatment effect, the multimodal image fusion algorithm for neurointerventional treatment proposed in the present invention specifically includes the following steps:

[0005] S1, acquiring MRI images, left lateral infrared thermal images, right lateral infrared thermal images, and two-dimensional DSA images;

[0006] S2. Reconstruct the three-dimensional structure of the brain based on MRI images;

[0007] S3. Performing a registration operation on the left side infrared thermal image and the right side infrared thermal image to obtain the registered left side infrared thermal image and the right side infrared thermal image;

[0008] S4. Correcting the three-dimensional brain structure based on the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures;

[0009] S5. Based on the registered left-side infrared thermal image and right-side infrared thermal image, candidate three-dimensional brain structures are screened to determine the final three-dimensional brain structure.

[0010] Preferably, in S1, the MRI image, the left side infrared thermal image and the right side infrared thermal image are acquired in the same body position before the operation, and the two-dimensional DSA image is acquired during the operation.

[0011] Preferably, in S3, the registration operation specifically includes:

[0012] S31. Virtually image the three-dimensional structure of the brain based on the pinhole imaging principle, and calculate and obtain a left-side two-dimensional plane projection image and a right-side two-dimensional plane projection image at the same acquisition angle as the left-side infrared thermal image and the right-side infrared thermal image;

[0013] S32, registering the left side infrared thermal image based on the left two-dimensional plane projection image to obtain a registered left side infrared thermal image;

[0014] S33. Register the right side infrared thermal image based on the right two-dimensional plane projection image to obtain the registered right side infrared thermal image.

[0015] Preferably, in S4, the specific process of correction is as follows:

[0016] S41, performing initial correction of the three-dimensional brain structure based on the vascular contours in the two-dimensional DSA image;

[0017] S42. Perform secondary correction on the three-dimensional brain structure based on the blood vessel size in the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures.

[0018] Preferably, in S41, the specific process of the initial calibration includes:

[0019] S411, rotating the three-dimensional brain structure according to the acquisition angle of the two-dimensional DSA image so that it is the same as the acquisition angle of the two-dimensional DSA image, and calculating the DSA two-dimensional plane projection of the three-dimensional brain structure;

[0020] S412, performing blood vessel segmentation on the DSA two-dimensional plane projection and the two-dimensional DSA image by an anisotropic diffusion method;

[0021] S413: Based on the landmark blood vessels, the DSA two-dimensional plane projection and the blood vessels in the two-dimensional DSA image are matched to obtain the corresponding blood vessels. By comparison, it is determined whether there are regions with different shapes in the corresponding blood vessels. If so, the process proceeds to S414; if not, the initial calibration ends.

[0022] S414, adjusting edge pixel distribution of blood vessels in the three-dimensional brain structure based on the shape-distinguishing region and a conventional brain blood vessel model;

[0023] S415 . Recalculate the DSA two-dimensional plane projection of the three-dimensional brain structure, and return to S412 .

[0024] Preferably, in S42, the specific process of the secondary correction includes:

[0025] S421, rotating the three-dimensional brain structure according to the acquisition angle of the two-dimensional DSA image so that it is the same as the acquisition angle of the two-dimensional DSA image, and calculating the DSA two-dimensional plane projection of the three-dimensional brain structure;

[0026] S422, performing blood vessel segmentation on the DSA two-dimensional plane projection and the two-dimensional DSA image by an anisotropic diffusion method;

[0027] S423, calculate size ratio based on landmark blood vessels;

[0028] S424. Based on the landmark blood vessels, the DSA two-dimensional plane projection and the blood vessels in the two-dimensional DSA image are matched to obtain unique blood vessels that only exist in the two-dimensional DSA image, and the width occupied by the unique blood vessels in the three-dimensional structure of the brain is calculated based on the size ratio.

[0029] Preferably, in S42, the anterior cerebral artery and the posterior cerebral artery are selected as landmark blood vessels.

[0030] Preferably, in S423, the average value of the ratio of the average width of the anterior cerebral artery to the average width of the posterior cerebral artery in the DSA two-dimensional plane projection and the two-dimensional DSA image is calculated as the size ratio.

[0031] Preferably, in S5, the specific process of screening is to calculate the matching coefficients of each candidate three-dimensional brain structure with the aligned left-side infrared thermal image and right-side infrared thermal image, and select the candidate three-dimensional brain structure with the highest matching coefficient as the final three-dimensional brain structure.

[0032] Preferably, the matching coefficient calculation process is to perform virtual imaging of the alternative brain three-dimensional structure based on the pinhole imaging principle, calculate and obtain the left alternative two-dimensional plane projection image and the right alternative two-dimensional plane projection image with the same acquisition angle as the aligned left side infrared thermal map and the right side infrared thermal map, binarize the left alternative two-dimensional plane projection image and the right alternative two-dimensional plane projection image by using the method of first expansion and then corrosion to obtain the corresponding binary image, and take the proportion of pixels with the same grayscale value between the binarized images of the aligned left side infrared thermal map and the left alternative two-dimensional plane projection image as the first similarity, and take the proportion of pixels with the same grayscale value between the binarized images of the aligned right side infrared thermal map and the right alternative two-dimensional plane projection image as the second similarity, and calculate the average of the first similarity and the second similarity as the matching coefficient.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] With the assistance of infrared thermal images, the intraoperative two-dimensional DSA images and preoperative MRI images are fused to obtain the precise three-dimensional structure of the brain, reflecting spatial information while accurately displaying the distribution of blood vessels. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the multimodal image fusion algorithm of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific implementation methods.

[0037] like Figure 1 As shown, the multimodal image fusion algorithm for neurointerventional therapy proposed in the present invention specifically includes the following steps:

[0038] S1. Obtain MRI images, left-side infrared thermal maps, right-side infrared thermal maps, and two-dimensional DSA images. The MRI images, left-side infrared thermal maps, and right-side infrared thermal maps are acquired in the same body position before surgery, and the two-dimensional DSA images are acquired during surgery.

[0039] S2. Reconstruct the three-dimensional structure of the brain based on MRI images.

[0040] S3. Perform a registration operation on the left side infrared thermal image and the right side infrared thermal image to obtain the registered left side infrared thermal image and the right side infrared thermal image. The registration operation specifically includes:

[0041] S31. Virtually image the three-dimensional structure of the brain based on the pinhole imaging principle, and calculate and obtain a left-side two-dimensional plane projection image and a right-side two-dimensional plane projection image with the same acquisition angle as the left-side infrared thermal image and the right-side infrared thermal image.

[0042] S32. The left side infrared thermal map is aligned based on the left two-dimensional plane projection map to obtain the aligned left side infrared thermal map. Specifically, the left side infrared thermal map and the left two-dimensional plane projection map are binarized by a method of first dilation and then corrosion to obtain a corresponding binary image. The obtained binary image is normalized, and a minimum value ε is accumulated for areas where pixels of 0 exist in the normalized image. The optimal transmission theory method is used to align the images that have undergone the above preprocessing to obtain the aligned left side infrared thermal map.

[0043] S33. The right side infrared thermal map is aligned based on the right two-dimensional plane projection image to obtain the aligned right side infrared thermal map. Specifically, the right side infrared thermal map and the right two-dimensional plane projection image are binarized by a method of first dilation and then corrosion to obtain a corresponding binary image. The obtained binary image is normalized, and a minimum value ε is accumulated for areas where pixels of 0 exist in the normalized image. The optimal transmission theory method is used to align the images that have undergone the above preprocessing to obtain the aligned right side infrared thermal map.

[0044] S4. Correct the three-dimensional brain structure based on the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures. The specific process of correction is as follows:

[0045] S41. Performing an initial correction of the three-dimensional brain structure based on the vascular contours in the two-dimensional DSA image. The specific process of the initial correction includes:

[0046] S411. Rotate the three-dimensional brain structure according to the acquisition angle of the two-dimensional DSA image to make it the same as the acquisition angle of the two-dimensional DSA image, and calculate the DSA two-dimensional plane projection of the three-dimensional brain structure.

[0047] S412. Perform blood vessel segmentation on the DSA two-dimensional plane projection and the two-dimensional DSA image using an anisotropic diffusion method.

[0048] S413: Based on the landmark vessels, the DSA 2D projection and the vessels in the 2D DSA image are aligned to obtain the corresponding vessels. The corresponding vessels are then compared to determine whether there are regions of distinct shapes in the corresponding vessels. If so, the process proceeds to S414; if not, the initial calibration ends. Specifically, the anterior cerebral artery and the posterior cerebral artery are selected as landmark vessels.

[0049] S414: Adjust the edge pixel distribution of blood vessels in the three-dimensional brain structure based on the shape-distinguishing region and the conventional brain blood vessel model.

[0050] S415 . Recalculate the DSA two-dimensional plane projection of the three-dimensional brain structure, and return to S412 .

[0051] S42: Perform secondary correction on the three-dimensional brain structure based on the blood vessel size in the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures. The specific process of the secondary correction includes:

[0052] S421. Rotate the three-dimensional brain structure according to the acquisition angle of the two-dimensional DSA image so that it is the same as the acquisition angle of the two-dimensional DSA image, and calculate the DSA two-dimensional plane projection of the three-dimensional brain structure.

[0053] S422. Perform blood vessel segmentation on the DSA two-dimensional plane projection and the two-dimensional DSA image using an anisotropic diffusion method.

[0054] S423. Calculate the size ratio based on the landmark blood vessels. Specifically, select the anterior cerebral artery and the posterior cerebral artery as the landmark blood vessels, and calculate the average of the average width ratio of the anterior cerebral artery and the average width ratio of the posterior cerebral artery in the DSA two-dimensional plane projection and the two-dimensional DSA image as the size ratio.

[0055] S424: Based on the landmark vessels, the DSA 2D projection and the vessels in the 2D DSA image are mapped to obtain unique vessels that exist only in the 2D DSA image. The width of the unique vessels in the 3D brain structure is calculated based on the size ratio. Specifically, the anterior cerebral artery and the posterior cerebral artery are selected as landmark vessels.

[0056] S425. Based on a conventional brain vascular model, the extension angle of each unique blood vessel in three-dimensional space is determined according to the pinhole imaging principle, and the extension angles of each unique blood vessel are arranged and combined to obtain multiple alternative three-dimensional brain structures. In each alternative three-dimensional brain structure, the length of the unique blood vessel is determined according to the extension angle of the unique blood vessel, and pixel interpolation is performed based on the length of the unique blood vessel and the width occupied by the unique blood vessel in the three-dimensional brain structure to construct the contour pixels of the unique blood vessel.

[0057] S5. Based on the registered left and right infrared thermal images, candidate 3D brain structures are screened to determine the final 3D brain structure. The specific screening process involves calculating the matching coefficient between each candidate 3D brain structure and the registered left and right infrared thermal images, and selecting the candidate 3D brain structure with the highest matching coefficient as the final 3D brain structure. The matching coefficient calculation process is to perform virtual imaging of the alternative brain three-dimensional structure based on the pinhole imaging principle, calculate and obtain the left alternative two-dimensional plane projection image and the right alternative two-dimensional plane projection image with the same acquisition angle as the registered left side infrared thermal image and the right side infrared thermal image, and binarize the left alternative two-dimensional plane projection image and the right alternative two-dimensional plane projection image using the method of first dilation and then erosion to obtain the corresponding binary image. The proportion of pixels with the same grayscale value between the binarized images of the registered left side infrared thermal image and the left alternative two-dimensional plane projection image is used as the first similarity, and the proportion of pixels with the same grayscale value between the binarized images of the registered right side infrared thermal image and the right alternative two-dimensional plane projection image is used as the second similarity. The average of the first similarity and the second similarity is calculated as the matching coefficient.

[0058] The above disclosure is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. It should be noted that for those skilled in the art, any equivalent changes made to the present invention without departing from the design structure and principles of the present invention are considered to be within the scope of protection of the present invention.

Claims

1. A multimodal image fusion algorithm for neurointerventional therapy, characterized by ,The multimodal image fusion algorithm specifically includes the following steps: S1, acquiring MRI images, left lateral infrared thermal images, right lateral infrared thermal images, and two-dimensional DSA images; S2. Reconstruct the three-dimensional structure of the brain based on MRI images; S3. Performing a registration operation on the left side infrared thermal image and the right side infrared thermal image to obtain the registered left side infrared thermal image and the right side infrared thermal image; S4. Correcting the three-dimensional brain structure based on the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures; S5. Screening candidate three-dimensional brain structures based on the registered left-side infrared thermal image and the right-side infrared thermal image to determine a final three-dimensional brain structure; In S4, the specific process of correction is as follows: S41, performing initial correction of the three-dimensional brain structure based on the vascular contours in the two-dimensional DSA image; S42, performing secondary correction on the three-dimensional brain structure based on the blood vessel size in the two-dimensional DSA image to obtain multiple candidate three-dimensional brain structures; In S5, the specific process of screening is to calculate the matching coefficients between each candidate brain three-dimensional structure and the registered left side infrared thermal image and right side infrared thermal image, and select the candidate brain three-dimensional structure with the highest matching coefficient as the final brain three-dimensional structure; The matching coefficient calculation process is to perform virtual imaging on the alternative brain three-dimensional structure based on the pinhole imaging principle, calculate and obtain the left alternative two-dimensional plane projection image and the right alternative two-dimensional plane projection image with the same acquisition angle as the aligned left side infrared thermal image and the right side infrared thermal image, binarize the left alternative two-dimensional plane projection image and the right alternative two-dimensional plane projection image using the method of first dilation and then corrosion to obtain corresponding binary images, and use the proportion of pixels with the same grayscale value between the binarized images of the aligned left side infrared thermal image and the left alternative two-dimensional plane projection image as the first similarity, and use the proportion of pixels with the same grayscale value between the binarized images of the aligned right side infrared thermal image and the right alternative two-dimensional plane projection image as the second similarity, and calculate the average of the first similarity and the second similarity as the matching coefficient.

2. The multimodal image fusion algorithm according to claim 1, characterized in that: In S1, the MRI image, the left side infrared thermal image, and the right side infrared thermal image are acquired in the same body position before the operation, and the two-dimensional DSA image is acquired during the operation.

3. The multimodal image fusion algorithm according to claim 1, characterized in that: In S3, the registration operation specifically includes: S31. Virtually image the three-dimensional structure of the brain based on the pinhole imaging principle, and calculate and obtain a left-side two-dimensional plane projection image and a right-side two-dimensional plane projection image at the same acquisition angle as the left-side infrared thermal image and the right-side infrared thermal image; S32, registering the left side infrared thermal image based on the left two-dimensional plane projection image to obtain a registered left side infrared thermal image; S33. Register the right side infrared thermal image based on the right two-dimensional plane projection image to obtain the registered right side infrared thermal image.

4. The multimodal image fusion algorithm according to claim 1, characterized in that: In S41, the specific process of the initial calibration includes: S411, rotating the three-dimensional brain structure according to the acquisition angle of the two-dimensional DSA image so that it is the same as the acquisition angle of the two-dimensional DSA image, and calculating the DSA two-dimensional plane projection of the three-dimensional brain structure; S412, performing blood vessel segmentation on the DSA two-dimensional plane projection and the two-dimensional DSA image by an anisotropic diffusion method; S413: Based on the landmark blood vessels, the DSA two-dimensional plane projection and the blood vessels in the two-dimensional DSA image are matched to obtain the corresponding blood vessels. By comparison, it is determined whether there are regions with different shapes in the corresponding blood vessels. If so, the process proceeds to S414; if not, the initial calibration ends. S414, adjusting edge pixel distribution of blood vessels in the three-dimensional brain structure based on the shape-distinguishing region and a conventional brain blood vessel model; S415 . Recalculate the DSA two-dimensional plane projection of the three-dimensional brain structure, and return to S412 .

5. The multimodal image fusion algorithm according to claim 1, characterized in that: In S42, the specific process of the secondary correction includes: S421, rotating the three-dimensional brain structure according to the acquisition angle of the two-dimensional DSA image so that it is the same as the acquisition angle of the two-dimensional DSA image, and calculating the DSA two-dimensional plane projection of the three-dimensional brain structure; S422, performing blood vessel segmentation on the DSA two-dimensional plane projection and the two-dimensional DSA image by an anisotropic diffusion method; S423, calculate size ratio based on landmark blood vessels; S424. Based on the landmark blood vessels, the DSA two-dimensional plane projection and the blood vessels in the two-dimensional DSA image are matched to obtain unique blood vessels that only exist in the two-dimensional DSA image, and the width occupied by the unique blood vessels in the three-dimensional structure of the brain is calculated based on the size ratio.

6. The multimodal image fusion algorithm according to claim 5, characterized in that: In S42, the anterior cerebral artery and the posterior cerebral artery are selected as landmark blood vessels.

7. The multimodal image fusion algorithm according to claim 6, characterized in that: In the above S423, the average value of the ratio of the average width of the anterior cerebral artery to the average width of the posterior cerebral artery in the DSA two-dimensional plane projection and the two-dimensional DSA image is calculated as the size ratio.

Images