A three-dimensional medical image visualization system for CT-MRI combined reconstruction

Abstract

The application discloses a CT-MRI combined reconstruction three-dimensional medical image visualization system, relates to the medical image processing technical field, and has the technical points that the system comprises a data acquisition and preprocessing module, a multi-modal registration module, a structural feature extraction module, a cross-modal fusion reconstruction module, a three-dimensional volume rendering module and an interactive visualization module; through standardized processing and high-precision registration of CT and MRI images, bone tissue and soft tissue structural features are extracted, and voxel-level adaptive fusion reconstruction is realized based on structure guidance, so that fused three-dimensional image data is generated; and three-dimensional visualization display and interactive operation are further completed. The application can take into account bone structure clarity and soft tissue detail expression, improves image fusion quality and three-dimensional reconstruction precision, and is suitable for preoperative planning and medical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to a three-dimensional medical image visualization system based on CT-MRI combined reconstruction. Background Technology

[0002] With the continuous development of medical imaging technology, computed tomography (CT) and magnetic resonance imaging (MRI) have become the two most commonly used imaging methods in clinical diagnosis. CT, based on the principle of X-ray attenuation, features fast imaging speed and high spatial resolution, making it particularly suitable for displaying bone tissue, calcified structures, and high-density lesions. MRI, based on the principle of nuclear magnetic resonance, has excellent soft tissue contrast and can clearly present soft tissue structures such as brain tissue, muscles, nerves, and tumors. Due to the differences in imaging mechanisms and information representation between the two imaging methods, in actual clinical practice, it is usually necessary to combine CT and MRI images for comprehensive judgment to obtain more complete diagnostic information.

[0003] However, in current technologies, the combined application of multimodal medical imaging is mostly limited to separate image reading or simple image overlay. Doctors typically need to view CT and MRI images separately on different devices or interfaces and make subjective comparisons based on experience, which not only increases their workload but also easily leads to judgment biases due to inconsistencies in observation angles or slice positions. Some existing technologies attempt to achieve multimodal image integration through image registration and fusion methods, but they generally suffer from insufficient registration accuracy and limited fusion methods. For example, rigid registration alone cannot handle spatial differences caused by soft tissue deformation, while simple weighted overlay methods cannot distinguish the quality of information from different tissues in different modalities, easily leading to blurred bone structure boundaries or loss of soft tissue details.

[0004] Furthermore, in 3D reconstruction, existing methods mostly model single-modal data, lacking a unified modeling mechanism for multimodal information. This results in reconstruction results that tend to favor certain modal characteristics, making it difficult to simultaneously maintain structural integrity and detailed representation. Especially in complex anatomical regions or when lesion boundaries are blurred, single-modal 3D models struggle to accurately reflect the true spatial relationships. Simultaneously, existing visualization systems have limited interactive functions, lacking the ability to display and dynamically analyze multimodal fused data in a layered manner, failing to meet the high demands of preoperative planning and precision medicine for 3D imaging.

[0005] Therefore, how to effectively fuse CT and MRI images while ensuring spatial registration accuracy, construct a high-quality 3D model that can simultaneously express bone structure and soft tissue information, and provide intuitive and efficient interactive visualization methods has become a pressing technical problem in the field of medical image processing.

[0006] Therefore, there is an urgent need to design a three-dimensional medical image visualization system that combines CT and MRI reconstruction to solve these problems. Summary of the Invention

[0007] The purpose of this invention is to solve the technical problems mentioned in the background section and to provide a three-dimensional medical image visualization system based on CT-MRI combined reconstruction.

[0008] The objective of this invention is achieved through the following technical solution: A 3D medical image visualization system based on combined CT-MRI reconstruction includes: The system includes a data acquisition and preprocessing module, a multimodal registration module, a structural feature extraction module, a cross-modal fusion and reconstruction module, a 3D volume rendering module, and an interactive visualization module. The data acquisition and preprocessing module is used to acquire CT and MRI image data of the same subject, and to perform spatial resolution unification, grayscale normalization, noise suppression and inter-slice interpolation on the image data to output normalized volume data. The multimodal registration module is connected to the data acquisition and preprocessing module and is used to perform spatial alignment processing on the standardized CT image data and MRI image data to generate registered multimodal image data. The structural feature extraction module is connected to the multimodal registration module and is used to extract bone tissue structural features from the registered CT image data, extract soft tissue structural features from the MRI image data, and construct structural guidance information. The cross-modal fusion reconstruction module is connected to the structural feature extraction module and is used to perform voxel-level fusion of CT image data and MRI image data based on the structural guidance information to generate fused three-dimensional image data. The 3D volume rendering module is connected to the cross-modal fusion reconstruction module and is used to perform volume rendering or surface reconstruction processing on the fused 3D image data to generate a 3D visualization model. The interactive visualization module is connected to the 3D volume drawing module and is used to realize multi-view display, structural layer display and interactive operation of the 3D visualization model.

[0009] Furthermore, the spatial resolution unification processing in the data acquisition and preprocessing module includes voxel size resampling of CT image data and MRI image data to ensure that the voxel size of the two types of image data remains consistent in three-dimensional space. The grayscale normalization process includes mapping different modal image data to a uniform grayscale range to eliminate grayscale differences between different imaging devices.

[0010] Furthermore, the multimodal registration module includes a coarse registration unit and a fine registration unit; The coarse registration unit is used to perform initial spatial alignment of CT image data and MRI image data based on rigid transformation. The fine registration unit is used to refine the registration of local structures based on the initial spatial alignment using a non-rigid deformation model, thereby improving the spatial consistency between multimodal images.

[0011] Furthermore, the multimodal registration module employs a similarity measurement method based on mutual information during the registration process, and optimizes the mutual information value to achieve the best alignment effect between images of different modalities.

[0012] Furthermore, the structural feature extraction module includes: The CT structure extraction unit is used to extract bone tissue contour features based on gradient information or edge detection algorithms. MRI structure extraction unit, used to extract soft tissue structure information based on region segmentation or texture analysis methods; The structure guidance construction unit is used to generate a structure guidance map based on the bone tissue contour features and soft tissue structure information.

[0013] Furthermore, the cross-modal fusion reconstruction module includes a voxel weight allocation unit, which is used to assign corresponding CT weights and MRI weights to each voxel according to the structural guidance map, thereby achieving differentiated fusion of different tissue regions.

[0014] Furthermore, the voxel weight allocation unit adjusts the weights according to the tissue type to which the voxel belongs: Increase the weight of CT image data when the voxel belongs to the bone tissue region; Increase the weight of MRI image data when voxels belong to soft tissue regions; When voxels are in the tissue transition region, CT image data and MRI image data are smoothly fused.

[0015] Furthermore, the cross-modal fusion reconstruction module also includes a deep fusion unit, which is used to perform feature-level fusion of CT image data and MRI image data based on a three-dimensional convolutional neural network, so as to further improve the structural integrity and detail expression of the fused image.

[0016] Furthermore, the 3D volume rendering module includes: Volume rendering unit, used to generate semi-transparent 3D images based on volume rendering algorithms; Surface reconstruction unit, used to generate a 3D mesh model based on the isosurface extraction algorithm; The organization layer display unit is used to render and display the 3D model in layers according to different organization categories.

[0017] Furthermore, the interactive visualization module includes: Multi-plane reconstruction unit, used to generate two-dimensional cross-sectional images in arbitrary directions; The 3D interactive control unit is used to realize the rotation, scaling and sectioning operations of the 3D model; The annotation and measurement unit is used to annotate the target area and measure distance, area, or volume. The display control unit is used to switch between bone tissue, soft tissue, or fusion display modes according to user instructions.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves the synergistic expression of bone and soft tissue information by constructing a multimodal joint reconstruction mechanism combining CT and MRI. Compared to traditional single-modal imaging or simple overlay fusion methods, this invention, based on structure-guided and voxel-level weight allocation strategies, enables different tissue regions to adaptively select the optimal information source. This allows for the simultaneous presentation of clear bony outlines and detailed soft tissue hierarchies within the same three-dimensional image, effectively solving the problems of "clear bone structure but blurred soft tissue" or "clear soft tissue but missing bone structure" in existing technologies, significantly improving the information integrity and diagnostic value of medical images.

[0019] 2. This invention improves the spatial consistency and fusion accuracy between CT and MRI images by introducing a multimodal registration optimization mechanism and a gradient-feature-based structure-guided modeling method. Through mutual information-driven non-rigid registration, high-precision alignment of different modalities in anatomical structures is achieved. Simultaneously, a structure-guided function is constructed using gradient magnitudes, and voxel weights are constrained through a neighborhood smoothing strategy, effectively reducing artifacts caused by noise, local mismatches, or grayscale fluctuations during the fusion process. This results in a more natural and continuous fusion result in boundary transition regions, leading to a more stable and reliable overall image.

[0020] 3. This invention, through the collaborative design of 3D volume rendering and interactive visualization modules, achieves intuitive display and multi-dimensional interactive operation of fused images. The system supports functions such as multi-view rotation, arbitrary section reconstruction, structural layering display, and lesion annotation and measurement, enabling doctors to intuitively understand the spatial relationships between different tissues and the distribution of lesions in a unified 3D space. This reduces reliance on repeated comparisons of multiple 2D slides, thereby improving diagnostic efficiency and accuracy. Simultaneously, the system possesses excellent scalability and engineering implementation capabilities, and can be applied to various scenarios such as preoperative planning, precision medicine, medical teaching, and AI-assisted diagnosis, demonstrating high clinical application value. Attached Figure Description

[0021] Figure 1 This is a system block diagram of the present invention; Figure 2This is a diagram showing the composition of the structural feature extraction module of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with embodiments and appendices. Figures 1-2 The present invention will be further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] Example 1: This example describes a 3D medical image visualization system based on CT-MRI combined reconstruction. It is primarily used for the collaborative processing of CT and MRI image data acquired from the same patient at different times or during the same treatment process. Through unified preprocessing, spatial registration, structural feature extraction, cross-modal fusion reconstruction, 3D volume rendering, and interactive visualization of the two types of images, a 3D medical image model with both clear bony structures and soft tissue display capabilities is obtained. This provides high-quality visualization support for applications such as preoperative assessment, lesion localization, anatomical relationship assessment, surgical path planning, and teaching demonstrations. The system generally includes a data acquisition and preprocessing module, a multimodal registration module, a structural feature extraction module, a cross-modal fusion reconstruction module, a 3D volume rendering module, and an interactive visualization module. These modules are sequentially connected according to the medical image processing flow, forming a complete technical chain from raw data input to 3D model output and then to clinical interactive operation.

[0024] In this embodiment, the data acquisition and preprocessing module receives CT and MRI image data of the same subject. CT image data typically comes from spiral CT or multi-slice CT equipment and mainly reflects information about bones, calcifications, dense tissues, and high-density structural boundaries. MRI image data typically comes from 1.5T or 3.0T MRI equipment and mainly reflects information about brain tissue, muscles, ligaments, tumor boundaries, perivascular tissues, and other soft tissues. Because the two imaging methods differ significantly in imaging principles, spatial resolution, grayscale range, slice thickness, noise characteristics, and scanning posture, they cannot be directly used for fusion reconstruction. Instead, the data acquisition and preprocessing module first performs standardization processing. Specifically, this module first checks the data integrity of the received CT and MRI sequences to confirm that the slice order, slice number, voxel spacing, and patient identification information are consistent. After confirming that the two types of data belong to the same subject, the data in DICOM format or other standard medical image formats is parsed into three-dimensional volumetric data suitable for subsequent processing.

[0025] Furthermore, the aforementioned data acquisition and preprocessing module performs spatial resolution unification processing on CT and MRI image data. Since CT often has high planar resolution while MRI often has high soft tissue contrast but significant variations in slice thickness and interslice spacing, resampling is necessary to map both types of images to a unified voxel size. Specifically, trilinear interpolation, cubic convolution interpolation, or neighborhood-preserving resampling methods can be used to ensure that CT and MRI volumetric data have a consistent voxel scale in the same three-dimensional coordinate system, thus providing a foundation for subsequent one-to-one voxel-level fusion. After spatial resolution unification, grayscale normalization is further performed. For CT data, truncation and window transformation can be performed based on its Hounsfield unit range, mapping the grayscale values ​​in bone windows, soft tissue windows, or other windows of interest to a preset standard grayscale range. For MRI data, linear or percentile normalization of pixel intensity is performed based on different sequence types, such as T1-weighted, T2-weighted, FLAIR, or enhanced sequences, to reduce grayscale drift caused by different equipment and scanning parameters. After grayscale normalization, the system can perform subsequent calculations on image data of different modalities in a unified numerical domain, improving the stability of the registration and fusion process.

[0026] After achieving resolution unification and grayscale normalization, the data acquisition and preprocessing module performs noise suppression and inter-slice interpolation on the image data. CT data is often affected by quantum noise and reconstruction artifacts, while MRI data may be affected by thermal noise, motion artifacts, and magnetic field inhomogeneity. Therefore, anisotropic diffusion filtering, nonlocal mean filtering, median filtering, or bias field correction can be used to suppress noise, improving image quality while ensuring that structural edges are not blurred as much as possible. Inter-slice interpolation is mainly used for cases where the original scan has large slice thickness, inconsistent slice intervals, or local slice gaps. It generates more continuous volume data through compensation, reducing jagged edges and tomographic phenomena in subsequent 3D reconstruction. After the above processing, normalized volume data is output as input to the subsequent multimodal registration module.

[0027] The multimodal registration module is connected to the data acquisition and preprocessing module to perform spatial alignment processing on standardized CT and MRI image data, generating registered multimodal image data. Due to differences in body position, respiratory motion, slight head movement, equipment coordinate system differences, and tissue deformation during CT and MRI acquisition, the two types of data, even from the same subject, are difficult to align naturally. Therefore, in this embodiment, the multimodal registration module preferably adopts a two-stage implementation method of "coarse registration + fine registration". The coarse registration stage mainly uses rigid transformation to align the overall position of CT and MRI volume data, specifically including three-dimensional translation and three-dimensional rotation adjustments, so that the main anatomical regions of the two volume data roughly overlap. This stage can be based on the patient's body surface contour, scanning bed orientation information, skull edges, or manually / automatically labeled anatomical landmarks for initial estimation. After coarse registration, the fine registration stage further introduces a local deformation adjustment mechanism to compensate for inconsistencies caused by differences in scanning posture, soft tissue deformation, or local deformation. This fine registration process can employ B-spline free deformation, optical flow-based non-rigid registration, thin-plate spline transformation, or other deformation models commonly used in medical imaging to perform continuous, smooth, and anatomically reasonable alignment corrections on local areas.

[0028] In the registration process, to achieve effective matching between different modalities, this embodiment preferably adopts a similarity measurement mechanism based on mutual information. Since the grayscale meanings of CT and MRI differ, the mean square error or minimum grayscale difference criteria commonly used for images of the same modality cannot be simply applied. Therefore, by calculating the statistical correlation between the grayscale distributions of CT and MRI, mutual information is used to evaluate the degree of information sharing between the two under the current spatial transformation, and iterative optimization is performed with the goal of maximizing the mutual information value. In other words, the more accurate the registration result, the stronger the statistical correlation between CT and MRI at the corresponding anatomical locations, and the higher their mutual information value. The system can continuously adjust rigid and non-rigid deformation parameters through optimization algorithms until the mutual information reaches its maximum or the convergence threshold meets the preset requirements. After the above processing, the multimodal registration module outputs spatially consistent CT-MRI registration body data, laying the foundation for subsequent structural feature extraction and voxel-level fusion.

[0029] The structural feature extraction module is connected to the multimodal registration module to extract bone tissue structural features from registered CT image data and soft tissue structural features from MRI image data, and to construct structural guidance information. The purpose of this module is not simply to extract edges, but to establish a structural expression mechanism that reflects the dominant regions of different modalities, enabling the system to know "which regions should be trusted more by CT and which regions should be trusted more by MRI" during fusion. In this embodiment, the structural feature extraction module includes at least a CT structural extraction unit, an MRI structural extraction unit, and a structural guidance construction unit. For the CT structural extraction unit, it mainly leverages the characteristic of CT to clearly display high-density tissues, performing bone tissue structure enhancement and edge extraction on the registered CT volume data. Specifically, it can identify skull, vertebrae, joints, fracture lines, calcifications, or other high-density boundary structures through threshold segmentation, gradient amplitude calculation, Sobel operator, Canny edge detection, or region connectivity analysis, while preserving bone surface contours, sharp edges, and bony canal information. For the MRI structure extraction unit, the focus is on the high sensitivity of MRI to soft tissues, lesion boundaries, and anatomical layers. It performs texture analysis, region growing, hierarchical segmentation, or tissue recognition based on trained models on MRI volume data to extract structural information of brain parenchyma, muscle tissue, tumor regions, hematoma regions, spinal cord, intervertebral discs, ligaments, and perivascular soft tissues. After extraction, corresponding tissue labels, boundary information, and probability distribution information can be generated for different tissue categories.

[0030] The structure-guided construction unit generates a structure-guided map based on the aforementioned CT bone contour features and MRI soft tissue structure information. This structure-guided map can be understood as a voxel-level tissue attribute reference map, which not only includes a classification of each spatial location as leaning more towards bone or soft tissue structure, but also reflects comprehensive information such as edge clarity, texture complexity, and modal reliability. For locations with clear bone boundaries but weak signals in the corresponding MRI region, the structure-guided map assigns a higher CT priority label; for locations with clear lesion edges and high tissue contrast in MRI but insufficient visualization on CT, the structure-guided map assigns a higher MRI priority label; and for transitional regions at the bone-soft tissue interface or contributing to both modalities, they are recorded as balanced fusion regions. This structure-guided map is not the final displayed image, but rather serves as a key control basis in the subsequent cross-modal fusion reconstruction module, upgrading the fusion process from simple image addition to structure-guided fusion with tissue recognition capabilities.

[0031] The cross-modal fusion reconstruction module is connected to the structural feature extraction module and is used to perform voxel-level fusion of CT and MRI image data based on the structural guidance information to generate fused 3D image data. This module is the core part of the joint reconstruction in this embodiment. Its task is not simply to intuitively overlay the two types of images, but to differentiate the expression contribution of each voxel in different modalities according to the structural guidance map, thereby taking into account both the bone structure clarity of CT and the soft tissue discernibility of MRI. Specifically, the cross-modal fusion reconstruction module includes a voxel weight allocation unit. This unit assigns corresponding fusion weights to the CT and MRI data respectively based on the tissue attributes, edge state, local texture features, and consistency of adjacent voxels in the structural guidance map of each voxel. In bone tissue regions, the system automatically increases the weight of CT image data, making the boundaries of bony structures sharper and fracture lines or calcified areas clearer in the final 3D image. In soft tissue regions, the system increases the weight of MRI image data, making information such as tumor boundaries, brain tissue layers, and soft tissue lesion morphology more complete. In tissue transition regions, the system uses continuously changing weights for smooth fusion, avoiding abrupt boundary jumps or modal switching traces in the final image.

[0032] Furthermore, to improve the spatial continuity and clinical readability of the fused 3D image data, the cross-modal fusion reconstruction module can perform local consistency constraint processing and boundary optimization processing on the fusion result after voxel-level weighted fusion. Local consistency constraint is mainly used to ensure that adjacent voxels have continuous performance when their structural properties are similar, thereby avoiding display flickering and small patch artifacts caused by local noise or isolated misjudgments. Boundary optimization processing is used to retain sufficient edge clarity at bone-soft tissue junctions, lesion peripheries, and surgically relevant areas, ensuring that the 3D display result is neither overly smooth, leading to detail loss, nor exhibiting ghosting due to modal superposition. For applications requiring a higher degree of automation, the cross-modal fusion reconstruction module may also include a deep fusion unit. This deep fusion unit can use a 3D convolutional neural network to perform feature-level fusion of CT and MRI image data, that is, first extracting deep features from CT and MRI at different scales, then performing joint encoding and decoding in the latent space, ultimately outputting a structurally complete and richly layered 3D fused image. This deep fusion method is particularly suitable for complex lesions, areas with multiple interwoven tissues, and situations where traditional rule-based methods are difficult to accurately balance. Regardless of whether the classic fusion method based on weight mapping or the deep fusion method is used, the output is fused 3D image data that can be used for 3D reconstruction.

[0033] The 3D volume rendering module is connected to the cross-modal fusion reconstruction module and is used to perform volume rendering or surface reconstruction processing on fused 3D image data to generate a 3D visualization model. Since different clinical application scenarios have different requirements for display formats, the 3D volume rendering module in this embodiment includes a volume rendering unit, a surface reconstruction unit, and a tissue layering display unit. The volume rendering unit is mainly used to generate a semi-transparent 3D medical volume data representation. By setting the transparency mapping, grayscale mapping, and illumination response parameters of different tissues, bone contours, soft tissue encapsulation relationships, and internal lesion locations can be displayed simultaneously in the same view, allowing doctors to observe the spatial embedding relationships and mutual occlusion between target structures. The surface reconstruction unit mainly targets tissue structures with well-defined boundaries, converting volume data into a 3D mesh model through isosurface extraction, thereby generating a triangular mesh surface model suitable for rotational observation, sectional analysis, and surgical path evaluation. The tissue layering display unit, based on previously established tissue category information, performs layered rendering and display of bone tissue, soft tissue, lesion tissue, vascular regions, or other structures of interest, allowing users to choose to display only a certain type of structure or overlay multiple types of structures as needed, improving information extraction efficiency.

[0034] The interactive visualization module is connected to the 3D volume rendering module to enable multi-view display, hierarchical structural representation, and interactive operation of the 3D visualization model. This module is directly geared towards doctors, technicians, and researchers, and is a crucial component in this embodiment for enhancing system usability and clinical application value. Specifically, the interactive visualization module includes a multi-plane reconstruction unit, a 3D interactive control unit, an annotation and measurement unit, and a display control unit. The multi-plane reconstruction unit generates axial, sagittal, coronal, and arbitrary-angle cross-sectional images from fused 3D image data or 3D models, allowing doctors to view the overall 3D relationships and also return to a 2D perspective for detailed observation of challenging areas. The 3D interactive control unit enables model rotation, scaling, translation, local sectioning, transparency adjustment, and preset switching of viewing angles, allowing different anatomical locations to be observed from the most suitable angle. The annotation and measurement unit provides textual annotations for lesion locations, fracture line locations, tumor boundaries, vascular proximity, or areas of preoperative interest, and can perform measurements such as distance, area, volume, and angle, providing quantitative references for clinical decision-making. The display control unit switches between bone tissue display mode, soft tissue display mode and fusion display mode according to the user's interactive commands. It can also further control whether to display a specific tissue layer, whether to enable semi-transparent mode, whether to overlay lesion markers, etc., so as to achieve customized display that is adapted to clinical work habits.

[0035] In a specific application scenario, taking the preoperative assessment of patients with intracranial lesions as an example, the patient's head CT and MRI data are first collected. After receiving the data, the data acquisition and preprocessing module performs parsing, resolution unification, grayscale normalization, noise suppression, and inter-slice interpolation, outputting standardized head CT and MRI volumetric data. Subsequently, the multimodal registration module first achieves approximate overlap between the overall skull contour and the brain parenchyma region through rigid transformation, then refines the alignment between the brain tissue edges and the surrounding soft tissue region through non-rigid registration, using maximum mutual information as the optimization criterion to obtain a spatially consistent registration result. Following this, the structural feature extraction module extracts skull boundaries, calcifications, and high-density bone window structures from the CT scan, and extracts the brain tissue contour, tumor region, and surrounding edema extent from the MRI scan, generating a structural guidance map. The cross-modal fusion reconstruction module assigns high CT weights to skull-related voxels and high MRI weights to brain tissue and tumor-related voxels based on the structural guidance map. It also uses a smooth transition method to fuse the periosteal and tumor-adjacent bone regions, resulting in fused 3D image data with both clear skull contours and well-defined soft tissue lesion boundaries. The 3D volume rendering module further constructs a 3D model from this fused data and allows for semi-transparent display of the skull, enabling surgeons to visually observe the spatial relationship of the tumor relative to the skull and midline structures. Finally, the interactive visualization module allows surgeons to freely rotate the model, switch viewing angles, dissect local tissues, mark tumor boundaries, and measure lesion volume, thus supporting surgical approach design, lesion exposure prediction, and preoperative communication.

[0036] For example, in the analysis of spinal lesions, CT scans are mainly used to display the bony structure of the vertebral body, pedicles, and fracture collapse, while MRI is mainly used to display intervertebral disc herniation, spinal cord compression, nerve root course, and soft tissue inflammatory changes. The system in this embodiment, through the collaborative work of the above modules, can simultaneously display the vertebral body structure, spinal canal morphology, spinal cord compression status, and the extent of adjacent soft tissue lesions in a unified 3D model, which is significantly superior to the fragmented judgment method when viewing CT or MRI slices individually. Especially when planning pedicle screw placement, tumor resection, or decompression surgery, doctors can more clearly identify the spatial relationship between bony pathways and nerve tissue, improving diagnostic efficiency and accuracy.

[0037] In this embodiment, the above modules can be integrated and implemented on the same server platform, or they can be distributed and implemented by multiple functional processing units. For example, the data acquisition and preprocessing module, the multimodal registration module, the structural feature extraction module, and the cross-modal fusion reconstruction module can be deployed in the background image processing server, while the 3D volume rendering module and the interactive visualization module can be deployed in the front-end workstation or clinical image reading terminal; alternatively, all modules can be uniformly deployed on a high-performance graphics workstation for direct use by doctors locally. Data communication between modules can be achieved through memory sharing, message bus, database retrieval, or network interface calls.

[0038] Example 2: Based on Example 1, this example further presents a CT-MRI joint reconstruction method that introduces a quantitative computational mechanism. Under the premise of maintaining consistency in system structure and module division, it provides a clear mathematical model for cross-modal registration, structure-guided construction, and voxel-level fusion process.

[0039] In this embodiment, the CT image volume data processed by the data acquisition and preprocessing module is represented as follows: MRI image volume data are represented as ,in Represents voxel coordinates in three-dimensional space; the This represents the CT grayscale value at voxel coordinate x; the... This represents the MRI grayscale value at voxel coordinate x. First, in the multimodal registration module, the MRI image undergoes a spatial transformation to align it with the CT image. The spatial transformation function is defined as:

[0040] Where T(x) represents the transformed coordinates; u(x) represents the deformation vector field at voxel coordinate x. This transformation maps the MRI image to the CT coordinate system, resulting in the registered MRI image. ; in, This represents the registered MRI grayscale value at voxel coordinate x.

[0041] To obtain the optimal registration result, this embodiment uses mutual information as the registration objective function, defined as: ; Wherein, MI represents the mutual information value; The grayscale entropy of a CT image; This represents the grayscale entropy of the registered MRI image; This represents the joint information entropy of the CT and registered MRI images. By continuously adjusting the deformation vector field u(x), the mutual information MI is maximized, thereby obtaining the optimal registration result.

[0042] After registration is completed, the structural feature extraction stage begins. First, gradient amplitudes are calculated on the CT images to characterize the edge features of the bone tissue. ; in, This represents the CT gradient magnitude at voxel coordinate x, used to reflect the strength of bone structure boundaries. Similarly, gradient magnitudes are calculated for registered MRI images to characterize changes in soft tissue structures. ; in, This represents the MRI gradient magnitude at voxel coordinate x. To construct structure-guided information, a structure response function is defined: ; Where S(x) represents the structural guidance coefficient at voxel coordinate x; ε is a small constant to prevent the denominator from being zero. This structural guidance coefficient is used to characterize whether the current voxel is more inclined towards bone structure or soft tissue structure. When S(x) approaches 1, it indicates that the position is more CT-dominated, and when S(x) approaches 0, it indicates that it is more MRI-dominated.

[0043] Based on this, construct a voxel-level weighting function: ; in, This represents the fusion weight of the CT image at voxel coordinate x; This represents the fusion weight of the MRI image at voxel coordinate x, and satisfies... .

[0044] Furthermore, to enhance the smoothness and spatial consistency of the fusion results, a neighborhood smoothing term is introduced to modify the weighting function: ; in, This represents the smoothed CT weights; Indicates the smoothed MRI weights; Represents the neighborhood set centered at voxel x; This represents the number of voxels in the neighborhood.

[0045] Finally, in the cross-modal fusion reconstruction module, voxel-level fusion calculations are completed based on the aforementioned weights to obtain fused 3D image data: ; in, This represents the grayscale value of the fused image at voxel coordinate x.

[0046] To further enhance the contrast of the boundary region, an edge enhancement term is introduced based on the fusion result: ; in, This indicates the enhanced fused image; This represents the edge enhancement coefficient, used to adjust the degree of edge information enhancement. It is obtained through the above calculations. The fused 3D image data, as the final output, is input into the 3D volume rendering module. Based on this data, the 3D volume rendering module performs volume rendering or isosurface extraction to generate a 3D visualization model; the interactive visualization module then performs multi-view display, arbitrary section reconstruction, and annotation measurement operations on the model, thereby completing the entire 3D medical image visualization process of CT-MRI combined reconstruction.

[0047] It should be noted that in all the above formulas, the meaning of the voxel coordinate x remains consistent, representing the position index in three-dimensional space. All represent the image grayscale values ​​at corresponding voxel positions in the same coordinate system; Always represents the gradient magnitude of the corresponding mode at that voxel; All of these represent voxel-level weights or their transformations, and their meanings remain consistent throughout the text and have not changed.

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the present invention is not limited to these embodiments. Equivalent modifications made by those skilled in the art without departing from the principles of the present invention should fall within the protection scope of the present invention.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional medical image visualization system based on CT-MRI combined reconstruction, characterized in that, include: The system includes a data acquisition and preprocessing module, a multimodal registration module, a structural feature extraction module, a cross-modal fusion and reconstruction module, a 3D volume rendering module, and an interactive visualization module. The data acquisition and preprocessing module is used to acquire CT and MRI image data of the same subject, and to perform spatial resolution unification, grayscale normalization, noise suppression and inter-slice interpolation on the image data to output normalized volume data. The multimodal registration module is connected to the data acquisition and preprocessing module and is used to perform spatial alignment processing on the standardized CT image data and MRI image data to generate registered multimodal image data. The structural feature extraction module is connected to the multimodal registration module and is used to extract bone tissue structural features from the registered CT image data, extract soft tissue structural features from the MRI image data, and construct structural guidance information. The cross-modal fusion reconstruction module is connected to the structural feature extraction module and is used to perform voxel-level fusion of CT image data and MRI image data based on the structural guidance information to generate fused three-dimensional image data. The 3D volume rendering module is connected to the cross-modal fusion reconstruction module and is used to perform volume rendering or surface reconstruction processing on the fused 3D image data to generate a 3D visualization model. The interactive visualization module is connected to the 3D volume drawing module and is used to realize multi-view display, structural layering display and interactive operation of the 3D visualization model.

2. The system according to claim 1, characterized in that, The spatial resolution unification processing in the data acquisition and preprocessing module includes voxel size resampling of CT image data and MRI image data to keep the voxel size of the two types of image data consistent in three-dimensional space. The grayscale normalization process includes mapping different modal image data to a uniform grayscale range to eliminate grayscale differences between different imaging devices.

3. The system according to claim 1, characterized in that, The multimodal registration module includes a coarse registration unit and a fine registration unit; The coarse registration unit is used to perform initial spatial alignment of CT image data and MRI image data based on rigid transformation. The fine registration unit is used to refine the registration of local structures based on the initial spatial alignment using a non-rigid deformation model, thereby improving the spatial consistency between multimodal images.

4. The system according to claim 1, characterized in that, The multimodal registration module employs a similarity measurement method based on mutual information during the registration process, and optimizes the mutual information value to achieve the best alignment effect between images of different modalities.

5. The system according to claim 1, characterized in that, The structural feature extraction module includes: The CT structure extraction unit is used to extract bone tissue contour features based on gradient information or edge detection algorithms. MRI structure extraction unit, used to extract soft tissue structure information based on region segmentation or texture analysis methods; The structure guidance construction unit is used to generate a structure guidance map based on the bone tissue contour features and soft tissue structure information.

6. The system according to claim 1, characterized in that, The cross-modal fusion reconstruction module includes a voxel weight allocation unit, which is used to assign corresponding CT weights and MRI weights to each voxel according to the structural guidance map, thereby achieving differentiated fusion of different tissue regions.

7. The system according to claim 6, characterized in that, The voxel weight allocation unit adjusts the weights according to the tissue type to which the voxel belongs: Increase the weight of CT image data when the voxel belongs to the bone tissue region; Increase the weight of MRI image data when voxels belong to soft tissue regions; When voxels are in the tissue transition region, CT image data and MRI image data are smoothly fused.

8. The system according to claim 1, characterized in that, The cross-modal fusion reconstruction module also includes a deep fusion unit, which is used to perform feature-level fusion of CT image data and MRI image data based on a three-dimensional convolutional neural network, so as to further improve the structural integrity and detail expression of the fused image.

9. The system according to claim 1, characterized in that, The 3D volume rendering module includes: Volume rendering unit, used to generate semi-transparent 3D images based on volume rendering algorithms; Surface reconstruction unit, used to generate a 3D mesh model based on the isosurface extraction algorithm; The organization layer display unit is used to render and display the 3D model in layers according to different organization categories.

10. The system according to claim 1, characterized in that, The interactive visualization module includes: Multi-plane reconstruction unit, used to generate two-dimensional cross-sectional images in arbitrary directions; The 3D interactive control unit is used to realize the rotation, scaling and sectioning operations of the 3D model; The annotation and measurement unit is used to annotate the target area and measure distance, area, or volume. The display control unit is used to switch between bone tissue, soft tissue, or fusion display modes according to user instructions.

Images