Three-dimensional reconstruction system of left atrium fusing multi-modal images

Abstract

The application relates to the technical field of image processing, in particular to a left atrium three-dimensional reconstruction system fusing multi-modal images. The system comprises a memory and a processor. The processor executes a computer program stored in the memory to realize the following steps: acquiring CT images and MRI images of a target object left atrium region; classifying feature corner points in each type of image according to feature vectors of the feature corner points; evaluating cross-modal fusion reliability according to the gray scale distribution of pixels in the neighborhood of each feature corner point in the CT images and the motion characteristics of corner points in the neighborhood of each feature corner point; determining the matching results of each feature corner point in the CT images in the MRI images based on the distribution of the cross-modal fusion reliability of the feature corner points in each type of corner point in the CT images and the feature vectors, and further constructing a left atrium three-dimensional model. The application realizes high-precision three-dimensional reconstruction of the left atrium.

Description

A three-dimensional reconstruction system for the left atrium integrating multimodal imaging Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a three-dimensional reconstruction system for the left atrium that fuses multimodal images. Background Technology

[0002] The left atrium and its adjacent structures play a crucial role in various cardiac interventional treatments. However, the left atrium has a complex anatomy and diverse disease mechanisms. Traditional preoperative imaging, such as non-invasive vascular imaging, can provide good global anatomical information, but it cannot be updated in real time. Single-modal imaging often cannot simultaneously take into account both global anatomy and real-time dynamic information, making it difficult to meet clinical needs.

[0003] Currently, strategies for 3D reconstruction of the left atrium using multimodal images largely rely on simple splicing of features from different modalities, such as edges, grayscale, and dynamic trajectories, to achieve feature fusion. This approach fails to effectively model and utilize the inherent correlations and complementarities between different modal features, leading to topological errors in key anatomical structures of the left atrium during feature fusion, such as voids or discontinuities in the model. This results in distortions such as structural blurring, positional shifts, or loss of dynamic motion information in the 3D model, severely restricting the precise localization of surgical targets in interventional cardiac treatment. Summary of the Invention

[0004] To address the distortion issues inherent in existing methods for three-dimensional reconstruction of the left atrium, this invention aims to provide a three-dimensional reconstruction system for the left atrium that integrates multimodal images. The specific technical solution employed is as follows:

[0005] This invention provides a three-dimensional reconstruction system for the left atrium based on multimodal imaging. The system includes a memory and a processor, the processor executing a computer program stored in the memory to achieve the following steps:

[0006] Acquire continuous multi-frame CT and MRI images of the left atrial region of the target object from different perspectives;

[0007] By combining the motion characteristics of each feature corner point in each type of image and the similarity of the motion characteristics of each feature corner point with its neighboring feature corner points, the feature vector of each feature corner point is determined; based on the feature vector, the feature corner points in each type of image are classified to obtain the corner points of each type.

[0008] The cross-modal fusion reliability of each feature corner point is evaluated based on the stability of the gray-level distribution of pixels in the neighborhood of each feature corner point in all CT images and the motion characteristics of the corner points in the neighborhood of each feature corner point. Based on the distribution and feature vector of the cross-modal fusion reliability of feature corner points in each type of corner point in CT images, the matching results of each feature corner point in CT images in MRI images are determined according to priority.

[0009] A three-dimensional model of the left atrium is constructed based on the matching results.

[0010] Preferably, the step of determining the feature vector of each feature corner point by combining the motion features of each feature corner point in each type of image and the similarity of the motion features of each feature corner point with its neighboring feature corner points includes:

[0011] For each type of image, the motion vector of each feature corner point in each two adjacent CT images is calculated using the optical flow method. The motion vector is used to reflect the motion characteristics of the feature corner point.

[0012] Based on the correlation coefficient between two adjacent motion vectors of each feature corner point, the motion correlation value corresponding to each feature corner point is determined; based on the standard deviation of the magnitude of the motion vector of each feature corner point and the motion correlation value, the comprehensive consistency index of each feature corner point is obtained.

[0013] Based on the Pearson correlation coefficient between each feature corner point and the motion vectors of its neighboring feature corner points, the motion coordination coefficient corresponding to each feature corner point is obtained.

[0014] The normalized value of the magnitude of the motion vector of each feature corner point, the comprehensive consistency index, and the motion coordination coefficient constitute the feature vector of each feature corner point.

[0015] Preferably, the step of obtaining the comprehensive consistency index of each feature corner point based on the standard deviation of the magnitude of the motion vector of each feature corner point and the motion correlation value includes:

[0016] Calculate the standard deviation of the magnitude of the motion vector at each feature corner point and the first sum of the preset zero-prevention parameters;

[0017] The ratio of the motion-related value to the first sum is determined as the comprehensive consistency index of each feature corner point.

[0018] Preferably, the step of classifying the feature corner points in each type of image based on the feature vector to obtain various types of corner points includes:

[0019] Based on the similarity between the feature vectors, all feature corner points are divided into three categories;

[0020] The feature corner points with the largest average modulus of the feature vectors are designated as structural critical corner points, the feature corner points with the smallest average modulus of the feature vectors are designated as noise artifact corner points, and the other type of feature corner points, excluding structural critical corner points and noise artifact corner points, are designated as dynamic response corner points.

[0021] Preferably, the evaluation of the cross-modal fusion reliability of each feature corner point based on the stability of the grayscale distribution of pixels in the neighborhood of each feature corner point in all CT images and the motion characteristics of the neighborhood corner points of each feature corner point includes:

[0022] For any feature corner point:

[0023] The gray-scale mean of all pixels in the neighborhood of any feature corner point in each CT image is obtained and denoted as the first gray-scale mean of any feature corner point in each CT image; the variance of the first gray-scale mean of any feature corner point in all CT images is calculated, and the local structural stability index of any feature corner point is obtained based on the variance.

[0024] Based on the similarity between the motion vector of any feature corner point in each image and the motion vector of its neighboring pixels, the motion representativeness coefficient of any feature corner point is obtained.

[0025] By combining the local structural stability index and the motion representativeness coefficient, the cross-modal fusion reliability factor of any feature corner point is obtained. The cross-modal fusion reliability factor is used to reflect the cross-modal fusion reliability.

[0026] Preferably, obtaining the motion representativeness coefficient of any feature corner point based on the similarity between its motion vector in each image and the motion vectors of its neighboring pixels includes:

[0027] Calculate the average vector of motion vectors of all pixels in the neighborhood of any given feature corner point in each CT image;

[0028] The cosine similarity between the motion vector of any feature corner point in each CT image and the corresponding average vector is used; the average value of the cosine similarity of any feature corner point in all CT images is used as the motion representativeness coefficient of any feature corner point.

[0029] Preferably, obtaining the cross-modal fusion reliability factor of any feature corner point by combining the local structural stability index and the motion representativeness coefficient includes: determining the cross-modal fusion reliability factor of any feature corner point by multiplying the local structural stability index and the motion representativeness coefficient.

[0030] Preferably, the method of determining the matching results of each feature corner point in the CT image in the MRI image according to priority based on the distribution and feature vector of the cross-modal fusion reliability of feature corner points in each type of corner point in the CT image includes:

[0031] For any type of corner point in a CT image:

[0032] In descending order of cross-modal fusion reliability factor, all feature corner points of any given type of corner point are registered with feature corner points of the same type in the MRI image.

[0033] During registration, the initial matching radius is corrected by the cross-modal fusion reliability factor of the corner point to be matched, and the corrected matching radius of the corner point to be matched is obtained. The cross-modal fusion reliability factor is negatively correlated with the corrected matching radius.

[0034] Using the corner point to be matched in the CT image as the center, among the same type of corner points in the MRI image, all feature corner points whose spatial coordinates are within the range of the corrected matching radius are selected to form a candidate corner point set for the corner point to be matched.

[0035] Calculate the straight-line distance in three-dimensional space between the corner point to be matched and each feature corner point in the candidate corner point set in the CT image, as well as the Euclidean distance between the feature vectors; combine the straight-line distance and the Euclidean distance to determine the matching point of the corner point to be matched in the MRI image;

[0036] The corner point to be matched is any feature corner point among any of the corner points of the aforementioned category.

[0037] Preferably, determining the matching point of the corner point to be matched in the MRI image by combining the linear distance and the Euclidean distance includes:

[0038] The product of the straight-line distance and the corresponding Euclidean distance between the corner point to be matched and each feature corner point in the candidate corner point set is used as a quantification index of the difference between the corner point to be matched and each feature corner point in the candidate corner point set.

[0039] The feature corner points in the candidate corner point set corresponding to the smallest difference quantification index are used as the matching points of the corner points to be matched in the MRI image.

[0040] Preferably, constructing a three-dimensional model of the left atrium based on the matching results includes:

[0041] Based on epipolar geometry constraints, the matching results of feature corner points in CT images and feature corner points in MRI images are converted into three-dimensional spatial coordinates using a triangulation algorithm to obtain the three-dimensional corner point set of the left atrium.

[0042] The three-dimensional corner point set is subjected to Delaunay triangulation, and the mesh is verified and corrected in combination with the anatomical topological features of the left atrium to generate a three-dimensional model of the left atrium.

[0043] The present invention has at least the following beneficial effects:

[0044] This invention first extracts feature vectors from feature corner points in continuous multi-frame CT and MRI images of the left atrial region of the target object from different perspectives. Based on these feature vectors, the feature corner points are classified. The cross-modal fusion reliability of the feature corner points is dynamically evaluated based on the stability of the grayscale distribution of pixels in the neighborhood of each feature corner point in all CT images and the motion characteristics of the neighborhood corner points. Then, based on the distribution and feature vectors of the cross-modal fusion reliability of feature corner points in each category of corner points in the CT images, the matching results of each feature corner point in the CT images in the MRI images are determined according to priority, establishing accurate cross-modal corner point correspondences. Finally, a three-dimensional model of the left atrium is constructed based on the matching results, achieving high-precision, highly dynamic, and consistent three-dimensional reconstruction of the left atrium, providing reliable three-dimensional model data for cardiac interventional surgery. Attached Figure Description

[0045] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 is a flowchart of a method performed by a left atrium three-dimensional reconstruction system that integrates multimodal images, as provided in an embodiment of the present invention. Detailed Implementation

[0047] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of a three-dimensional reconstruction system for the left atrium based on multimodal imaging, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0049] The following description, in conjunction with the accompanying drawings, details a specific scheme for a three-dimensional reconstruction system for the left atrium based on multimodal imaging provided by this invention.

[0050] Example of a three-dimensional reconstruction system for the left atrium integrating multimodal images:

[0051] This embodiment proposes a three-dimensional reconstruction system for the left atrium based on multimodal imaging. The system includes a memory and a processor. The processor executes a computer program stored in the memory. As shown in Figure 1, the method performed by the three-dimensional reconstruction system for the left atrium based on multimodal imaging in this embodiment includes the following steps:

[0052] Step S1: Acquire consecutive multi-frame CT and MRI images of the left atrial region of the target object from different perspectives.

[0053] CT and MRI scanners were used to perform multi-view scans of the left atrial region (covering the area from the root of the aorta to the floor of the left ventricle) of the target subject, acquiring continuous multi-frame CT and MRI images of the left atrial region from different perspectives. The CT scanner parameters were set to 0.5-1mm, tube voltage 120-140kV, and frame rate 30 frames / second. The acquisition angle and the number of different angles were set by the implementer according to the specific circumstances.

[0054] Gaussian filtering was used to denoise all CT and MRI images acquired from each modality. Simultaneously, spatial coordinate features of each modality were extracted to establish a three-dimensional coordinate system with the left atrial center as the origin. A time warping matching algorithm was used to synchronize the images acquired from each modality spatiotemporally, assigning them the same timestamp. Then, an affine transformation + elastic registration algorithm was used to align the MRI images to the CT image coordinate system. Spatial coordinate normalization was achieved through coordinate transformation, generating a multimodal fusion dataset containing two types of image data: MRI and CT images. A 3D U-Net model was used to segment each frame in the multimodal fusion dataset, extracting the left atrial mask image.

[0055] Step S2: Combine the motion characteristics of each feature corner point in each type of image with the similarity of the motion characteristics of each feature corner point with its neighboring feature corner points to determine the feature vector of each feature corner point; classify the feature corner points in each type of image according to the feature vector to obtain the corner points of each type.

[0056] Normally, cardiac motion exhibits a strict temporal dependence, meaning that the structural deformation and motion patterns differ significantly across different phases of the cardiac cycle. This embodiment uses the duration of continuously acquiring 30 frames of images as a cycle, and the illustration is based on image data acquired within one cycle.

[0057] The central processing unit receives the multimodal fusion dataset output by S1. In the multimodal fusion dataset, CT images are classified as one type of image, and MRI images are classified as another type of image. This embodiment uses an image from one perspective as an example for illustration. Images from other perspectives can be processed using the method provided in this embodiment.

[0058] The following embodiment uses one type of image as an example for explanation. Other types of images can be processed using the method provided in this embodiment.

[0059] For each type of image, optical flow is used to calculate the motion vector of each pixel in every two adjacent frames. The motion vector is used to reflect the motion characteristics of the pixel. Optical flow is an existing technology and will not be elaborated on further here.

[0060] The Shi-Tomasi corner detection algorithm is used to detect the feature corners in the mask image corresponding to the image. The feature corners obtained at this time are matched one-to-one with the points in the image, that is, the feature corners in the image are obtained.

[0061] The different adjacent fixed structures in different anatomical regions of the left atrium lead to significant differences in the amplitude and degree of motion during the cardiac cycle.

[0062] The correlation coefficient between two adjacent motion vectors of each feature corner point is calculated. Based on this correlation coefficient, the motion correlation value corresponding to each feature corner point is determined. Specifically, the average of all correlation coefficients corresponding to each feature corner point is taken as the motion correlation value for each feature corner point. The larger the motion correlation value, the more continuous the motion of adjacent frame corner points within the period, consistent with cardiac physiological characteristics. In this embodiment, the correlation coefficient is the autocorrelation coefficient. The standard deviation of the magnitude of all motion vectors of each feature corner point is calculated, and the sum of this standard deviation and the preset zero-prevention parameter is recorded as the first sum. The ratio of the motion correlation value of each feature corner point to the first sum is determined as the comprehensive consistency index of each feature corner point. The larger the comprehensive consistency index, the more regular the motion of the corresponding corner point within a period. In this embodiment, the preset zero-prevention parameter is 0.001. In specific applications, the implementer can set it according to specific circumstances.

[0063] The left atrium is usually divided into an active contraction zone and a passive displacement zone according to its anatomical function. The distribution of myocardial fibers and the neural regulation mechanism of the two types of zones are different, resulting in significant differences in local motor coordination.

[0064] The Pearson correlation coefficient between each feature corner point and the motion vector of each feature corner point in its neighborhood is calculated. The average of these Pearson correlation coefficients is taken as the motion coordination coefficient for each feature corner point. The motion coordination coefficient reflects the degree of coordination between the corner point and its surrounding structure during motion, and is used to distinguish between active contraction regions and passive displacement regions. The size of the neighborhood can be set manually, which will not be elaborated further here.

[0065] For any feature corner point, the magnitude of its motion vector is normalized. The normalized value of the motion vector's magnitude, the comprehensive consistency index of the feature corner point, and the motion coordination coefficient of the feature corner point constitute the feature vector of that feature corner point. In this embodiment, the magnitude of the motion vector is normalized using a maximum-minimum normalization method. However, other existing data normalization methods can also be used in other implementations.

[0066] Based on the similarity between the feature vectors of all feature corner points in each image class, the K-means clustering algorithm is used to cluster the feature corner points. The value of k during clustering is 3, and all feature corner points are divided into three classes. The K-means clustering algorithm is an existing clustering algorithm, and will not be described in detail here.

[0067] The larger the magnitude of the eigenvector, the more stable the motion and the stronger the neighborhood coordination. The more likely the corresponding location is a key anatomical region, such as the pulmonary vein opening or the root of the left atrial appendage. A moderate magnitude of the eigenvector indicates regular motion and weak neighborhood coordination. The corresponding location is more likely to be the tip of the left atrial appendage or the atrial wall contraction zone. A smaller magnitude of the eigenvector indicates random motion and no neighborhood coordination.

[0068] Based on the above characteristics, the average magnitude of the feature vectors of each type of feature corner point is calculated. The feature corner points with the largest average magnitude of the feature vectors are designated as structural critical corner points, the feature corner points with the smallest average magnitude of the feature vectors are designated as noise artifact corner points, and the other type of feature corner points, excluding structural critical corner points and noise artifact corner points, are designated as dynamic response corner points.

[0069] Thus far, using the above method, the feature corner points have been classified, noise artifact corner points have been removed, and the cluster of key structural corner points and dynamic response corner points have been retained as the core corner point set for subsequent cross-modal feature association. It should be noted that only the key structural corner points and dynamic response corner points will be analyzed and matched in the subsequent analysis.

[0070] Step S3: Evaluate the cross-modal fusion reliability of each feature corner point based on the stability of the grayscale distribution of pixels in the neighborhood of each feature corner point in all CT images and the motion characteristics of the corner points in the neighborhood of each feature corner point; Based on the distribution and feature vector of the cross-modal fusion reliability of feature corner points in each type of corner point in CT images, determine the matching results of each feature corner point in CT images in MRI images according to priority.

[0071] The central processing unit calls the purified structural key corner point and dynamic response corner point data in the left atrial mask and maps all feature corner points to the CT image space and MRI image space.

[0072] Under normal circumstances, there are significant physiological differences in the structural rigidity of different anatomical regions of the left atrium: the pulmonary vein opening and the root of the left atrial appendage are adjacent to rigid structures such as the aorta and pulmonary veins or fibrous rings, and their local geometry does not change significantly during the cardiac cycle; while the tip of the left atrial appendage and the systolic zone of the anterior wall of the atrium are composed only of myocardial tissue and will undergo significant deformation with the cardiac cycle.

[0073] The following embodiment uses a single feature corner point as an example for explanation. Other feature corner points can be processed using the method provided in this embodiment.

[0074] For any characteristic corner point in a CT image:

[0075] The grayscale mean of all pixels in the neighborhood of the feature corner point in each CT image is obtained and recorded as the first grayscale mean of the feature corner point in each CT image. The variance of the first grayscale mean of the feature corner point in all CT images is calculated, and the sum of the variance and the preset zero-prevention parameter is calculated and recorded as the second sum. The reciprocal of the second sum is used as the local structural stability index of the feature corner point. The smaller the variance of the first grayscale mean of the feature corner point in all CT images, the smaller the grayscale fluctuation and the stronger the structural rigidity of the local structure where the feature corner point is located during the cardiac cycle, and the greater the local structural stability index of the feature corner point.

[0076] Feature points with larger local structural stability indices are more likely to be rigid or low-deformation tissues of the left atrial structure (such as the junction of the pulmonary vein opening and the atrium). These structurally stable features exhibit consistent gray-scale fluctuation patterns across modalities and can serve as anchoring references for cross-modal matching. Feature points with smaller local structural stability indices are more likely to be high-deformation tissues of the left atrial structure (such as the free wall at the tip of the left atrial appendage). Further verification of matching reliability is needed by combining motion patterns to avoid misassociations of cross-modal features due to structural deformation.

[0077] Normally, the movement of the left atrial myocardium exhibits strict physiological synergy, meaning that myocardial fibers in the same anatomical region are regulated by the same neural pathways, and their movement direction and amplitude are highly synchronized.

[0078] Furthermore, for any feature corner point, the average vector of the motion vectors of all pixels in the neighborhood of that feature corner point in each CT image is calculated. The cosine similarity between the motion vector of the feature corner point in each CT image and the corresponding average vector is used to quantify the directional consistency between the motion of a single feature corner point and the overall motion of the region. The average value of the cosine similarity of the feature corner point in all CT images is used as the motion representativeness coefficient of the feature corner point. The larger the motion representativeness coefficient, the stronger the coordination between the motion of the feature corner point and the overall motion of the region, which conforms to the physiological motion law of myocardium and can be used as an effective sample for cross-modal motion feature association. The smaller the motion representativeness coefficient, the more the motion of the feature corner point deviates from the overall law of the region, and its fusion priority needs to be reduced.

[0079] Next, by combining the local structural stability index and the motion representativeness coefficient, a quantitative index is constructed to characterize the cross-modal fusion reliability of the feature corner points.

[0080] Specifically, the product of the local structural stability index of the feature corner point and the motion representativeness coefficient of the feature corner point is determined as the cross-modal fusion reliability factor of the feature corner point. The cross-modal fusion reliability factor is used to reflect the cross-modal fusion reliability. The stronger the structural stability and motion coordination of the feature corner point, the higher the reliability of cross-modal feature matching, the higher the fusion confidence, that is, the larger the cross-modal fusion reliability factor.

[0081] The root cause of topological errors (holes, breaks) in traditional 3D reconstruction methods lies in erroneous feature matching during multimodal image fusion. Therefore, this embodiment only allows matching of corner points of the same type across different modal images.

[0082] The following example uses any one type of corner point from the structural critical corner points or dynamic response corner points in CT images for illustration.

[0083] Specifically, for any type of corner point among structural critical corner points or dynamic response corner points in CT images:

[0084] In descending order of cross-modal fusion reliability factor, all feature corner points of this type of corner point are registered with feature corner points of the same type in the MRI image. Any feature corner point in this type of corner point is recorded as the corner point to be matched. During registration, the initial matching radius is corrected using the cross-modal fusion reliability factor of the corner point to be matched, and the corrected matching radius of the corner point to be matched is obtained. The cross-modal fusion reliability factor and the corrected matching radius are negatively correlated.

[0085] As a concrete example, the specific formula for calculating the corrected matching radius is given. The corrected matching radius of the corner point to be matched can be expressed as:

[0086] ;

[0087] in, This represents the matching radius after the corner point to be matched has been corrected. Indicates the initial matching radius. represents the cross-modal fusion reliability factor of the corner points to be matched, and e represents the natural constant.

[0088] The initial matching radius can be set to 3-5mm. In specific applications, the implementer can set it according to the specific situation. This embodiment will not elaborate further.

[0089] The higher the cross-modal fusion reliability factor of the corner point to be matched, the more likely it corresponds to a structurally complex and clinically critical region, such as the pulmonary vein opening. The smaller the value, the smaller the corrected matching radius, avoiding the introduction of adjacent non-corresponding corner points due to an excessively large matching range, thus reducing the risk of mismatch. The smaller the cross-modal fusion reliability factor of the corner point to be matched, the more likely it is to correspond to a region with large deformation, such as the tip of the left atrial appendage. The larger the corrected matching radius, the more likely it is to capture the spatial offset corner points caused by deformation.

[0090] Using the method described above, the corrected matching radius of each feature corner point in this class can be calculated. Next, based on the matching radius, it is matched with corner points of the same class in the MRI image.

[0091] Specifically, taking the corner point to be matched in the CT image as the center, all feature corner points in the same type of corner points in the MRI image whose spatial coordinates are within the corrected matching radius are selected to form a candidate corner point set for the corner point to be matched. It should be noted that if there is no feature corner point within the corrected matching radius, the initial matching radius is increased by 10%, the corrected matching radius is recalculated, and the candidate corner point set is determined again until the candidate corner point set contains at least one feature corner point to ensure matching continuity and avoid model breakage.

[0092] Calculate the straight-line distance in three-dimensional space between the corner point to be matched in the CT image and each corner point in the candidate corner point set, as well as the Euclidean distance between the feature vectors of the corner point to be matched in the CT image and each corner point in the candidate corner point set.

[0093] Next, by combining the straight-line distance between the corner to be matched and each corner in the candidate corner set in three-dimensional space and the Euclidean distance between the corner to be matched and the feature vectors of each feature corner in the candidate corner set, the matching point of the corner to be matched in the MRI image is determined. Specifically, the product of the straight-line distance between the corner to be matched and each feature corner in the candidate corner set and the corresponding Euclidean distance is used as the difference quantification index between the corner to be matched and each feature corner in the candidate corner set. The feature corner in the candidate corner set corresponding to the smallest difference quantification index is used as the matching point of the corner to be matched in the MRI image.

[0094] The above methods enable the matching of key structural corners in CT images with key structural corners in MRI images, as well as the matching of dynamic response corners in CT images with dynamic response corners in MRI images.

[0095] Step S4: Construct a three-dimensional model of the left atrium based on the matching results.

[0096] In this embodiment, the dynamic response corner points in CT images and MRI images were matched in the above steps, and multiple matching point pairs were obtained.

[0097] The central processing unit calls the matching point pair and combines it with the three-dimensional coordinate system constructed in step S1 with the center point of the left atrium as the origin to obtain the CT image set and the MRI image set.

[0098] Based on epipolar geometry constraints, a triangulation algorithm is used to convert the feature corner points in CT images and the matching point pairs in MRI images into three-dimensional spatial coordinates, thus obtaining a set of three-dimensional corner points of the left atrium. The three-dimensional corner point set is then subjected to Delaunay triangulation, and the triangulation mesh is verified and corrected in combination with the anatomical topological features of the left atrium (such as closed cavity structures), finally generating a three-dimensional model of the left atrium.

[0099] Thus, the construction of the three-dimensional model of the left atrium has been completed using the method provided in this embodiment.

[0100] This embodiment first extracts feature vectors of feature corner points from the motion characteristics of feature corner points in continuous multi-frame CT and MRI images of the left atrial region of the target object from different perspectives. Based on these feature vectors, the feature corner points are classified. The cross-modal fusion reliability of feature corner points is dynamically evaluated based on the stability of the grayscale distribution of pixels in the neighborhood of each feature corner point in all CT images and the motion characteristics of the neighborhood corner points. Then, based on the distribution and feature vectors of the cross-modal fusion reliability of feature corner points in each class of corner points in the CT images, the matching results of each feature corner point in the CT images in the MRI images are determined according to priority, establishing an accurate cross-modal corner point correspondence. Based on the matching results, a three-dimensional model of the left atrium is constructed, achieving high-precision and highly dynamic consistent three-dimensional reconstruction of the left atrium, providing reliable three-dimensional model data for cardiac interventional surgery.

[0101] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional reconstruction system for the left atrium integrating multimodal images, characterized in that, The system includes a memory and a processor. The processor executes a computer program stored in the memory to perform the following steps: acquiring multiple consecutive frames of CT and MRI images of the left atrial region of the target object from different perspectives; determining the feature vector of each feature corner point by combining the motion characteristics of each feature corner point in each type of image and the similarity of the motion characteristics of each feature corner point with its neighboring feature corner points; classifying the feature corner points in each type of image according to the feature vectors to obtain various types of corner points; evaluating the cross-modal fusion reliability of each feature corner point based on the stability of the gray-level distribution of pixels in the neighborhood of each feature corner point in all CT images and the motion characteristics of the neighborhood corner points of each feature corner point; determining the matching results of each feature corner point in the CT images in the MRI images according to priority based on the distribution and feature vectors of the cross-modal fusion reliability of feature corner points in each type of corner point in the CT images; and further classifying the feature corner points in each type of corner point in the CT images according to the feature vectors. The matching results are used to construct a three-dimensional model of the left atrium. The evaluation of the cross-modal fusion reliability of each feature corner point includes: for any feature corner point: obtaining the gray-scale mean of all pixels in the neighborhood of each feature corner point in each CT image, denoted as the first gray-scale mean of the feature corner point in each CT image; calculating the variance of the first gray-scale mean of the feature corner point in all CT images, and obtaining a local structural stability index of the feature corner point based on the variance; obtaining a motion representativeness coefficient of the feature corner point based on the similarity between the motion vector of the feature corner point in each image and the motion representativeness coefficient of its neighborhood pixels; and combining the local structural stability index and the motion representativeness coefficient to obtain a cross-modal fusion reliability factor for the feature corner point, wherein the cross-modal fusion reliability factor is used to reflect the cross-modal fusion reliability.

2. The left atrium three-dimensional reconstruction system based on multimodal imaging according to claim 1, characterized in that, The process of determining the feature vector of each feature corner point by combining the motion characteristics of each feature corner point in each image class and the similarity of the motion characteristics of each feature corner point with the feature characteristics of its neighboring feature corner points includes: for each image class, calculating the motion vector of each feature corner point in each two adjacent CT images using optical flow method, the motion vector being used to reflect the motion characteristics of the feature corner point; determining the motion correlation value corresponding to each feature corner point based on the correlation coefficient between two adjacent motion vectors of each feature corner point; obtaining the comprehensive consistency index of each feature corner point based on the standard deviation of the magnitude of the motion vector of each feature corner point and the motion correlation value; obtaining the motion coordination coefficient corresponding to each feature corner point based on the Pearson correlation coefficient between the motion vector of each feature corner point and the feature vector of its neighboring feature corner points; the normalized value of the magnitude of the motion vector of each feature corner point, the comprehensive consistency index, and the motion coordination coefficient constitute the feature vector of each feature corner point.

3. The left atrium three-dimensional reconstruction system based on multimodal imaging according to claim 2, characterized in that, The step of obtaining a comprehensive consistency index for each feature corner point based on the standard deviation of the magnitude of the motion vector of each feature corner point and the motion correlation value includes: calculating the first sum of the standard deviation of the magnitude of the motion vector of each feature corner point and a preset zero-prevention parameter; and determining the ratio of the motion correlation value to the first sum as the comprehensive consistency index for each feature corner point.

4. The left atrium three-dimensional reconstruction system based on fused multimodal images according to claim 1, characterized in that, The step of classifying feature corner points in each type of image based on the feature vectors to obtain various types of corner points includes: dividing all feature corner points into three categories based on the similarity between the feature vectors; taking the feature corner points of the category with the largest average modulus of the feature vectors as structural critical corner points; taking the feature corner points of the category with the smallest average modulus of the feature vectors as noise artifact corner points; and taking the other type of feature corner points, excluding structural critical corner points and noise artifact corner points, as dynamic response corner points.

5. The three-dimensional reconstruction system for the left atrium based on multimodal imaging according to claim 1, characterized in that, The step of obtaining the motion representativeness coefficient of any feature corner point based on the similarity between the motion vector of any feature corner point in each image and the motion vector of its neighboring pixels includes: calculating the average vector of the motion vectors of all pixels in the neighborhood of any feature corner point in each CT image; calculating the cosine similarity between the motion vector of any feature corner point in each CT image and the corresponding average vector; and taking the average value of the cosine similarity of any feature corner point in all CT images as the motion representativeness coefficient of any feature corner point.

6. The left atrium three-dimensional reconstruction system based on multimodal imaging according to claim 1, characterized in that, The step of combining the local structural stability index and the motion representativeness coefficient to obtain the cross-modal fusion reliability factor of any feature corner point includes: determining the cross-modal fusion reliability factor of any feature corner point by multiplying the local structural stability index and the motion representativeness coefficient.

7. The three-dimensional reconstruction system for the left atrium based on multimodal imaging according to claim 1, characterized in that, The method, based on the distribution and feature vectors of cross-modal fusion reliability of feature corner points in each type of corner point in CT images, determines the matching results of each feature corner point in CT images in MRI images according to priority. This includes: for any type of corner point among structural critical corner points or dynamic response corner points in CT images: registering all feature corner points of that type of corner point with feature corner points of the same type in MRI images in descending order of cross-modal fusion reliability factors; during registration, correcting the initial matching radius using the cross-modal fusion reliability factor of the corner point to be matched to obtain the corrected matching radius of the corner point to be matched. The cross-modal fusion reliability factor is negatively correlated with the corrected matching radius. Taking the corner point to be matched in the CT image as the center, all feature corner points in the same type of corner points in the MRI image whose spatial coordinates are within the range of the corrected matching radius are selected to form a candidate corner point set for the corner point to be matched. The straight-line distance and the Euclidean distance between the corner point to be matched in the CT image and each feature corner point in the candidate corner point set are calculated in three-dimensional space. The matching point of the corner point to be matched in the MRI image is determined by combining the straight-line distance and the Euclidean distance. The corner point to be matched is any feature corner point in any type of corner point.

8. The left atrium three-dimensional reconstruction system fused with multimodal images according to claim 7, characterized in that, The step of combining the straight-line distance and the Euclidean distance to determine the matching point of the corner to be matched in the MRI image includes: taking the product of the straight-line distance and the corresponding Euclidean distance between the corner to be matched and each feature corner in the candidate corner set as a difference quantification index between the corner to be matched and each feature corner in the candidate corner set; and taking the feature corner in the candidate corner set corresponding to the smallest difference quantification index as the matching point of the corner to be matched in the MRI image.

9. The three-dimensional reconstruction system for the left atrium based on multimodal imaging according to claim 1, characterized in that, The construction of a three-dimensional model of the left atrium based on the matching results includes: based on epipolar geometric constraints, using a triangulation algorithm, converting the matching results of feature corner points in CT images and feature corner points in MRI images into three-dimensional spatial coordinates to obtain a set of three-dimensional corner points of the left atrium; performing Delaunay triangulation on the set of three-dimensional corner points, and verifying and correcting the mesh in combination with the anatomical topological features of the left atrium to generate a three-dimensional model of the left atrium.

Images