A cross-dimensional vessel non-rigid registration method and system based on feature matching

Abstract

The application discloses a cross-dimension vessel non-rigid registration method based on feature matching, comprising the following steps: based on the anatomical characteristics of different vessel parts, complex deformation is decoupled into global macroscopic displacement and local microscopic disturbance and fused; based on the vessel centerline as a processing object, a double-flow feature extraction pipeline is constructed, preoperative features are extracted through a 3D point cloud backbone network, preoperative projection and intraoperative features are extracted through a 2D image backbone network; through a cascaded attention mechanism including intra-modal self-attention, intra-modal cross-attention mechanism and inter-modal cross-attention mechanism; the cross-dimension registration problem is decomposed into learning of anatomical consistency and deformation invariance, and a high-dimensional joint feature of fused 3D space structure and 2D deformation prior is output; a high-confidence 3D point-2D pixel corresponding relationship is calculated; a global non-rigid deformation field is obtained, and high-precision non-rigid planar alignment of preoperative anatomical projection and real intraoperative angiography is realized. The application also discloses a system, an electronic device and a computer readable storage medium.

Description

Technical Field

[0001] This invention relates to the field of medical image registration technology, specifically to a cross-dimensional non-rigid vascular registration method and system based on feature matching. Background Technology

[0002] Currently, cardiovascular and cerebrovascular diseases remain the leading cause of death worldwide. For these diseases, minimally invasive vascular interventional procedures are widely used clinically, including percutaneous coronary intervention, cerebral thrombectomy or embolization, and peripheral vascular intervention. During these procedures, surgeons primarily rely on intraoperative XA or DSA images for navigation. However, these images are 2D projection images, which suffer from overlapping vessels and missing depth information, increasing the difficulty for surgeons in judging the spatial structure of blood vessels. In contrast, preoperative 3D vascular imaging data (such as CTA or MRA) contains complete 3D anatomical information. Therefore, obtaining cross-dimensional spatial mapping and planar alignment relationships between preoperative 3D vascular data and its projection and intraoperative 2D angiographic images through matching and registration algorithms, thereby providing preoperative topological prior information support for 2D images, is an important technical means to achieve intelligent surgical navigation. Currently, existing technologies for registering 3D vascular projections with 2D angiographic images mainly include the following solutions: (1) Rigid Registration: The most widely adopted strategy in this type of method is to treat registration as an iterative optimization problem with 6 degrees of freedom (6-DoF), including 3 rotation parameters and 3 translation parameters. The specific process is usually as follows: using algorithms such as ray tracing to generate digitally reconstructed radiographs (DRR) of the preoperative 3D vessel volume, and then continuously adjusting the 6-DoF parameters to minimize the hand-designed similarity measure (such as similarity based on pixel intensity, geometry or gradient features) between the preoperative DRR projection and the intraoperative real 2D angiographic image.

[0003] (2) Non-rigid Registration: Due to the severe deformation of blood vessels caused by heartbeat, breathing, or instrument insertion, rigid registration cannot meet the accuracy requirements, so non-rigid registration is introduced. Traditional iterative methods include Coherent Point Drift (CPD), Bayesian Coherent Point Drift (BCPD), or Support Vector Regression (SVR). The implementation scheme of these methods is to extract the preoperative DRR projection and the real 2D blood vessels during the operation into a feature point set, and model the point set as a Gaussian mixture model with equal probability distribution. The deformation field is solved by minimizing the distribution difference through the expectation-maximization algorithm; or by using Thin Plate Splines (TPS) interpolation and its variants TPS-L2, TPS-KC, etc. for smooth deformation estimation; or by directly estimating deformation through deep learning methods. In addition, there are cross-modal registration learning methods based on feature matching. In recent years, with the development of deep learning and computer vision technologies, cross-modal registration methods based on feature matching have gradually become a research hotspot. These methods typically extract feature representations from different modal data through neural networks, establish correspondences through feature matching, and then estimate registration parameters. For example, some methods use joint modeling of image features and point cloud features, and achieve cross-modal information interaction through attention mechanisms or transformer structures. However, existing methods are mainly applied to scenarios such as autonomous driving, which assume a rigid environment and relatively regular target structures. In medical vascular intervention scenarios, vascular structures have complex topological relationships and are accompanied by significant non-rigid deformations, making the above methods difficult to apply directly. Therefore, how to achieve stable cross-dimensional feature matching under complex non-rigid deformation conditions still requires further research.

[0004] In image-guided panvascular interventional procedures (percutaneous coronary intervention, cerebrovascular and neurovascular intervention, peripheral vascular intervention, etc.), when performing cross-dimensional non-rigid registration between preoperative high-precision 3D vascular data, such as computed tomography angiography (CTA) or magnetic resonance angiography (MRA), and intraoperative real-time 2D vascular image data, such as X-ray angiography (XA) or digital subtraction angiography (DSA), existing technologies struggle to balance accuracy, deformation modeling capabilities, computational efficiency, and data dependency. Specifically, existing technologies suffer from at least the following technical deficiencies: (1) Domain difference between preoperative and intraoperative images: Existing deep learning-based registration and matching techniques have serious mechanistic defects in cross-dimensional scenarios. Existing registration can be divided into three technical routes: 2D-2D registration, 3D-3D registration, and 3D-2D registration. Among them, the 2D-2D registration scheme usually projects the preoperative 3D volume data into 2D digitally reconstructed radiographic images (DRR) before registration. This dimensionality reduction process leads to serious loss of depth information. The alignment effect is unstable in complex anatomical structures such as overlapping blood vessels and dense branches. Simply projecting the preoperative 3D data into 2D space and then performing pure 2D-2D feature matching will completely lose the depth information and the anatomical geometric constraints of 3D space, making the matching very prone to errors. The existing 3D-3D registration requires real-time reconstruction of intraoperative 2D vascular images into intraoperative 3D vascular volume data and then registration with preoperative 3D volume data. However, most intraoperative images are currently acquired by a single C-arm, and it is difficult to reconstruct monocular 2D images lacking depth information into 3D volume data. Finally, for 3D-2D registration schemes, existing 3D-2D cross-dimensional feature matching methods (such as matching schemes based on autonomous driving radar and cameras) are mostly designed for rigid body scenes. They fail to design specific network mechanisms for complex non-rigid physiological deformations caused by breathing, heartbeat, or instrument insertion in medical scenarios. They cannot maintain the robustness of the correspondence under deformation states and cannot be transferred across boundaries to handle the complex cross-dimensional registration problem of blood vessels in medical images. Therefore, how to achieve stable cross-dimensional registration between preoperative and intraoperative images while performing cross-dimensional feature matching is an urgent technical problem to be solved.

[0005] (2) Insufficient ability to model complex non-rigid deformations: Human blood vessels undergo extremely complex non-rigid deformations during surgery. For example, in the coronary artery scenario, there are periodic deformations caused by myocardial contraction and patient respiration; in the cerebrovascular scenario, there are the effects of unconscious micro-movements of the patient's head and cerebrospinal fluid pulsation; and in various vascular interventions, the physical traction and local morphological distortion caused by surgical instruments (such as guidewires, microcatheters, stents, etc.) when pushed into the blood vessel. Most existing registration algorithms are based on rigid assumptions or on non-rigid, manually designed similarity metrics (such as intensity, geometry, or gradient features), which are difficult to accurately characterize the above-mentioned complex deformations and easily lead to local inconsistencies in the alignment results. Therefore, how to effectively model the complex non-rigid deformations of blood vessels is an important technical problem that needs to be solved.

[0006] (3) The problem of computational efficiency failing to meet the real-time requirements of intraoperative surgery: Existing registration methods based on preoperative dense 3D volumetric data mostly rely on multimodal volumetric rendering or high-frequency iterative optimization calculations, which are extremely costly and time-consuming. In actual interventional surgery, image data needs to be dynamically updated with the surgical process, and existing methods cannot output registration results in a timely manner, thus affecting clinical applications. Therefore, how to improve computational efficiency to meet the real-time application requirements of intraoperative surgery while ensuring registration accuracy is an urgent technical problem to be solved.

[0007] (4) The problem of lack of high-quality cross-modal pairing data: Existing deep learning-based registration models rely heavily on perfectly paired labeled data for training. However, in the clinical setting, due to changes in patient positioning, dynamic distribution of contrast agents, and the uncontrollability of the surgical process, it is difficult to obtain precisely aligned preoperative and intraoperative cross-dimensional pairing data. In real medical imaging clinical workflows, due to differences in patient positioning, changes in contrast angles, and real-time physiological movements, obtaining pixel-level precisely aligned cross-modal pairing data is physically impractical. The difficulty in data acquisition limits model training, and existing random data augmentation methods make deep learning registration models prone to overfitting, making it difficult to generalize to unseen real clinical intraoperative data, thus affecting their generalization ability in actual clinical scenarios. Therefore, how to perform self-supervised training is a key technical problem that needs to be solved. Summary of the Invention

[0008] The purpose of this invention is to provide a cross-dimensional non-rigid vascular registration method and system based on feature matching. This invention abandons the traditional "dimensionality reduction image registration" and "dense iterative optimization" approach, and reconstructs "cross-dimensional non-rigid vascular registration" into a new computational paradigm of "high-dimensional feature matching of sparse centerline points to pixels + control point-driven non-rigid deformation estimation".

[0009] A first aspect of the present invention provides a cross-dimensional non-rigid vascular registration method based on feature matching, comprising: S1, based on the anatomical characteristics of different blood vessel sites, decouples complex deformation into global macroscopic displacement and local microscopic disturbance and then fuses them; S2, a dual-flow feature extraction pipeline is constructed based on the vascular centerline as the processing object, and preoperative features are extracted through a 3D point cloud backbone network based on the dual-flow feature extraction pipeline, and preoperative projection and intraoperative features are extracted through a 2D image backbone network; wherein, the vascular centerline is sparse and contains rich topological information; S3, through a cascaded attention mechanism, introduces the projection space as a bridge to decompose the cross-dimensional registration problem into the learning of anatomical consistency and deformation invariance, and finally outputs a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior; wherein the cascaded attention mechanism includes intramodal self-attention, intramodal cross attention mechanism and intermodal cross attention mechanism. S4, calculate the final point-voxel matching relationship based on the high-dimensional joint features; the point-voxel matching relationship is used to characterize the high-confidence 3D point-2D pixel correspondence relationship, and obtain the spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure; wherein, the 3D point-2D pixel correspondence relationship includes the 3D point-3D point projection correspondence relationship and the 3D point projection-2D pixel correspondence relationship; S5. Based on the high-confidence 3D point projection-2D pixel correspondence, a global non-rigid deformation field is obtained, and based on the global non-rigid deformation field, high-precision non-rigid plane alignment between the preoperative anatomical projection and the intraoperative real angiographic image is achieved.

[0010] Preferably, S1 includes: S11 simulates global macroscopic motion, thereby simulating the macroscopic displacement caused by changes in the patient's vital signs or body position. S12, simulate local micro-random perturbations to simulate the physical traction, straightening and local morphological distortion of the blood vessel wall when surgical instruments are pushed into the blood vessel lumen; the simulation of local micro-random perturbations includes using continuous spatial noise to generate a spatially continuous random deformation field to apply perturbation to the blood vessel and model the local non-rigid deformation during the operation. S13, Based on the anatomical spatial anchoring weight mechanism, the global macroscopic motion and local microscopic random disturbance are fused to form a composite deformation field; the anatomical spatial anchoring weight mechanism includes: assigning different deformation weights according to the anatomical anchor points of different blood vessels, thereby fusing the global macroscopic motion and local microscopic random disturbance.

[0011] Preferably, the 2D image backbone network is any image feature extractor based on convolutional neural networks or visual Transformers; and the 3D point cloud backbone network is any encoder based on convolutional neural networks or visual Transformers, or any encoder capable of capturing local and global geometric features in 3D space.

[0012] Preferably, S2 includes: S21, based on 2D image feature flow, utilizes 2D image backbone network to extract pixel-level features from 2D projection images containing preoperative 3D centerline and real intraoperative 2D angiography images to form target feature set; S22, based on 3D point cloud feature flow, utilizes a 3D point cloud backbone network to directly extract point-level features from the preoperative 3D vascular centerline spatial coordinates to form a source feature set.

[0013] Preferably, S3 includes: S31, based on the intramodal self-attention mechanism, combined with relative position encoding, self-attention calculation is performed inside the 2D image feature stream and the 3D point cloud feature stream respectively, thereby enhancing the geometric topological structure features and contextual semantic learning within a single modality; S32, based on the intramodal cross-attention mechanism, in the 2D image feature flow, the 2D projection features of the preoperative 3D blood vessels and the real intraoperative 2D angiography features are encouraged to interact with each other through cross-attention. S33, based on the intermodal cross-attention mechanism, the features processed by the 2D image feature stream and carrying 2D deformation invariance information are exchanged with the spatial features in the 3D point cloud feature stream through bidirectional cross-attention information exchange, thereby finally outputting a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior.

[0014] Preferably, the calculation of the final point-voxel matching relationship based on the high-dimensional joint features includes: calculating the final point-voxel matching relationship based on the high-dimensional joint features using a double normalization exponential function combined with a rejection absorption node mechanism, characterizing a high-confidence 3D point-2D pixel correspondence, and obtaining a spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure, including: S41, establish an allocation probability matrix including an additional rejection absorption node mechanism; wherein, the rejection absorption node mechanism is used to absorb unmatchable free points caused by the common problems of blood vessel occlusion, overlap or segmentation defects in cross-dimensional matching. S42, by jointly calculating and outputting the partial allocation probability matrix through the double normalized exponential function and the matching score predicted by the network; S43, based on the partial allocation probability matrix, obtain the point-voxel matching relationship, characterize the high-confidence 3D point-2D pixel correspondence, and obtain the spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure. The high-confidence 3D point-2D pixel correspondence includes the 3D point-3D point projection correspondence and the 3D point projection-2D pixel correspondence.

[0015] Preferably, the step of calculating the final point-voxel matching relationship based on the high-dimensional joint features includes: calculating the final point-voxel matching relationship based on the high-dimensional joint features using the Sinkhorn algorithm, an allocation algorithm based on optimal transport theory, or a graph matching algorithm.

[0016] Preferably, S5 includes: S51, Receive the high-confidence 3D point projection-2D pixel correspondence as a control point; S52, solve the global non-rigid deformation field based on the mathematical model of the control points and the non-rigid deformation solver.

[0017] Preferably, step S52 includes: using the control points to drive a thin-plate spline plane interpolation algorithm to calculate the global non-rigid deformation field; or, using the control point displacements of high-confidence matching pairs output by the network as input parameters to directly drive a B-spline model or a free-form deformation (FFD) model for deformation smoothing; or, cascading a lightweight multilayer perceptron network at the back end of the registration network corresponding to the registration method to regress a dense deformation field end-to-end as the global non-rigid deformation field. S53, the global non-rigid deformation field is applied to the entire preoperative 3D vascular projection to achieve high-precision non-rigid alignment between the preoperative anatomical projection and the real intraoperative angiographic image.

[0018] A second aspect of the present invention provides a cross-dimensional non-rigid vascular registration system based on feature matching for implementing the method of the first aspect, comprising: A generalized deformation generation module is used to decouple complex deformations into global macroscopic displacements and local microscopic perturbations based on the anatomical characteristics of different blood vessel locations, and then fuse them. The dual-flow feature extraction module is used to construct a dual-flow feature extraction pipeline based on the vessel centerline as the processing object, and to extract preoperative features through a 3D point cloud backbone network based on the dual-flow feature extraction pipeline, and to extract preoperative projection and intraoperative features through a 2D image backbone network; wherein the vessel centerline is sparse and contains rich topological information; The attention-based cross-dimensional information interaction module is used to introduce the projection space as a bridge through a cascaded attention mechanism, decompose the cross-dimensional registration problem into the learning of anatomical consistency and deformation invariance, and finally output a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior; wherein the cascaded attention mechanism includes intramodal self-attention, intramodal cross attention mechanism and intermodal cross attention mechanism. The matching probability estimation and spatial mapping module is used to calculate the final point-voxel matching relationship based on the high-dimensional joint features; the final point-voxel matching relationship is used to characterize the high-confidence 3D point-2D pixel correspondence relationship and obtain the spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure; the high-confidence 3D point-2D pixel correspondence relationship includes the 3D point-3D point projection correspondence relationship and the 3D point projection-2D pixel correspondence relationship.

[0019] The global non-rigid deformation solving module is used to obtain the global non-rigid deformation field based on the high-confidence 3D point projection-2D pixel correspondence, and to achieve high-precision non-rigid plane alignment between the preoperative anatomical projection and the intraoperative real angiographic image based on the global non-rigid deformation field.

[0020] A third aspect of the present invention provides an electronic device including a processor and a memory, the memory storing a plurality of instructions, the processor being configured to read the instructions and execute the method as described in the first aspect.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium storing a plurality of instructions which can be read by a processor and executed as described in the first aspect.

[0022] The beneficial effects of the method and system of the present invention are as follows: (1) Overcoming the differences between cross-dimensional domains, achieving extremely high non-rigid registration accuracy and robustness: Existing technologies are easily affected by vessel overlap, occlusion, and contrast agent artifacts when processing intraoperative angiography (XA / DSA), leading to topological tearing. This invention abandons the traditional direct multimodal comparison scheme and innovatively introduces the "projection space" bridge representation. Since this architecture cleverly decouples the complex 3D-2D correspondence into two sub-relationships, "anatomical consistency" and "deformation invariance," it not only successfully preserves the preoperative 3D topological prior but also accurately captures the real 2D deformation information during the operation. This deep integration of high-dimensional semantics and spatial geometry enables this invention to effectively combat severe intraoperative vessel occlusion and overlap, eliminate the problem of traditional iterative methods easily getting trapped in local optima, and achieve extremely high-precision cross-dimensional non-rigid vessel registration.

[0023] (2) Significantly reduced computational complexity, perfectly meeting the real-time requirements of clinical intervention: Existing registration methods based on preoperative 3D volume heavily rely on computationally expensive multimodal volume rendering and high-frequency iterative optimization. This invention extracts the extremely sparse but richly topological "vascular centerline" as the processing object and uses forward propagation neural network dual-stream feature matching to directly solve the correspondence. This mechanism eliminates the cumbersome 3D rendering and iteration process, perfectly meeting the stringent real-time requirements for image updates in clinical interventional surgical navigation.

[0024] (3) It eliminates the dependence on real paired clinical data and has excellent generalization ability: Existing deep learning registration models face the fatal bottleneck of the difficulty in obtaining large-scale, high-precision "preoperative-intraoperative" paired label data in real clinical environments. Existing random data augmentation methods make deep learning registration models prone to overfitting and difficult to generalize to unseen real clinical intraoperative data. Relying on the "generalization deformation generation" training module of this invention, the system can continuously generate highly realistic synthetic paired data containing macroscopic physiological movements and microscopic instrument perturbations based on physical and mathematical models. This enables the network to complete training under the condition of "zero" real paired labels, completely breaking the data dependence bottleneck, and demonstrating excellent robustness and zero-shot generalization ability on completely unseen real clinical intraoperative datasets. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of the present invention, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 The flowchart is provided by an embodiment of the present invention for a cross-dimensional non-rigid registration method for blood vessels based on feature matching. Figure 2 This is a schematic diagram of the principle architecture of a cross-dimensional non-rigid vascular registration system based on feature matching according to an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the principle of the 3D point-2D voxel space mapping method provided in an embodiment of the present invention; Figure 4 This is a structural diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0029] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0030] Explanation of terms used in the embodiments: Preoperative imaging: High-precision 3D imaging data of patients acquired before surgery, such as computed tomography angiography (CTA) and magnetic resonance angiography (MRA).

[0031] Intraoperative imaging: 2D images acquired in real time during the operation using a C-arm X-ray machine, such as X-ray angiography (XA) and digital subtraction angiography (DSA).

[0032] Cross-dimensional vascular matching and registration: This process involves spatial mapping and planar alignment of the 3D vascular model and its projection obtained from preoperative reconstructed image segmentation with the 2D vascular model obtained from intraoperative real-time image segmentation, using matching and registration algorithms. This provides surgeons / surgical robots with instrument spatial / planar positioning and surgical path guidance, thereby improving the accuracy and efficiency of surgical operations and effectively reducing the risk of X-ray radiation exposure for patients.

[0033] The specific objectives and technical solutions of this embodiment include: (1) In view of the shortcomings of existing technologies such as local optima and time consumption, the present invention extracts sparse and topologically rich features of the blood vessel centerline and uses deep neural networks to perform feature matching in a high-dimensional semantic space. This reduces the dependence of traditional methods on initial position and manually designed similarity measurement, and fundamentally eliminates local erroneous association and topological tearing caused by blood vessel overlap and artifacts. At the same time, it replaces the heavy volume rendering and iteration with lightweight network forward propagation to achieve real-time high-precision registration that meets the needs of clinical navigation.

[0034] (2) To address the shortcomings of existing technologies that lack prior knowledge and deformation modeling, this embodiment of the invention constructs a unique "projection space" bridge representation and utilizes a cascaded cross-attention mechanism to decouple the extremely complex cross-dimensional correspondence between "preoperative 3D points and intraoperative 2D pixels" into two steps: first, an anatomical consistency association between "preoperative 3D points and preoperative 3D point projections" based on the principle of perspective projection; and second, a deformation-invariant association between "preoperative 3D point projections and intraoperative 2D pixels" based on motion context feature interaction. Thus, stable 3D point-2D pixel matching can still be established even in cross-modal, cross-dimensional situations with severe deformation.

[0035] (3) In view of the shortcomings of existing technologies due to data dependence bottleneck, this invention proposes a generalized deformation generation strategy. By synthesizing a large number of deformation training samples with high clinical simulation through physical and mathematical modeling, the network can completely get rid of its dependence on paired real clinical data and show excellent generalization ability in actual clinical surgery. Example 1

[0036] like Figure 1 As shown, this embodiment provides a cross-dimensional non-rigid vascular registration method based on feature matching, including: decoupling complex deformations into global macroscopic displacements and local microscopic perturbations based on the anatomical characteristics of different vascular sites and fusing them; constructing a dual-flow feature extraction pipeline based on the vascular centerline as the processing object, extracting preoperative features through a 3D point cloud backbone network, and extracting preoperative projections and intraoperative features through a 2D image backbone network; using a cascaded attention mechanism including intramodal self-attention, intramodal cross-attention, and intermodal cross-attention mechanisms; introducing a "projection space" as a bridge to decompose the cross-dimensional registration problem into learning anatomical consistency and deformation invariance and outputting high-dimensional joint features that fuse 3D spatial structure and 2D deformation priors; calculating high-confidence 3D point-2D pixel correspondences (3D point-3D point projection correspondences and 3D point projection-2D pixel correspondences) to achieve spatial mapping between preoperative anatomical structures and intraoperative real angiographic images; obtaining a global non-rigid deformation field and achieving high-precision non-rigid plane alignment between preoperative anatomical projections and intraoperative real angiographic images.

[0037] Specifically, the method in this embodiment includes: S1, based on the anatomical characteristics of different blood vessel sites, decouples complex deformation into global macroscopic displacement and local microscopic disturbance and then fuses them; To overcome the technical bottleneck of obtaining perfect preoperative-intraoperative paired labeled data in clinical settings, this invention proposes a generalized training strategy driven by global physiological deformation and local instrument perturbation in step S1. This strategy can generate preoperative-intraoperative synthetic paired data that conforms to real physical laws and has high anatomical realism, which can then be used to drive the training of the subsequent network.

[0038] In a preferred embodiment, S1 includes: S11 simulates global macroscopic motion, thereby simulating macroscopic displacement caused by changes in the patient's vital signs (such as breathing and heartbeat) or body position.

[0039] In this embodiment, for the coronary artery, the macroscopic displacement is specifically instantiated as a rigid transformation matrix caused by diaphragmatic breathing and a sinusoidal waveform field caused by myocardial contraction; for the liver vessels and peripheral blood vessels of the lower limbs, the displacement is set as rigid offset or stretching deformation along the respiratory axis according to the degree of anatomical fixation of the organs; for neurointervention scenarios such as cerebrovascular diseases, the displacement is simplified to rigid micro-movement offset of the head.

[0040] S12, simulate local micro-random perturbations to simulate the physical traction, straightening and local morphological distortion of the blood vessel wall when surgical instruments (such as guidewires, microcatheters, stents, etc.) are pushed into the blood vessel lumen; the simulation of local micro-random perturbations includes using continuous spatial noise to generate a spatially continuous random deformation field to apply perturbation to the blood vessel to model local non-rigid deformation during the operation. In this embodiment, a spatially continuous random deformation field is generated using continuous spatial noise to perturb the blood vessel. This mechanism is not limited to specific organs and is universally applicable to intraoperative local non-rigid deformation modeling of all types of blood vessels (including coronary arteries, peripheral blood vessels, cerebral blood vessels, etc.).

[0041] S13, Based on the anatomical spatial anchoring weight mechanism, the global macroscopic motion and local microscopic random perturbation are fused to form a composite deformation field; the anatomical spatial anchoring weight mechanism includes: assigning different deformation weights according to the anatomical anchor points of different blood vessels (for example, the coronary artery is fixed at the root of the aorta, and the cerebral blood vessels are fixed at the internal carotid artery or the passage of the skull), thereby fusing the global macroscopic motion and local microscopic random perturbation; S2, a dual-flow feature extraction pipeline is constructed based on the vascular centerline as the processing object, and preoperative features are extracted through a 3D point cloud backbone network based on the dual-flow feature extraction pipeline, and preoperative projection and intraoperative features are extracted through a 2D image backbone network; wherein, the vascular centerline is sparse and contains rich topological information; To significantly reduce the computational complexity of multimodal dense matching and highlight the anatomical topological features of blood vessels, step S2 abandons the computationally expensive dense preoperative 3D volume rendering images and intraoperative 2D full-frame angiography images, and instead uses sparse and topologically rich blood vessel centerlines as the processing object to construct a dual-flow feature extraction pipeline.

[0042] In practical applications, as a preferred embodiment, the 2D image backbone network is any image feature extractor based on convolutional neural networks (CNNs, such as the ResNet series) or visual Transformers (such as ViT); and the 3D point cloud backbone network is any encoder based on convolutional neural networks (CNNs, such as the ResNet series) or visual Transformers (such as ViT) that is capable of capturing local and global geometric features in 3D space.

[0043] In a preferred embodiment, S2 includes: S21, based on 2D image feature flow, utilizes 2D image backbone network to extract pixel-level features from 2D projection images containing preoperative 3D centerline and real intraoperative 2D angiography images to form target feature set; S22, based on 3D point cloud feature flow, utilizes a 3D point cloud backbone network to directly extract point-level features from the preoperative 3D vascular centerline spatial coordinates to form a source feature set.

[0044] The design of this embodiment establishes a natural and explicit spatial correspondence between "3D points and 2D projections" at the feature level.

[0045] S3, through a cascaded attention mechanism, introduces the projection space as a bridge to decompose the cross-dimensional registration problem into the learning of anatomical consistency and deformation invariance, and finally outputs a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior; wherein the cascaded attention mechanism includes intramodal self-attention, intramodal cross attention mechanism and intermodal cross attention mechanism. In a preferred embodiment, S3 includes: S31, based on the intramodal self-attention mechanism and combined with relative position encoding, self-attention calculation is performed inside the 2D image feature stream and the 3D point cloud feature stream respectively, thereby enhancing the geometric topological structure features and contextual semantic learning within a single modality; S32, based on the intramodal cross-attention mechanism, promotes the cross-attention interaction between the 2D projection features of the preoperative 3D blood vessels and the real intraoperative 2D angiography features in the 2D image feature stream.

[0046] The core objective of step S32 is to capture the deformation context prior between the two, thereby establishing a "point projection-pixel" matching relationship based on deformation invariance; at the same time, in the 3D point cloud feature stream, the replicated and amplified 3D features are subjected to symmetrical cross-attention enhancement.

[0047] S33, based on the intermodal cross-attention mechanism, the features processed by the 2D image feature stream and carrying 2D deformation invariance information are exchanged with the spatial features in the 3D point cloud feature stream through bidirectional cross-attention information exchange, thereby finally outputting a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior.

[0048] Step S33 enforces strict fluoroscopic projection anatomy consistency constraints on the "preoperative 3D real point" and the "preoperative 3D projection point", and finally outputs a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior.

[0049] S4, calculate the final point-voxel matching relationship based on the high-dimensional joint features; the final point-voxel matching relationship is used to characterize the high-confidence 3D point-2D pixel correspondence relationship, and obtain the spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure; the high-confidence 3D point-2D pixel correspondence relationship includes the 3D point-3D point projection correspondence relationship and the 3D point projection-2D pixel correspondence relationship.

[0050] In a preferred embodiment, the calculation of the final point-voxel matching relationship based on the high-dimensional joint features includes: calculating the final point-voxel matching relationship based on the high-dimensional joint features using a double normalized exponential function combined with a rejection absorption node mechanism, including: S41, establish an allocation probability matrix including an additional rejection absorption node mechanism; wherein, the rejection absorption node mechanism is used to absorb unmatchable free points caused by the common problems of blood vessel occlusion, overlap or segmentation defects in cross-dimensional matching. S42, by jointly calculating and outputting the partial allocation probability matrix through the double normalized exponential function and the matching score predicted by the network; S43, obtain the point-voxel matching relationship based on the partial allocation probability matrix.

[0051] As a preferred embodiment, the step of calculating the final point-voxel matching relationship based on the high-dimensional joint features includes: calculating the final point-voxel matching relationship based on the high-dimensional joint features using the Sinkhorn algorithm, an allocation algorithm based on optimal transport theory, or a graph matching algorithm.

[0052] S5. Based on the high-confidence 3D point projection-2D pixel correspondence, a global non-rigid deformation field is obtained, and based on the global non-rigid deformation field, high-precision non-rigid plane alignment between the preoperative anatomical projection and the intraoperative real angiographic image is achieved.

[0053] In a preferred embodiment, S5 includes: S51, Receive the high-confidence 3D point projection-2D pixel correspondence as a control point; S52, Solve the global non-rigid deformation field based on the mathematical model of the control points and non-rigid deformation solver; In a preferred embodiment, S52 includes: using the control point-driven thin plate spline (TPS) spatial interpolation algorithm to calculate the global non-rigid deformation field; or, using the high-confidence matching pairs (control point displacements) output by the network as input parameters to directly drive the B-spline (B-Spline) model or free deformation (FFD) model for deformation smoothing; or, cascading a lightweight multi-layer perceptron (MLP) network at the back end of the registration network corresponding to the registration method to regress the dense deformation field end-to-end as the global non-rigid deformation field.

[0054] S53, the global non-rigid deformation field is applied to the entire preoperative 3D vascular projection to achieve high-precision non-rigid alignment between the preoperative anatomical structure and the real intraoperative angiographic image. Example 2

[0055] like Figure 2 As shown, this embodiment provides a cross-dimensional non-rigid vascular registration system based on feature matching. Its core architecture consists of five collaborative modules: a generalized deformation generation module, a two-stream feature extraction module, an attention-based cross-dimensional information interaction module, a matching probability estimation module, and a global non-rigid deformation estimation module. The specific technical implementation schemes of each module are as follows: A generalized deformation generation module is used to decouple complex deformations into global macroscopic displacements and local microscopic perturbations based on the anatomical characteristics of different blood vessel locations, and then fuse them. Dual-stream feature extraction module, such as Figure 2 The point cloud backbone and image backbone shown in the figure are used to construct a dual-flow feature extraction pipeline based on the vessel centerline as the processing object, and to extract preoperative features through a 3D point cloud backbone network and extract preoperative projection and intraoperative features through a 2D image backbone network based on the dual-flow feature extraction pipeline; wherein, the vessel centerline is sparse and contains rich topological information; Attention-based cross-dimensional information interaction modules, such as Figure 2As shown in the diagram, this module is the core decoupled architecture of the network, consisting of intramodal self-attention, intramodal cross-attention, and intermodal cross-attention mechanisms. It is used to decompose the cross-dimensional registration problem into learning anatomical consistency and deformation invariance through cascaded attention mechanisms, and finally output high-dimensional joint features that fuse 3D spatial structure and 2D deformation priors. The cascaded attention mechanisms include intramodal self-attention, intramodal cross-attention, and intermodal cross-attention mechanisms. Matching probability estimation module, such as Figure 3 As shown, the final point-voxel matching relationship is calculated based on the high-dimensional joint features; the final point-voxel matching relationship is used to characterize the high-confidence 3D point-2D pixel correspondence. Global non-rigid deformation solver module, such as Figure 2 As shown in the iterative registration, the post-processing execution unit of the registration system is used to obtain the global non-rigid deformation field based on the high-confidence 3D point-2D pixel correspondence, and to achieve high-precision non-rigid alignment between the preoperative anatomical structure and the intraoperative real angiographic image based on the global non-rigid deformation field.

[0056] The present invention also provides a memory that stores multiple instructions for implementing the method as described in Embodiment 1.

[0057] like Figure 4 As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301. The memory 302 stores a plurality of instructions, which can be loaded and executed by the processor to enable the processor to perform methods as described in Embodiments 2 and 3.

[0058] This embodiment employs a 3D-2D cross-dimensional registration overall architecture based on "projection space" as a bridge, and a decoupled architecture design for cross-dimensional matching based on "projection space". This architecture breaks away from traditional direct cross-dimensional multimodal matching, innovatively decoupling the complex registration problem into two related computational steps: first, establishing an anatomical consistency association between "preoperative 3D real points and preoperative 3D point projections" based on geometric perspective projection; second, establishing a deformation-invariant association between "preoperative 3D point projections and intraoperative 2D real pixels" based on feature matching. Secondly, it adopts a cross-modal dual-stream feature extraction mechanism. In the multimodal feature extraction stage, preoperative features are extracted through a 3D point cloud network backbone, and preoperative projection and intraoperative features are extracted through a 2D image backbone network. Third, a cascaded attentional interaction control flow within and between modalities is employed to protect the feature information flow transmission logic of "intramodal cross-attention" and "intermodal cross-attention" sequentially executed in the network architecture. First, intramodal interaction allows the projection points to be fused with real intraoperative images to capture deformation invariance. Then, intermodal interaction feeds this information back to 3D spatial points to implement strict anatomical consistency constraints. Fourth, a generalized deformation generation method and a medical image synthesis training scheme that does not require real-world paired labels are adopted. This training scheme includes three key steps: simulating macroscopic global displacement caused by vital signs (such as respiration and heartbeat) or changes in body position; using continuous spatial noise to simulate microscopic local traction and morphological disturbance of the blood vessel wall caused by the intervention of surgical instruments (such as guidewires and catheters); and introducing anatomical spatial anchoring weights (such as based on the physical characteristics of fixed vessel roots and free distal ends) to weightedly fuse macroscopic and microscopic deformations.

[0059] Furthermore, it should be noted that although the core embodiments of this invention are described in detail using the deformation field of the "coronary artery" affected by respiration and heartbeat as an example, the "3D point cloud-2D pixel feature cross-modal matching and deformation invariance decoupling" architecture and the generalized deformation simulation method of "local noise micro-perturbation" proposed in this invention can be directly applied as alternatives to 3D-2D intraoperative registration of cerebral blood vessels (neurointerventional thrombectomy / embolization surgery) and peripheral blood vessels. In these alternative application scenarios, the core matching network and non-rigid solution network architecture do not require any structural changes; only the simulation parameters of macroscopic displacement need to be adjusted according to the physiological characteristics of the target site (for example, cerebral blood vessel registration only needs to simulate the rigid micro-movement of the head and extremely slight brain tissue deformation, without adding a sinusoidal heartbeat waveform).

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-dimensional non-rigid registration method for blood vessels based on feature matching, characterized in that, include: S1, based on the anatomical characteristics of different blood vessel sites, decouples complex deformation into global macroscopic displacement and local microscopic disturbance and then fuses them; S2, a dual-flow feature extraction pipeline is constructed based on the vascular centerline as the processing object, and preoperative features are extracted through a 3D point cloud backbone network based on the dual-flow feature extraction pipeline, and preoperative projection and intraoperative features are extracted through a 2D image backbone network; wherein, the vascular centerline is sparse and contains rich topological information; S3, through a cascaded attention mechanism, introduces the projection space as a bridge to decompose the cross-dimensional registration problem into the learning of anatomical consistency and deformation invariance, and finally outputs a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior. The cascaded attention mechanisms mentioned above include intramodal self-attention, intramodal cross-attention, and intermodal cross-attention mechanisms. S4, calculate the point-voxel matching relationship based on the high-dimensional joint features. The point-voxel matching relationship is used to characterize the high-confidence 3D point-2D pixel correspondence relationship to obtain the spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure. The high-confidence 3D point-2D pixel correspondence relationship includes the 3D point-3D point projection correspondence relationship and the 3D point projection-2D pixel correspondence relationship. S5. Based on the high-confidence 3D point projection-2D pixel correspondence, a global non-rigid deformation field is obtained, and based on the global non-rigid deformation field, high-precision non-rigid plane alignment between the preoperative anatomical projection and the intraoperative real angiographic image is achieved.

2. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 1, characterized in that, S1 includes: S11 simulates global macroscopic motion, thereby simulating the macroscopic displacement caused by changes in the patient's vital signs or body position. S12, simulate local micro-random perturbations to simulate the physical traction, straightening and local morphological distortion of the blood vessel wall when surgical instruments are pushed into the blood vessel lumen; the simulation of local micro-random perturbations includes using continuous spatial noise to generate a spatially continuous random deformation field to apply perturbation to the blood vessel and model the local non-rigid deformation during the operation. S13, Based on the anatomical spatial anchoring weight mechanism, the global macroscopic motion and local microscopic random disturbance are fused to form a composite deformation field; the anatomical spatial anchoring weight mechanism includes: assigning different deformation weights according to the anatomical anchor points of different blood vessels, thereby fusing the global macroscopic motion and local microscopic random disturbance.

3. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 2, characterized in that, The 2D image backbone network is any image feature extractor based on convolutional neural networks or visual Transformers; and the 3D point cloud backbone network is any encoder based on convolutional neural networks or visual Transformers, or any encoder capable of capturing local and global geometric features in 3D space.

4. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 3, characterized in that, S2 includes: S21, based on 2D image feature flow, utilizes 2D image backbone network to extract pixel-level features from 2D projection images containing preoperative 3D centerline and real intraoperative 2D angiography images to form target feature set; S22, based on 3D point cloud feature flow, utilizes a 3D point cloud backbone network to directly extract point-level features from the preoperative 3D vascular centerline spatial coordinates to form a source feature set.

5. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 4, characterized in that, S3 includes: S31, based on the intramodal self-attention mechanism, combined with relative position encoding, self-attention calculation is performed inside the 2D image feature stream and the 3D point cloud feature stream respectively, thereby enhancing the geometric topological structure features and contextual semantic learning within a single modality; S32, based on the intramodal cross-attention mechanism, in the 2D image feature flow, the 2D projection features of the preoperative 3D blood vessels and the real intraoperative 2D angiography features are encouraged to interact with each other through cross-attention. S33, based on the intermodal cross-attention mechanism, the features processed by the 2D image feature stream and carrying 2D deformation invariance information are exchanged with the spatial features in the 3D point cloud feature stream through bidirectional cross-attention information exchange, thereby finally outputting a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior.

6. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 5, characterized in that, The calculation of the final point-voxel matching relationship based on the high-dimensional joint features includes: calculating the final point-voxel matching relationship based on the high-dimensional joint features using a double normalized exponential function combined with a rejection absorption node mechanism, characterizing a high-confidence 3D point-2D pixel correspondence, and obtaining a spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure, including: S41, establish an allocation probability matrix including an additional rejection absorption node mechanism; wherein, the rejection absorption node mechanism is used to absorb unmatchable free points caused by the common problems of blood vessel occlusion, overlap or segmentation defects in cross-dimensional matching. S42, by jointly calculating and outputting the partial allocation probability matrix through the double normalized exponential function and the matching score predicted by the network; S43, based on the partial allocation probability matrix, obtain the point-voxel matching relationship, characterize the high-confidence 3D point-2D pixel correspondence, and obtain the spatial mapping of the 2D intraoperative surgical progress in the 3D preoperative anatomical structure; the high-confidence 3D point-2D pixel correspondence includes the 3D point-3D point projection correspondence and the 3D point projection-2D pixel correspondence.

7. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 6, characterized in that, The calculation of the final point-voxel matching relationship based on the high-dimensional joint features includes: calculating the final point-voxel matching relationship based on the high-dimensional joint features using the Sinkhorn algorithm, an allocation algorithm based on optimal transport theory, or a graph matching algorithm.

8. The cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 7, characterized in that, S5 includes: S51, Receive the high-confidence 3D point projection-2D pixel correspondence as a control point; S52, solve the global non-rigid deformation field based on the mathematical model of the control points and the non-rigid deformation solver.

9. A cross-dimensional non-rigid registration method for blood vessels based on feature matching according to claim 8, characterized in that, S52 includes: using the control points to drive the thin-plate spline plane interpolation algorithm to solve the global non-rigid deformation field; or, using the control point displacements of the high-confidence matching pairs output by the network as input parameters to directly drive the B-spline model or free deformation model for deformation smoothing; or, cascading a lightweight multilayer perceptron network at the back end of the registration network corresponding to the registration method to regress the dense deformation field end-to-end as the global non-rigid deformation field. S53, the global non-rigid deformation field is applied to the entire preoperative 3D vascular projection to achieve high-precision non-rigid alignment between the preoperative anatomical projection and the real intraoperative angiographic image.

10. A cross-dimensional non-rigid vascular registration system based on feature matching, used to implement the method described in any one of claims 1-9, characterized in that, include: A generalized deformation generation module is used to decouple complex deformations into global macroscopic displacements and local microscopic perturbations based on the anatomical characteristics of different blood vessel locations, and then fuse them. The dual-flow feature extraction module is used to construct a dual-flow feature extraction pipeline based on the vessel centerline as the processing object, and to extract preoperative features through a 3D point cloud backbone network based on the dual-flow feature extraction pipeline, and to extract preoperative projection and intraoperative features through a 2D image backbone network; wherein the vessel centerline is sparse and contains rich topological information; The attention-based cross-dimensional information interaction module is used to introduce the projection space as a bridge through a cascaded attention mechanism, decompose the cross-dimensional registration problem into the learning of anatomical consistency and deformation invariance, and finally output a high-dimensional joint feature that integrates 3D spatial structure and 2D deformation prior; wherein the cascaded attention mechanism includes intramodal self-attention, intramodal cross attention mechanism and intermodal cross attention mechanism. The matching probability estimation and spatial mapping module is used to calculate point-voxel matching relationships based on the high-dimensional joint features; the point-voxel matching relationships are used to characterize high-confidence 3D point-2D pixel correspondences to obtain the spatial mapping of 2D intraoperative surgical progress in 3D preoperative anatomical structures; the high-confidence 3D point-2D pixel correspondences include 3D point-3D point projection correspondences and 3D point projection-2D pixel correspondences. The global non-rigid deformation solving module is used to obtain the global non-rigid deformation field based on the high-confidence 3D point projection-2D pixel correspondence, and to achieve high-precision and non-rigid alignment between the preoperative anatomical projection and the intraoperative real angiographic image based on the global non-rigid deformation field.

Images