An unsupervised coronary angiography segmentation method

Abstract

This invention relates to an unsupervised coronary angiography segmentation method that solves the problems of missing structural information and annotation dependence in two-dimensional segmentation. The method includes the following steps: First, a three-dimensional implicit representation space is constructed based on EG3D, and its neural radiation field is used to model the three-dimensional topology of blood vessels. Second, a dual-channel adversarial generation architecture is designed, and adversarial training between parallel foreground and background generators achieves decoupling of vascular and non-vascular features. Then, combining the multi-angle projection characteristics of angiography, a three-dimensional attention fusion module is used to constrain the geometric consistency of multiple views. Next, a domain-adaptive encoder is developed to embed real images into the EG3D latent space, achieving geometric feature alignment between the real and synthetic domains. Finally, the implicit three-dimensional representation is converted into multi-view projection through differentiable volume rendering, and the foreground generator outputs a spatially continuous three-dimensional blood vessel segmentation mask. This invention can generate three-dimensional structural segmentation maps of blood vessels in coronary angiography under unsupervised conditions.

Description

Technical Field

[0001] This invention relates to an unsupervised method for generating vascular segmentation maps in the field of medical image processing, and in particular to an unsupervised method for generating vascular three-dimensional anatomical structure segmentation maps of coronary angiography vessels based on EG3D neural radiation field and dual-channel adversarial generative architecture that combines three-dimensional information. Background Technology

[0002] Coronary angiography vessel segmentation is a key technology for the diagnosis of cardiovascular diseases. Existing methods mainly rely on two-dimensional image segmentation algorithms, but suffer from problems such as missing three-dimensional structural information, poor continuity of vessel branches, and high dependence on labeled data. Traditional U-Net-based supervised learning methods require a large amount of manually labeled masks and are difficult to model the complex three-dimensional topology of vessels. While unsupervised methods based on generative adversarial networks can alleviate the labeling dependency, they are limited by two-dimensional feature extraction capabilities and are prone to losing the spatial coherence of vessels. Recently, the neural radiation field (NeRF)-based three-dimensional representation method (EG3D) has provided a new approach to implicitly modeling three-dimensional anatomical structures, but when directly applied to real coronary angiography data, it faces problems such as insufficient domain adaptability and coupling between vessel and background features. Existing three-dimensional segmentation methods mostly rely on pre-training with synthetic data, which leads to a decrease in segmentation accuracy in the real image domain due to geometric feature offsets, and cannot achieve end-to-end unsupervised three-dimensional segmentation mask generation. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of traditional two-dimensional segmentation methods in three-dimensional continuous modeling and dependence on labeled data. It proposes an unsupervised coronary angiography vessel segmentation method that combines three-dimensional implicit representation with dual-channel adversarial generation to achieve high-fidelity reconstruction of vascular anatomy. The technical solution is as follows: An unsupervised coronary angiography segmentation method: (1) Construction of three-dimensional feature space: The neural radiation field architecture of the EG3D pre-trained model is used to construct a three-dimensional implicit representation space. Through its multi-plane feature fusion capability, the input coronary angiography image is mapped into a three-plane feature tensor. The three-plane features encode the spatial topology of the blood vessel through orthogonal projection, and capture the three-dimensional anatomical features of the coronal plane, sagittal plane and transverse plane respectively, providing implicit three-dimensional geometric priors for subsequent segmentation.

[0004] (2) Dual-channel generation architecture design: A foreground generator and a background generator are designed to work in parallel. The foreground generator adds an Alpha branch network to the EG3D backbone network and extracts the foreground features of blood vessels through multi-scale feature fusion and channel dimensionality reduction. The background generator freezes the high-level semantic related layers in EG3D and retains the low-level texture generation channels to reconstruct non-vascular regions. The two synthesize images through dynamic α mixing coefficients and introduce a weak discriminator for adversarial training, forcing the network to separate the coupling features of blood vessels and background, and achieving feature decoupling under unsupervised conditions.

[0005] (3) Multi-view collaborative optimization: In view of the multi-angle projection characteristics of coronary angiography, a three-dimensional attention fusion module is designed to apply geometric consistency constraints through cross-view feature comparison learning. This module utilizes the spatial transformation invariance of three-dimensional implicit features to adaptively weight and fuse projection features at different rotation angles, ensuring the topological coherence of vascular branches under any viewpoint and avoiding structural breakage problems in two-dimensional projection.

[0006] (4) Real Image Latent Space Embedding: To address the geometric shift in the real image domain caused by pre-training of synthetic data, this invention develops a domain adaptation encoder based on adversarial training. This encoder employs a residual network structure and learns the nonlinear mapping from real coronary angiography images to the EG3D latent space through multi-layer convolution and adaptive instance normalization. During feature alignment, a maximum mean difference loss function is introduced, using kernel space to measure the distribution difference between the synthetic data domain and the real image domain, and the encoder parameters are optimized in conjunction with the adversarial domain alignment loss. Specifically, the encoder compresses the high-dimensional pixel features of the real image into the three-plane latent space of EG3D, while constraining the latent code distribution to maintain manifold consistency with the geometric features of the synthetic data, thereby eliminating the representational shift of the vascular topology between domains. Furthermore, by using a gradient inversion layer to dynamically balance the discriminative power of the domain classifier during adversarial training, the encoder is forced to generate domain-invariant three-dimensional implicit features, ensuring the generalization of the segmentation model in real-world scenarios.

[0007] (5) Generation of three-dimensional blood vessel segmentation mask: Based on differentiable volume rendering technology, implicit three-dimensional features are dynamically projected onto a multi-view two-dimensional plane to generate an initial segmentation mask; the blood vessel connectivity of the mask under different views is constrained by projection consistency loss, and gradient smoothing constraints are designed in combination with anatomical priors to optimize the continuity at blood vessel bifurcation; finally, a three-dimensional blood vessel mask with spatial continuity is output through the segmentation decoder to support the calculation of blood vessel stenosis rate and three-dimensional reconstruction.

[0008] The beneficial effects of this invention are: (1) By combining EG3D three-dimensional implicit representation with dual-channel adversarial generative architecture, this method achieves the segmentation of the anatomical topology of coronary vessels under unsupervised conditions, solves the problem of dependence on labeled data in traditional methods, and improves the segmentation accuracy of small vessel branches. (2) The generated 3D vascular mask can be directly used in clinical scenarios of vascular stenosis quantification and hemodynamic simulation, providing high-precision 3D anatomical model support for smart healthcare. Attached Figure Description

[0009] Figure 1 This is an illustration of the unsupervised segmentation network of the present invention; Detailed Implementation

[0010] To make the technical solution of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. The present invention is implemented in specific steps: The specific implementation method is as follows: (1) Construction of three-dimensional feature space: Collect dynamic sequences and multi-angle angiography parameters from multiple centers to construct a segmented unlabeled vessel dataset. Preprocess the images, including dynamic range normalization, respiratory motion artifact suppression and multi-view spatiotemporal alignment, and extract contrast agent enhancement temporal features. Use the EG3D pre-trained model to construct a three-dimensional implicit representation space, learn the implicit tri-plane feature distribution of vascular anatomy through its neural radiation field (NeRF) architecture, and map the input image into a tri-plane feature tensor to capture the three-dimensional topological information of the vessel; (2) Dual-channel generation architecture setup: Constructing parallel foreground and background generators: Based on the EG3D backbone network, its Tri-Plane feature encoding and Neural Radiation Field (NeRF) generation capabilities are retained, and an Alpha network branch is added. The Alpha network consists of multi-level upsampling modules that directly apply to the EG3D tri-plane features. For the EG3D-generated tri-plane features, since the original feature dimension is as high as 9,000+, direct processing would lead to computational redundancy and noise interference. Therefore, feature compression is first performed, reducing the channel dimension of the EG3D Tri-Plane features (XYZ tri-plane) by using 1×1 convolutions to compress the feature dimension from 9,000+ to 512, filtering out irrelevant noise. Secondly, multi-scale fusion is performed, integrating the 4×4 to 64×64 layers of EG3D features through cross-resolution skip connections to enhance the perception of local details of blood vessel branches. Finally, Alpha mask prediction is performed, adding a Sigmoid activation layer at the end to output pixel-wise Alpha values, representing the probability of the blood vessel foreground, as shown in the formula: in, It is the mean of the three-plane characteristics. These are learnable weights.

[0011] The background generator is based on a modified EG3D architecture and uses gradient backpropagation analysis to locate background-related feature channels. First, it calculates the gradient contribution of each StyleGAN2 layer to the background region, locking the low-level (4×4 resolution) tRGB channels as the main pathway for background generation. Second, it freezes the foreground-related intermediate layers in EG3D (16×16 and above resolution), allowing only the low-level tRGB channels to participate in background generation. Furthermore, to enhance the consistency of foreground transparency prediction and 3D structure alignment, an Alpha consistency loss function is introduced, applying pixel-level constraints to the Alpha images from multiple viewpoints to maintain similar transparency in overlapping areas, thereby optimizing the 3D geometric representation capability during volume rendering.

[0012] The dual-channel collaborative mechanism synthesizes the final image through an α-mixing formula: in Foreground generator, The background generator is designed with dual constraints: First, a weak discriminator D distinguishes between real and synthetic images, forcing the α network to accurately separate blood vessels from the background. Second, a projection consistency loss is used. Forced 3D masks maintain blood vessel connectivity and prevent rupture from multiple viewpoints.

[0013] This architecture achieves efficient decoupling of blood vessels from the background under unsupervised conditions through the collaboration of implicit 3D representation of EG3D and Alpha network.

[0014] (3) Multi-view collaborative optimization: Integrating the multi-angle projection characteristics of coronary angiography, a three-dimensional attention fusion module is designed to apply geometric consistency constraints between the acquired angiography angles. Cross-view feature alignment is optimized through contrastive learning, and the loss function is defined as: in, It is a three-dimensional implicit feature. The perspective is rotated. Simultaneously, temporal consistency constraints are introduced to optimize the spatiotemporal continuity of vascular branches using the dynamic characteristics of contrast agent perfusion.

[0015] (4) Latent Space Embedding of Real Images: A domain-adaptive encoder was developed to embed real coronary angiography images into the EG3D latent space using an adversarial domain alignment loss. The maximum mean difference (MMD) was used to measure the difference in latent space distribution between the synthetic data and the real images. The loss function was: The problem of accurate embedding of real images into the latent space is solved by forcing the alignment of geometric features between real images and synthetic data in the three-dimensional feature space.

[0016] (5) Generation of 3D blood vessel segmentation masks: Based on differentiable volume rendering technology, the implicit 3D representation is converted into a multi-view 2D projection, and a segmentation decoder is jointly trained to generate anatomically reasonable 3D blood vessel segmentation masks. Spatial continuity is optimized through projection consistency loss, and the loss function is defined as: in, For a 3D mask, Let be the two-dimensional pseudo-label from the k-th perspective. Further, an anatomical rationality constraint is introduced: Force smooth mask gradients at the junctions of vascular branches to avoid breakage.

[0017] (6) Application Practice: It can input multi-angle coronary angiography sequences, output three-dimensional vessel segmentation masks in real time, quantify the vascular stenosis rate and plaque spatial distribution, and provide navigation support for percutaneous coronary intervention (PCI).

[0018] Through the above implementation methods, the present invention achieves fine three-dimensional segmentation of coronary vessels without manual annotation, saving annotation costs while improving segmentation accuracy. It is suitable as an upstream task for segmenting coronary angiography vessels and provides support for various downstream intelligent algorithms.

Claims

1. An unsupervised coronary angiography segmentation method, characterized in that, The steps involved in incorporating 3D information during unsupervised segmentation of coronary angiography vessels include: (1) Construction of three-dimensional feature space: The three-dimensional implicit representation space is established by using the EG3D pre-trained model, and the input image is mapped into a three-plane feature tensor to capture the three-dimensional topological information of blood vessels; (2) Dual-channel generation architecture design: Construct a parallel foreground generator and a background generator. The foreground generator is based on the EG3D backbone network and adds an Alpha network branch to predict the probability of blood vessel foreground. (3) Multi-view collaborative optimization: Design a three-dimensional attention fusion module to apply geometric consistency constraints between multi-angle projections in coronary angiography; (4) Real Image Latent Space Embedding: Develop a domain-adaptive encoder to embed real coronary angiography images into the EG3D latent space and establish a geometric feature mapping between real images and synthetic data. (5) Generation of 3D blood vessel segmentation mask: Based on volume rendering technology, the implicit 3D representation is converted into a multi-view 2D projection, and the segmentation decoder is trained to generate a 3D blood vessel segmentation mask; (6) Apply the trained segmentation model to real-world scenarios to provide upstream support for downstream intelligent algorithms in narrow detection, calcification assessment and 3D reconstruction.

2. The unsupervised coronary angiography segmentation method as described in claim 1, characterized in that: The Alpha network branch of the foreground generator in step (2) includes the following operations: The three-plane features extracted from EG3D are compressed to 512 channels, and multi-layer features are fused through cross-resolution skip connections to enhance the perception of vascular details; a pixel-wise alpha value is output to represent the probability of vascular foreground, defined as: in, It is the mean of the three-plane characteristics. These are learnable weights.

3. The unsupervised coronary angiography segmentation method as described in claim 1, characterized in that: The loss function for the geometric consistency constraint in step (3) is defined as: in, It is a three-dimensional implicit feature. For viewpoint rotation transformation; the network outputs pixel-by-pixel alpha values ​​through the Sigmoid activation function, representing the probability of blood vessel foreground.

4. The unsupervised coronary angiography segmentation method as described in claim 1, characterized in that: The projection consistency loss in step (5) is defined as: in, For a 3D mask, A two-dimensional pseudo-label is used for the k-th viewpoint; the three-dimensional blood vessel model is converted using volume rendering technology.

Images