A method for establishing a three-dimensional model of coronary arteries based on digital subtraction angiography

Abstract

The present invention relates to a method for establishing a single-view coronary artery three-dimensional model based on digital subtraction angiography images and EG3D three-dimensional modeling technology, which can solve the problem of the lack of single-view three-dimensional modeling technology in the field of medical images. The method comprises the following steps: first, reading digital subtraction angiography data, and preprocessing the image and its corresponding information, secondly, building and training an image classification network that can distinguish patients, and building a single-view coronary artery three-dimensional model generation network based on the EG3D network, redesigning the neural renderer part with transparency as the predicted value, and then training the generation network, and using the image classification network as a supervision model to optimize the quality of the generated image, and finally using the trained single-view coronary artery three-dimensional model generation network to establish a three-dimensional model of the coronary artery based on a single view. The present invention can establish a three-dimensional model of the coronary artery based on a single digital subtraction angiography image.

Description

Technical Field

[0001] The present invention relates to a method for establishing a single-view three-dimensional model, and in particular to a method for establishing a single-view coronary artery three-dimensional model based on digital subtraction angiography images and EG3D three-dimensional modeling technology. Background Art

[0002] Digital subtraction angiography is an X-ray-based vascular imaging technology that is commonly used in the detection of cardiovascular and cerebrovascular diseases. By using professional digital subtraction angiography instruments, angiography images from multiple angles can be obtained. However, the digital subtraction angiography method can only obtain two-dimensional images, and cannot obtain a three-dimensional model of the coronary artery. In the current coronary artery three-dimensional modeling technology, the quantitative blood flow fraction method uses digital subtraction angiography images from multiple positions and uses image segmentation and mathematical calculation methods to establish a three-dimensional model of a single blood vessel. This method has the defect of being unable to obtain an overall three-dimensional model of the coronary artery. In the current field of artificial intelligence three-dimensional modeling, there are a variety of single-view three-dimensional modeling methods, such as NERF, 3D-GAN and other methods, but these methods mostly focus on areas such as faces and real scenes, and cannot be well applied to X-ray-based medical image three-dimensional modeling. Summary of the invention

[0003] The purpose of the present invention is to fill the gap of single-view 3D modeling technology in the field of medical images and propose a method for establishing a 3D model of coronary arteries based on digital subtraction angiography. The technical solution is as follows:

[0004] A method for establishing a three-dimensional model of coronary arteries based on digital subtraction angiography:

[0005] (1) Reading digital subtraction angiography data in DICOM format to obtain digital subtraction angiography images and their corresponding information;

[0006] (2) Preprocessing the digital subtraction angiography images and their corresponding information, removing images that do not contain blood vessels or have unclear angiography, and making labels with information such as the patient number and camera parameters corresponding to the images;

[0007] (3) Build an image classification network, use the patient number corresponding to the image as the label, and train the neural network so that the neural network has the ability to distinguish different patients through images;

[0008] (4) A single-view coronary artery 3D model generation network was built based on the EG3D network, and the neural renderer of the EG3D network was redesigned based on the imaging principle of digital subtraction angiography. Transparency was used as the prediction variable, and the three-plane encoding of the digital subtraction angiography image was converted into voxel transparency by using an MLP layer;

[0009] (5) Using digital subtraction angiography images of multiple patients with different imaging device posture angles as training samples, and using the posture angles corresponding to the images as training labels, a single-view coronary artery 3D model generation network was trained, and an image classification network was used as a supervision model to optimize the quality of the generated images;

[0010] (6) The trained single-view coronary artery 3D model is used to generate a network, and a single digital subtraction angiography image that has not participated in the training and the target posture angle are used as the input of the network to obtain a predicted image of the target posture angle.

[0011] The beneficial effects of the present invention are:

[0012] (1) The present invention is a method for establishing a three-dimensional model of the coronary arteries from a single perspective, which can accurately establish an overall three-dimensional model of the coronary arteries using only a single digital subtraction angiography image.

[0013] (2) The present invention is a three-dimensional generative adversarial network suitable for digital subtraction angiography images, and its application scope can be extended to other types of X-ray-based medical images.

[0014] (3) This method has been verified through a large number of experiments, which effectively improves the reliability of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is the overall structure diagram of the present invention; DETAILED DESCRIPTION

[0016] To make the technical solution of the present invention clearer, the present invention is further described below in conjunction with the accompanying drawings. The present invention is specifically implemented according to the following steps:

[0017] (1) Read digital subtraction angiography data in DICOM format, obtain digital subtraction angiography images and their corresponding information, including patient number, serial number, camera position, patient position, frame number, angiography equipment posture angle, etc.;

[0018] (2) Preprocess the digital subtraction angiography images and their corresponding information, and create labels. In the image preprocessing stage, first remove images that do not contain blood vessels or have unclear angiography, and then rename the images according to the read digital subtraction angiography data information. The image name consists of information such as patient number, serial number, camera position, patient position, frame number, and angiography device posture angle, so that different images have unique names. In addition, two types of labels need to be created according to needs, including image classification labels and camera parameter labels. The image classification label format is a CSV file, and the content is the image name and its corresponding patient number; the camera parameter label format is a JSON file, and the content is the image name and its corresponding camera intrinsic parameters and camera extrinsic parameters;

[0019] (3) Build an image classification network as a supervision model. The feature extractor E of the image classification network can be set to a variety of neural networks such as ResNet18 or VGG16, and set the classifier according to the number of patient numbers in the training set. The classifier can estimate the probability that the image belongs to each patient based on a single image, thereby classifying the image. The loss function is set to cross entropy:

[0020]

[0021] Using the patient numbers corresponding to the digital subtraction angiography images as labels, training the classification network enables the classification network to distinguish different patients through the digital subtraction angiography images, thereby enabling it to have the ability to focus on the key features of the digital subtraction angiography images;

[0022] (4) Build a single-view coronary artery 3D model generation network, whose basic structure is based on the EG3D network used for face 3D reconstruction. Since the neural renderer of the EG3D network is built based on the principle of natural light imaging, while the digital subtraction angiography image is based on X-ray imaging, there are differences in their imaging principles, that is, natural light focuses on natural colors, while X-rays focus on penetration, so it is necessary to make the neural renderer use transparency rather than RGB value as the predicted value of the voxel. Redesign the neural renderer and use an MLP layer to convert the three-plane encoding value of the digital subtraction angiography image into the transparency of the voxel, see Figure 1 The neural renderer part in . After the model training in step (3), all voxels are combined to form the three-dimensional structure of the coronary artery;

[0023] (5) Using digital subtraction angiography images of multiple patients at different camera positions and different posture angles as training samples, and using the camera parameters corresponding to the images as training labels, the images and labels are input into the single-view coronary artery 3D model generation network for training. During the training process, the input image x and the generated image x are calculated. The L2 loss L L2 and LPIPS loss L LPIPS . Use the image classification network as a supervised model to optimize the quality of the generated image, and compare the input image x and the generated image At the same time, we input the image classification network and use the network’s feature extractor E to extract the input image x and the generated image respectively. The eigenvector f and And calculate f and The cosine similarity of COS For the loss function L L2 , L LPIPS , L COSThe weights λ1, λ2, and λ3 are used to weight the overall loss function L, and the network is trained to have the ability to establish a three-dimensional model of the coronary artery;

[0024]

[0025] L=λ1L L2 +λ2L LPIPS +λ3L COS

[0026] (6) Using the trained single-view coronary artery 3D model generation network, the single digital subtraction angiography image X that was not involved in the training was transformed into in and target attitude angle A obj As the input of the network G, the predicted image X of the target posture angle can be obtained obj .

[0027] X obj =G(X in , A obj ).

Claims

1. A method for establishing a three-dimensional model of coronary arteries based on digital subtraction angiography, characterized in that: The steps to build a 3D coronary artery model include: (1) Reading digital subtraction angiography data in DICOM format to obtain digital subtraction angiography images and their corresponding information; (2) Preprocessing the digital subtraction angiography images and their corresponding information, removing images that do not contain blood vessels or have unclear angiography, and making labels with the patient number and camera parameter information corresponding to the images; (3) Build an image classification network as a supervised model. The feature extractor of the image classification network can be set to a ResNet18 or VGG16 neural network, and set the classifier according to the number of patient numbers in the training set, with the loss function being the cross entropy. Use the patient number corresponding to the image as a label to train the neural network so that the classification network has the ability to distinguish different patients through digital subtraction angiography images. (4) A single-view coronary artery 3D model generation network was built based on the EG3D network, and the neural renderer of the EG3D network was redesigned based on the imaging principle of digital subtraction angiography. Transparency was used as the prediction variable, and the three-plane encoding of the digital subtraction angiography image was converted into voxel transparency by using an MLP layer; (5) Using digital subtraction angiography images of multiple patients with different imaging device posture angles as training samples, and using the posture angles corresponding to the images as training labels, a single-view coronary artery 3D model generation network was trained, and an image classification network was used as a supervision model to optimize the quality of the generated images; (6) The trained single-view coronary artery 3D model is used to generate a network, and a single digital subtraction angiography image that has not participated in the training and the target posture angle are used as the input of the network to obtain a predicted image of the target posture angle.

2. The method for establishing a three-dimensional model of coronary arteries based on digital subtraction angiography according to claim 1, characterized in that: The neural renderer method of redesigning the EG3D network in step (4) is: A single-view coronary artery 3D model generation network was built, whose basic structure was based on the EG3D network used for face 3D reconstruction. The neural renderer of the EG3D network was built based on the principle of natural light imaging, while the digital subtraction angiography image was based on X-ray imaging. The imaging principles of the two networks were different, that is, natural light focused on natural colors, while X-rays focused on penetration. Therefore, the neural renderer needed to use transparency rather than RGB value as the predicted value of the voxel. The neural renderer was redesigned, and an MLP layer was used to convert the three-plane encoding values ​​of the digital subtraction angiography image into the transparency of the voxel.

3. The method for establishing a three-dimensional model of coronary arteries based on digital subtraction angiography according to claim 1, characterized in that: The method for training a single-view coronary artery three-dimensional model generation network in step (5) is as follows: Using digital subtraction angiography images of multiple patients at different camera positions and posture angles as training samples, and the camera parameters corresponding to the images as training labels, the images and labels are input into a single-view coronary artery 3D model generation network for training; L=λ1L L2 +λ2L LPIPS +λ3L cos During the training process, the input image x and the generated image are calculated The L2 loss L L2 and LPIPS loss L LPIPS ; Use the image classification network as a supervision model to optimize the quality of the generated image, and compare the input image x and the generated image At the same time, we input the image classification network and use the network’s feature extractor E to extract the input image x and the generated image respectively. The eigenvector f and And calculate f and The cosine similarity of cos ; For the loss function L L2 , L LPIPS , L cos The weights λ1, λ2, and λ3 are used to weight the overall loss function L, and the network is trained to have the ability to establish a three-dimensional model of the coronary arteries.

Images