Coronary angiography blood vessel segmentation method and system based on conditional segmentation network
By introducing the shooting angle as a condition in coronary angiography vessel segmentation and combining the embedding layer, segmentation network and discriminant network, the problem of insufficient segmentation accuracy in traditional methods is solved, and higher segmentation accuracy and diagnostic accuracy are achieved.
Patent Information
- Application Number
- CN202411379502.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2025-01-24
AI Technical Summary
Traditional coronary angiography vascular segmentation methods rely on single-frame images and neural networks, and it is difficult to consider the impact of shooting angle on segmentation accuracy, resulting in insufficient segmentation accuracy.
The method based on conditional segmentation network is adopted, and by introducing the shooting angle as additional conditions, the prior knowledge of different blood vessel segments in the angiography image is provided, combined with the embedded layer, the segmentation network, and the judgment network to improve the segmentation accuracy.
It significantly improves the accuracy of coronary angiography blood vessel segmentation, provides a more accurate and objective basis for coronary heart disease diagnosis, and reduces the difficulty of segmentation.
Smart Images
Figure CN119360020A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of blood vessel segmentation image processing, and in particular to a coronary angiography blood vessel segmentation method and system based on a conditional segmentation network. Background Art
[0002] Coronary artery sclerosis is one of the diseases with the highest mortality rate. Coronary angiography (CAG) is a diagnostic method that uses X-ray technology to achieve real-time imaging of the coronary arteries. It is known as the "gold standard" for the diagnosis of coronary heart disease due to its high diagnostic efficiency and wide range of applications.
[0003] However, in the traditional diagnosis of coronary heart disease, physicians' evaluation of CAG images mainly relies on visual estimation or manual measurement. This evaluation method is easily interfered by subjective factors and it is difficult to ensure the accuracy and objectivity of the evaluation results. With the advancement of medical technology, diagnostic methods based on CAG images, such as quantitative flow fraction (QFR) and angiography fusion technology, have gradually gained clinical recognition. These emerging technologies are based on the accurate segmentation of coronary angiography vessels and the automatic identification of different vascular segments. The segmentation of coronary angiography vessels is a challenging task. Traditional segmentation methods are usually based on single-frame images only, relying on the color characteristics and structural characteristics of the blood vessels to design specific algorithms, or using the powerful computing power of neural networks to achieve blood vessel segmentation. However, due to the complexity of the coronary arteries themselves, it is difficult to achieve ideal segmentation results, and it is difficult to distinguish different vascular segments.
[0004] Considering the structural characteristics of coronary arteries, they will present specific morphological features at different observation angles. In clinical practice, doctors usually choose a specific angle range to observe the target blood vessel. Therefore, different observation angles directly affect the accuracy of blood vessel segmentation. However, when using neural networks for blood vessel segmentation in the prior art, blood vessel segmentation is only performed on the image itself, without considering the impact of angle problems, so there is a problem of accurate segmentation. Summary of the invention
[0005] Therefore, the purpose of the present invention is to provide a method and system for segmenting blood vessels in coronary angiography based on a conditional segmentation network, which introduces the shooting angle as an additional condition in the neural network structure, and provides prior knowledge of the approximate positions of different blood vessel segments in the angiography image. The difficulty of segmenting blood vessels in coronary angiography is reduced, the accuracy of segmentation is significantly improved, and a more accurate and objective basis is provided for the diagnosis of coronary heart disease.
[0006] In order to achieve the above object, the present invention provides a coronary angiography vessel segmentation method based on a conditional segmentation network, comprising:
[0007] Acquire a coronary angiography image of the blood vessel to be segmented and a corresponding angle vector;
[0008] The coronary angiography image and the corresponding angle vector are input into a conditional segmentation network, the coronary angiography image is segmented based on the angle vector using the conditional segmentation network, and the segmentation result is output.
[0009] Further, preferably, the angle vector is a vector composed of the primary angle and secondary angle of the locator;
[0010] The main angle of the locator is the angle of the main axis of rotation. In the cross section, the main axis of rotation is defined as 0 degrees perpendicular to the patient's chest, and the range is [-90°, +90°];
[0011] The secondary angle of the locator is the angle of the secondary axis of rotation, which is perpendicular to the primary axis of rotation. In the sagittal plane, the direction perpendicular to the patient's chest is zero degrees, +90 degrees corresponds to the skull direction, and -90 degrees corresponds to the foot direction.
[0012] Further, preferably, the conditional segmentation network includes an embedding layer E, a segmentation network S and a discriminant network D;
[0013] The embedding layer E is used to map the input angle vector into a high-dimensional matrix, and stack the high-dimensional matrix with the coronary angiography image as the input of the segmentation network;
[0014] The segmentation network S uses the high-dimensional matrix as prior knowledge to segment the coronary angiography image to obtain a segmented image;
[0015] The discriminant network D is used to discriminate between segmented images and labeled images during training.
[0016] Further, preferably, when the conditional segmentation network is trained, the contrast image i, the angle vector a and the manual annotation result l are input for training;
[0017] When the input image i and the segmentation result image l′ of the segmentation network S are compared, the discriminant network D judges it to be false; when the input image i and the manually annotated segmentation result image l are compared, the discriminant network D judges it to be true.
[0018] Further, preferably, the network parameters of the conditional segmentation network are adjusted according to the discrimination results to calculate the loss function; including: first fixing the parameters of the discriminant network D to update the parameters of the embedding layer E and the segmentation network S, and then fixing the updated parameters of the embedding layer E and the segmentation network S to update the parameters of the discriminant network D, and repeating the above steps until the loss function reaches a preset threshold or meets a preset number of iterations.
[0019] Furthermore, preferably, the overall loss function used in training is calculated using the following formula:
[0020]
[0021] Among them, λ is the weight coefficient, n represents the total number of annotated vascular segments, l t represents the area in image l that belongs to vessel segment t, |l t | indicates area l t The number of elements in .
[0022] The present invention also provides a coronary angiography vessel segmentation system based on a conditional segmentation network, comprising:
[0023] A data acquisition module, used for acquiring a coronary angiography image of a target blood vessel and a corresponding angle vector;
[0024] The conditional segmentation network module inputs the coronary angiography image and the corresponding angle into the conditional segmentation network, uses the conditional segmentation network to segment the coronary angiography image, and outputs the final segmentation result.
[0025] Further, preferably, when the conditional segmentation network is trained, it includes:
[0026] Inputting a manually annotated coronary angiography image and the segmentation result for discrimination; the manually annotated coronary angiography image is annotated with different colors according to different positions of the blood vessel segments;
[0027] Adjust the network parameters of the conditional segmentation network according to the discrimination results and calculate the loss function;
[0028] When the loss function reaches a preset threshold or meets the preset number of iterations, the training stops.
[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the coronary angiography vessel segmentation method based on the conditional segmentation network are implemented as described above.
[0030] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the coronary angiography vessel segmentation method based on the conditional segmentation network are implemented as described above.
[0031] The coronary angiography vessel segmentation method and system based on conditional segmentation network disclosed in the present application have at least the following advantages compared with the prior art:
[0032] 1. The conditional segmentation network provided by this application can use the angle as an additional condition to provide prior knowledge of the approximate position of different vascular segments in the angiography image. This reduces the difficulty of coronary angiography vascular segmentation, significantly improves the accuracy of segmentation, and provides a more accurate and objective basis for the diagnosis of coronary heart disease.
[0033] 2. The conditional segmentation network provided in this application combines a segmentation network and a discriminant network. The segmentation network focuses on learning how to distinguish the target organ or structure from the background, while the discriminant network can evaluate whether the entire segmentation result is reasonable from a global perspective, thereby improving the precision of the segmentation. The discriminant network can help capture subtle features that may be missed by the segmentation network, ensuring that small structures such as blood vessels are fully preserved.
[0034] 3. Through adversarial training, the segmentation network constantly tries to "cheat" the discriminant network, while the discriminant network tries to distinguish true and false segmentation results. This mutual game process can enhance the overall robustness of the model. Combining the training methods of the two networks helps to improve the model's adaptability to new data, so that the model can maintain good performance when facing input images of different types or qualities. It reduces the difficulty of coronary angiography vascular segmentation and significantly improves the accuracy of segmentation, providing a more accurate and objective basis for the diagnosis of coronary heart disease. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] FIG1( a ) is a schematic flow chart of a coronary angiography vessel segmentation method based on a conditional segmentation network provided by the present invention.
[0036] FIG1( b ) is a schematic diagram of the training process of the conditional segmentation network of the present invention.
[0037] Figure 2 A schematic diagram of the main angle of the positioner is provided for the present invention.
[0038] Figure 3 A secondary angle schematic diagram of a positioner is provided for the present invention.
[0039] Figure 4 This is a network structure diagram of the conditional segmentation network of the present invention.
[0040] Figure 5 This is a schematic diagram of the original image provided by the present invention.
[0041] Figure 6 This is a schematic diagram of manually annotated images in the present invention. DETAILED DESCRIPTION
[0042] The present invention is further described in detail below through the accompanying drawings and specific embodiments.
[0043] As shown in FIG1 , an embodiment of one aspect of the present invention provides a coronary angiography vessel segmentation method based on a conditional segmentation network, comprising the following steps:
[0044] S1, obtaining a coronary angiography image of the blood vessel to be segmented and the corresponding angle vector; further as follows Figure 2 and Figure 3 As shown, the angle vector is a vector composed of the main angle and the secondary angle of the locator;
[0045] The main angle of the locator is the angle of the main axis of rotation. In the cross section, the main axis of rotation is defined as 0 degrees perpendicular to the patient's chest, and the range is [-90°, +90°];
[0046] The secondary angle of the locator is the angle of the secondary axis of rotation, which is perpendicular to the primary axis of rotation. In the sagittal plane, the direction perpendicular to the patient's chest is 0 degrees, and the range is [-90°, +90°].
[0047] Specifically, Figure 2 and Figure 3 As shown, with the center of the chest in the patient's coronal plane as the origin, the main rotation axis and the secondary rotation axis intersect at the origin, the main rotation axis is defined at the intersection of the patient's coronal plane and the sagittal plane, the main angle α of the locator is defined in the transverse plane, the direction perpendicular to the patient's chest is zero degrees, the patient's left side (LAO) is +90 degrees, the patient's right side (RAO) is -90 degrees, and the effective range of the main angle α of the locator is -90 to +90 degrees; the secondary rotation axis is located at the intersection of the patient's coronal plane and the transverse plane, perpendicular to the main rotation axis, the secondary angle β of the locator is defined in the sagittal plane, the direction perpendicular to the patient's chest is zero degrees, +90 degrees corresponds to the skull direction (CRA), -90 degrees corresponds to the foot direction (CAU), and the effective range of the secondary angle β of the locator is -90 to +90 degrees.
[0048] The present application defines the angle vector a as a vector consisting of the locator primary angle α and the locator secondary angle β, that is, a = [α, β].
[0049] S2. Input the coronary angiography image and the corresponding angle vector into a conditional segmentation network, use the conditional segmentation network to segment the coronary angiography image based on the angle vector, and output the segmentation result.
[0050] Further preferably, Figure 4 As shown, the conditional segmentation network includes an embedding layer E, a segmentation network S and a discriminant network D;
[0051] The embedding layer E is used to map the input angle vector into a high-dimensional matrix, and stack the high-dimensional matrix with the coronary angiography image as the input of the segmentation network S; the coronary angiography image is recorded as the input image i. The embedding layer E maps the angle vector into a matrix equal to the size of the angiography image. The embedding layer E maps the input angle information to a high-dimensional feature space through feature abstraction and encoding. This process can describe the intrinsic characteristics of the conditional information in detail, providing key prior knowledge for subsequent image segmentation tasks, and guiding the segmentation network S to accurately capture the position and morphological characteristics of different vascular segments in the input image at this specific angle in subsequent segmentation tasks, thereby improving the accuracy of segmentation.
[0052] The segmentation network S segments the coronary angiography image using the angle vector formed based on the shooting angle as prior knowledge to obtain a segmented image;
[0053] The segmentation network S can be any segmentation network, such as Unet series, Yolo series, DeepLab series, etc. The segmentation network S outputs the segmentation result image l', and sends the input image i and the segmentation result image l' together to the discriminant network D for judgment. The discriminant network D is a convolutional neural network, which consists of a convolutional layer, a pooling layer and a fully connected layer, wherein multiple convolutional layers are used to extract image features, and the pooling layer is located between the convolutional layers to reduce the spatial dimension of the feature map. After feature extraction through multiple convolutional layers and pooling layers, the discrimination result is finally output through a fully connected layer.
[0054] The discriminant network D evaluates the reliability of the segmentation result of the segmentation network S by judging whether the input is the segmentation result image l′ from the segmentation network S or the segmentation result image l annotated by the doctor. As the number of training times increases, the segmentation network S generates a large number of false samples, and the discriminant network D continuously identifies them. The two compete with each other, and the segmentation ability of the segmentation network S and the identification ability of the discriminant network D become stronger and stronger. At this time, the segmentation result image l′ of the segmentation network S is getting closer and closer to the segmentation result image l annotated by the doctor, that is, the segmentation effect is getting better and better. Image l is obtained by professional physicians annotating image i based on their experience, such as Figure 6 As shown, professional physicians mark different vascular segments as different objects. In this example, colors are used to distinguish different objects. The guide catheter is marked in red, the left main trunk is marked in green, the anterior descending branch is marked in purple, the circumflex branch is marked in yellow, and the secondary branches including the diagonal branch and the sharp-margin branch are marked in cyan.
[0055] Specifically, when the conditional segmentation network is trained, the embedding layer E and the segmentation network S are trained simultaneously, and then the discriminant network D is trained, and the training is repeated alternately. Further preferably, when the conditional segmentation network is trained, the contrast image i (such as Figure 5 As shown), the angle vector a and the manual annotation result l are used as the data set for training;
[0056] The overall loss function used in training is calculated using the following formula:
[0057]
[0058] Among them, λ is the weight coefficient, n represents the total number of annotated vascular segments, l t represents the area in image l that belongs to vessel segment t, |l t | indicates area l t The number of elements in .
[0059] Specifically, the conditional segmentation network is trained using the following steps:
[0060] S201, obtaining a training coronary angiography image i, a corresponding angle vector, and an image l annotated by a physician;
[0061] S202, fixing the parameters of the discriminant network D, and training the embedding layer E and the segmentation network S at the same time; including inputting the training coronary angiography image i and the corresponding angle vector a into the conditional segmentation network, segmenting the coronary angiography image using the segmentation network S to obtain a segmentation result image l′, and inputting the segmentation result image l′ into the discriminant network D. Calculating a loss function based on the segmentation result image l′ of the segmentation network S and the labeled image l, and the judgment result of the discriminant network D on the segmentation result image l′, adjusting the network parameters of the embedding layer E and the segmentation network S based on the loss function, and updating the embedding layer E and the segmentation network S;
[0062] Further preferably, the discriminant network D discriminates the segmentation result image l′ and the input image i. When the input image is image i and the segmentation result image l′, the discriminant network D judges it to be false; when the input image is image i and the manually annotated segmentation result image l, the discriminant network D judges it to be true.
[0063] Further preferably, the loss function in this step is calculated using the following formula:
[0064]
[0065] S203, fixing the updated parameters of the embedding layer E and the segmentation network S, and training the discriminant network D; including inputting the training coronary angiography image i and the angle vector a into the updated conditional segmentation network, using the updated segmentation network S to segment the coronary angiography image i to obtain a new segmentation result image l′, using the new segmentation result image l′ and the coronary angiography image i as well as the image l and the coronary angiography image annotated by the physician to calculate the loss function, adjusting the network parameters of the segmentation network D based on the loss function, and updating the discriminant network D.
[0066] Further preferably, the loss function in this step is calculated using the following formula:
[0067]
[0068] S204, looping steps S202 and S203 until the loss function converges stably or meets the preset number of iterations, the conditional segmentation network training is completed.
[0069] It should be noted that when adjusting the network parameters of the conditional segmentation network based on the loss function, it is preferred to alternately train the embedding layer E, the segmentation network S and the discriminant network D, specifically including: first fixing the parameters of the discriminant network D to update the parameters of the embedding layer E and the segmentation network S, and then fixing the parameters of the embedding layer E and the segmentation network S to update the parameters of the discriminant network D, and repeating this process until the loss function converges stably or reaches a preset number of training iterations.
[0070] The present invention also provides a coronary angiography vessel segmentation system based on a conditional segmentation network, which is used to implement the steps of the coronary angiography vessel segmentation method based on a conditional segmentation network, including:
[0071] A data acquisition module, used for acquiring a coronary angiography image of a target blood vessel and a corresponding angle vector;
[0072] The conditional segmentation network module inputs the coronary angiography image and the corresponding angle into the conditional segmentation network, uses the conditional segmentation network to segment the coronary angiography image, and outputs the segmentation result.
[0073] It should be noted that the conditional segmentation network needs to be trained based on a large number of data sets before determining the final network parameters. During training, manually annotated coronary angiography images are input into the conditional segmentation network. The manually annotated coronary angiography images are labeled with different colors according to the location of the vascular segment.
[0074] During training, the discriminant network D is used to discriminate the segmentation result image l' and the input image i. When the input image is image i and the segmentation result image l', the discriminant network D judges it as false; when the input image is image i and the manually annotated segmentation result image l, the discriminant network D judges it as true. Whether the training is completed is determined by the loss function provided in the above method embodiment, which will not be repeated here.
[0075] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the coronary angiography vessel segmentation method based on the conditional segmentation network are implemented as described above.
[0076] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the coronary angiography vessel segmentation method based on the conditional segmentation network are implemented as described above.
[0077] The conditional segmentation network proposed in this application introduces the shooting angle as an additional condition on the basis of the classic segmentation network, providing prior knowledge of the approximate positions of different vascular segments in the angiography image, reducing the difficulty of coronary angiography vascular segmentation and improving the accuracy of segmentation; at the same time, a discriminant network is introduced during the segmentation network training process to make the segmentation results closer to the actual vascular distribution.
[0078] Obviously, the above embodiments are only examples for clear explanation, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.
Claims
1. A coronary angiography vessel segmentation method based on a conditional segmentation network, characterized in that: include: Acquire a coronary angiography image of the blood vessel to be segmented and a corresponding angle vector; The coronary angiography image and the corresponding angle vector are input into a conditional segmentation network, the coronary angiography image is segmented based on the angle vector using the conditional segmentation network, and the segmentation result is output.
2. The method for segmenting coronary angiography vessels based on a conditional segmentation network according to claim 1, characterized in that: The angle vector is a vector composed of the primary angle and secondary angle of the locator; The main angle of the locator is the angle of the main axis of rotation. In the cross section, the main axis of rotation is defined as 0 degrees perpendicular to the patient's chest, and the range is [-90°, +90°]; The secondary angle of the locator is the angle of the secondary axis of rotation, which is perpendicular to the primary axis of rotation. In the sagittal plane, the direction perpendicular to the patient's chest is zero degrees, +90 degrees corresponds to the skull direction, and -90 degrees corresponds to the foot direction.
3. The method for segmenting coronary angiography vessels based on a conditional segmentation network according to claim 1, characterized in that: The conditional segmentation network includes an embedding layer E, a segmentation network S and a discriminant network D; The embedding layer E is used to map the input angle vector into a high-dimensional matrix, and stack the high-dimensional matrix with the coronary angiography image as the input of the segmentation network; The segmentation network S uses the high-dimensional matrix as prior knowledge to segment the coronary angiography image to obtain a segmented image; The discriminant network D is used to discriminate between segmented images and labeled images during training.
4. The method for segmenting coronary angiography vessels based on a conditional segmentation network according to claim 1, characterized in that: When the conditional segmentation network is trained, the contrast image i, the angle vector a and the manual annotation result l are input for training; When the input image i and the segmentation result image l′ of the segmentation network S are compared, the discriminant network D judges it to be false; when the input image i and the manually annotated segmentation result image l are compared, the discriminant network D judges it to be true.
5. The method for segmenting coronary angiography vessels based on a conditional segmentation network according to claim 4, characterized in that: The method adjusts the network parameters of the conditional segmentation network according to the discrimination result and calculates the loss function; comprises: first fixing the parameters of the discriminant network D to update the parameters of the embedding layer E and the segmentation network S, then fixing the updated parameters of the embedding layer E and the segmentation network S to update the parameters of the discriminant network D, and repeating the above steps until the loss function reaches a preset threshold or meets a preset number of iterations.
6. The method for segmenting coronary angiography vessels based on a conditional segmentation network according to claim 5, characterized in that: The overall loss function used in training is calculated using the following formula: Among them, λ is the weight coefficient, n represents the total number of annotated vascular segments, l t represents the area in image l that belongs to vessel segment t, |l t | indicates area l t The number of elements in .
7. A coronary angiography vessel segmentation system based on conditional segmentation network, characterized in that: include: A data acquisition module, used for acquiring a coronary angiography image of a target blood vessel and a corresponding angle vector; The conditional segmentation network module inputs the coronary angiography image and the corresponding angle into the conditional segmentation network, uses the conditional segmentation network to segment the coronary angiography image, and outputs the final segmentation result.
8. The coronary angiography vessel segmentation system based on conditional segmentation network according to claim 7, characterized in that: When the conditional segmentation network is trained, it includes: Inputting a manually annotated coronary angiography image and the segmentation result for discrimination; the manually annotated coronary angiography image is annotated with different colors according to different positions of the blood vessel segments; Adjust the network parameters of the conditional segmentation network according to the discrimination results and calculate the loss function; When the loss function reaches a preset threshold or meets the preset number of iterations, the training stops.
9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the coronary angiography vessel segmentation method based on the conditional segmentation network as described in claims 1-6 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the coronary angiography vessel segmentation method based on a conditional segmentation network as described in claims 1-6 are implemented.