Annotation and sample generation method, computing device, and program product

Abstract

The application provides a labeling and sample generation method based on coronary CCTA three-dimensional images, a computing device and a program product. The method comprises initial key point extraction, initial key point coordinate refinement, path interpolation, multi-view slice generation, slice information extraction and sample-label pair generation. Only a small amount of manual labeling is required to obtain high-quality sample-label pairs of various types for training of various models.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional image data processing technology, specifically to a method, computing device, and program product for annotation and sample generation based on coronary CCTA three-dimensional images. Background Technology

[0002] Coronary computed tomography angiography (CCTA) is currently an important non-invasive imaging technique for the clinical diagnosis of coronary artery disease. In the post-processing and analysis of CCTA three-dimensional images, accurately extracting the centerline of the coronary artery is the prerequisite and foundation for assessing the degree of vascular stenosis, quantitative plaque analysis, and curved surface reconstruction (CPR).

[0003] Existing deep learning network architectures are diverse, such as 3D segmentation based on 3D U-Net, 3D tracking based on point clouds, and 2D object detection based on networks like YOLO. Training these deep learning networks requires a large number of accurately labeled samples. Manual annotation is costly and inconsistent, and the types of labels and corresponding sample data formats required vary for different deep learning networks.

[0004] In related technologies, Chinese invention patent application CN111178420A proposes a method and system for annotating coronary artery segments on two-dimensional angiography images. This annotation method is only applicable to a single sample data format and a single label type. Chinese invention patent application CN121416014A proposes a rapid annotation method, computer device, and computer program product for coronary CTA vessels. This annotation method outputs the boundary of a single three-dimensional vessel for a specific task, with a single annotation type; and it requires constructing a three-dimensional coronary artery model based on coronary CTA sequence images, making it unsuitable for multiple sample data formats. Summary of the Invention

[0005] This invention provides a method, computing device, and program product for annotation and sample generation based on coronary CCTA three-dimensional images, so as to efficiently and reliably realize the automatic generation of various types of sample-label pairs involving coronary artery images.

[0006] This invention provides the following technical solution: a method for annotation and sample generation based on coronary CCTA three-dimensional images, comprising: Initial key point annotation, wherein the initial key point is obtained by initially annotating the center point of the coronary artery in the coronary artery CCTA three-dimensional image. The initial key point coordinates are refined, wherein the initial central axis of the coronary artery is constructed based on the initial key points, the surface is straightened and reconstructed along the initial central axis, the two-dimensional cross-section where the initial key points are located is extracted in the surface reconstruction space, the vascular cross-sectional contour of the coronary artery is segmented in the extracted two-dimensional cross-section, and the geometric center of each segmented independent vascular cross-sectional contour is mapped to the three-dimensional image of the coronary artery CCTA to obtain the refined center point corresponding to the initial key points. Path interpolation, wherein spline interpolation is performed using the refined center point as the control anchor point to obtain a path sampling point set for the coronary artery centerline, the path sampling point set including multiple path sampling points; Multi-view slice generation, wherein the coronary CCTA three-dimensional image is reconstructed along multiple spatial planes through the path sampling points to obtain multiple two-dimensional slices, and the intersection of the spline interpolation curve and the two-dimensional slice is obtained as the vessel center point in the two-dimensional slice; Slice information extraction, wherein the blood vessel cross-sectional contour and the corresponding minimum bounding rectangle are extracted from the two-dimensional slice; Sample-label pair generation, wherein multiple types of sample-label pairs are generated based on one or more of the following: the path sampling point set, the two-dimensional slice, the vascular cross-sectional contour of the two-dimensional slice, the vascular center point of the two-dimensional slice, the two-dimensional cross-section, the vascular cross-sectional contour in the two-dimensional cross-section, and the minimum bounding rectangle.

[0007] In some implementations, the initial key points include at least: the center point of the coronary artery ostium, the center point of the coronary artery bifurcation, the center point of the free end of the coronary artery, and the center point of the path.

[0008] In some implementations, the degree of the spline interpolation is an odd number greater than or equal to 3.

[0009] In some implementations, the selected spatial plane in the multi-view slice generation step includes at least one of the following planes: a horizontal plane, a coronal plane, a sagittal plane, and an oblique plane that is different from the horizontal plane, the coronal plane, and the sagittal plane.

[0010] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the coronary CCTA three-dimensional image and the corresponding label is the path sampling point set.

[0011] In some embodiments, the method further includes: constructing a three-dimensional coronary vascular tree mask based on the vascular cross-sectional contour of the two-dimensional slice; among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the three-dimensional coronary CCTA image and the corresponding label is the three-dimensional coronary vascular tree mask constructed based on the vascular cross-sectional contour of the two-dimensional slice.

[0012] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the two-dimensional slice or the two-dimensional cross-section, and the corresponding label is the vascular cross-sectional profile of the two-dimensional slice or the vascular cross-sectional profile of the two-dimensional cross-section.

[0013] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the two-dimensional slice, and the corresponding label is the minimum bounding rectangle of the two-dimensional slice and the coordinates of the center point of the blood vessel in the two-dimensional slice.

[0014] The above is merely an exemplary demonstration of how the annotation information, including the path sampling point set, the two-dimensional slice, the vascular cross-sectional contour of the two-dimensional slice, the vascular center point of the two-dimensional slice, the two-dimensional cross-section, the vascular cross-sectional contour in the two-dimensional cross-section, and the minimum bounding rectangle, is used. In other embodiments, one, two, three, four, or more of the above annotation information can be selected to form the labels in the sample-label pair.

[0015] For example, among the various types of sample-label pairs, there exists a type where the sample is the two-dimensional slice, and the corresponding label is the minimum bounding rectangle of the two-dimensional slice. This sample-label pair can be used for training a two-dimensional medical image vascular region selection network.

[0016] For example, among the various types of sample-label pairs, there exists a type where the sample is the two-dimensional slice, and the corresponding label is the coordinates of the center point of the blood vessel in the two-dimensional slice. This sample-label pair can be used for training a two-dimensional medical image keypoint detection network.

[0017] The present invention provides the following technical solution: a computing device, including a memory and a processor, wherein the memory stores a program, and the processor runs the program to execute the above-described annotation and sample generation method based on coronary CCTA three-dimensional images.

[0018] The present invention provides the following technical solution: a program product, which executes the above-mentioned annotation and sample generation method based on coronary CCTA three-dimensional images when running on a processor.

[0019] The annotation and sample generation method based on coronary CCTA 3D images of the present invention requires only a small number of initial key points and can run automatically to obtain various types of sample-label pairs for training machine learning models. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the annotation and sample generation method based on coronary CCTA three-dimensional images of the present invention.

[0021] Figure 2 This is an example of obtaining a three-dimensional coronary vascular tree mask through an embodiment of the present invention.

[0022] Figure 3 This is a structural block diagram of the computing device of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0024] refer to Figure 1 The present invention provides a method for annotation and sample generation based on coronary CCTA three-dimensional images, including the following steps.

[0025] S1. Initial key point annotation, wherein the initial key point is obtained by initially annotating the center point of the coronary artery in the coronary artery CCTA three-dimensional image.

[0026] The initial key points include at least: the center point of the coronary artery ostium, the center point of the coronary artery bifurcation, the center point of the free end of the coronary artery, and the center point of the path.

[0027] In some implementations, 3D CCTA data in DICOM, NIFTI, NRRD, or other formats is imported into 3D medical visualization software. The 3D image or 2D slice image of the coronary artery is displayed in the software, and a small number of initial key points are selected and labeled on the main trunk and branches of the coronary artery in a manually interactive manner.

[0028] The initial keypoint annotation information includes: the 3D coordinates of the initial keypoint in the 3D CCTA image space and the category label of the initial keypoint. The category label is, for example: root node (representing the center point of the coronary artery ostium), path center point, center point of the coronary artery bifurcation, and center point of the free end of the coronary artery.

[0029] The path center point is the center point at any location other than the center point of the coronary artery ostium, the center point of the coronary artery bifurcation, and the center point of the free end of the coronary artery. For example, the path center point can be the center point of the vessel between the center point of the coronary artery bifurcation and the center point of the free end of the coronary artery.

[0030] In some implementations, the annotation information of the initial key points also includes richer anatomical information, such as distinguishing the root nodes from the left and right coronary artery root nodes.

[0031] S2. Initial key point coordinate refinement, wherein the initial central axis of the coronary artery is constructed based on the initial key points, and a straightened curved surface is reconstructed along the initial central axis. The two-dimensional cross-section where the initial key points are located is extracted in the curved surface reconstruction space. The vascular cross-sectional contour of the coronary artery is segmented in the extracted two-dimensional cross-section. The geometric center of each segmented independent vascular cross-sectional contour is mapped to the three-dimensional image of the coronary artery CCTA to obtain the refined center point corresponding to the initial key points.

[0032] A tree structure is generated based on the initial key points marked in step S1, with adjacent nodes connected by straight lines to form a relatively rough initial central axis.

[0033] Curved Planar Reformation (CPR) is divided into projection CPR, straightening CPR, stretching CPR and spiral CPR. In this invention, straightening CPR is selected.

[0034] When performing straightening CPR, the initial central axis is broken at the bifurcation point, resulting in multiple unbifled vascular paths. Each unbifled vascular path does not contain a bifurcation point. For example, the path from the root node to the first bifurcation point in the tree structure closest to the root node is considered an independent unbifled vascular path.

[0035] A two-dimensional cross-section containing the initial keypoints is extracted in the surface reconstruction space. This cross-section is perpendicular to the local orientation of the blood vessel. Adaptive thresholding is performed within this cross-section based on the contrast agent CT value. In other words, the image regions belonging to the blood vessel are determined within the two-dimensional cross-section. Methods for segmenting blood vessels within the two-dimensional cross-section are not limited to this; for example, they can also include seed-based region growing algorithms, circular structure filtering methods, or deep learning segmentation algorithms. Deep learning segmentation algorithms include network structures such as U-Net, nnU-Net, or MedSAM.

[0036] The geometric center of each segmented blood vessel cross-section contour is obtained by inverse coordinate transformation according to the spatial transformation relationship of the straightened curved surface reconstruction, and the coordinates in the original three-dimensional CCTA image space are used as the coordinates of the refinement center point.

[0037] The coordinates of the refined center point are more accurate than those of the initial center point, meaning they are closer to absolute centering, thus reducing the error of manual annotation.

[0038] S3. Path interpolation, wherein spline interpolation is performed using the refined center point as the control anchor point to obtain the path sampling point set of the coronary artery centerline, and the path sampling point set includes multiple path sampling points.

[0039] In some implementations, the degree of the spline interpolation is an odd number greater than or equal to 3. That is, cubic spline interpolation, quintic spline interpolation, etc., are preferred. Cubic spline interpolation ensures that the second derivative of the interpolated curve is continuous at all connection points, achieving a natural and smooth result. Quintic spline interpolation provides even smoother higher-order derivative information.

[0040] Even-order spline interpolation (such as quartic) suffers from complex boundary conditions and lacks intuitive physical meaning due to the asymmetry of the mathematical structure of its interpolation nodes, while also increasing the bandwidth of the computation matrix. Therefore, odd-order spline interpolation is preferred.

[0041] The recommended physical spacing between path sampling points should not exceed 1.0 mm (or 1 to 2 times the voxel size). The physical spacing between path sampling points should generally not be less than 0.05 mm (or not less than 0.1 times the voxel size). These upper and lower limits are only recommendations; the primary consideration should be the desired amount of data.

[0042] S4. Multi-view slice generation, wherein, through the path sampling points, the coronary CCTA three-dimensional image is reconstructed along multiple spatial planes to obtain multiple two-dimensional slices, and the intersection of the spline interpolation curve and the two-dimensional slice is obtained as the vessel center point in the two-dimensional slice.

[0043] In some embodiments, during the slice generation step, the selected spatial plane includes at least one of the following planes: a horizontal plane, a coronal plane, a sagittal plane, and an oblique plane that is different from the horizontal plane, the coronal plane, and the sagittal plane.

[0044] Specifically, the coronary CCTA three-dimensional image is interpolated and reconstructed along different spatial planes through the path sampling points to obtain an initial two-dimensional slice, and then the size of the initial two-dimensional slice is scaled to a preset uniform size.

[0045] If only a segment of a blood vessel is captured in the initial two-dimensional slice, then the center point of the blood vessel is the sampling point along the path.

[0046] If multiple blood vessels are cut from the initial two-dimensional slice, the coordinates of the remaining blood vessels in the initial two-dimensional slice are determined by the intersection of the spatial plane where the initial two-dimensional slice is located and the curve obtained by spline interpolation with the two-dimensional slice.

[0047] After determining the coordinates of the vessel center point in the initial two-dimensional slice, the coordinates of the vessel center point in the two-dimensional slice are then determined based on the scaling parameters of the initial two-dimensional slice.

[0048] S5. Extraction of slice information, wherein the blood vessel cross-sectional contour and the corresponding minimum bounding rectangle are extracted from the two-dimensional slice.

[0049] Referring to the preceding text, extracting the vascular cross-sectional contour from the two-dimensional slice can be achieved using conventional segmentation algorithms. This invention does not limit the specific method for extracting the vascular cross-sectional contour.

[0050] One or more blood vessel cross-sectional profiles can exist in a single two-dimensional slice.

[0051] Step S5 can output a blood vessel mask image. The image size is the same as the corresponding two-dimensional slice size. The pixel value of the blood vessel cross-section contour area is 1, and the pixel value of other locations is 0.

[0052] The method of assigning values ​​to the vascular mask image can also be more diverse. For example, the pixel value of the vascular cross-sectional contour region belonging to the right coronary artery system is 1, the pixel value of the vascular cross-sectional contour region belonging to the left coronary artery system is 2, and the pixel value of the non-vascular region is 0.

[0053] S6. Sample-label pair generation, wherein multiple types of sample-label pairs are generated based on one or more of the following: the path sampling point set, the two-dimensional slice, the blood vessel cross-sectional contour of the two-dimensional slice, the blood vessel center point of the two-dimensional slice, the two-dimensional cross-section, the blood vessel cross-sectional contour in the two-dimensional cross-section, and the minimum bounding rectangle.

[0054] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the coronary CCTA three-dimensional image and the corresponding label is the path sampling point set.

[0055] The sample-label pair can be used to train a neural network that extracts the coronary artery centerline point cloud from a 3D coronary CCTA image.

[0056] Coronary CCTA 3D images can be used to obtain sample-label pairs, which can also be used to train 3D spatial tracking networks.

[0057] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the coronary CCTA three-dimensional image and the corresponding label is a three-dimensional coronary vascular tree mask constructed based on the vascular cross-sectional contour of the two-dimensional slice.

[0058] This sample-label pair can be used to train a 3D medical image segmentation network.

[0059] The construction process of the three-dimensional coronary vascular tree mask is as follows.

[0060] S61. Initialize a blank three-dimensional matrix.

[0061] Initialize a three-dimensional blank matrix M in memory that is completely empty and has the same spatial dimensions (matrix size) as the original coronary CCTA three-dimensional image.

[0062] S62. Perform inverse spatial transformation on the vascular cross-sectional contour of the two-dimensional slice.

[0063] The vascular cross-sectional contour of the two-dimensional slice is accurately mapped back to the three-dimensional voxel coordinates corresponding to the blank matrix M through inverse coordinate transformation according to its corresponding spatial coordinate transformation relationship. The voxel values ​​mapped back to the blank matrix M are changed from 0 to 1.

[0064] S63, Spatial Interpolation.

[0065] Since there may be gaps between two-dimensional slices, in order to ensure the smoothness and continuity of the three-dimensional blood vessel wall, the algorithm performs three-dimensional morphological interpolation or distance field filling between two adjacent stained sections in three-dimensional space, and automatically fills in the missing continuous voxels to 1.

[0066] S64, Topology Consistency Verification and Denoising.

[0067] The connected component analysis algorithm is used to check whether the reconstructed 3D vascular tree satisfies the topological characteristic of being continuous and uninterrupted. If there are isolated abrupt noise points (sparse 1-labels not connected to the main body), they are automatically filtered out by morphological opening operations.

[0068] Step S64 is an optional step. Figure 2 This is an example of a three-dimensional coronary artery tree.

[0069] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the two-dimensional slice or the two-dimensional cross-section, and the corresponding label is the vascular cross-sectional profile of the two-dimensional slice or the vascular cross-sectional profile of the two-dimensional cross-section.

[0070] This sample-label pair can be used to train two-dimensional medical image segmentation networks, such as two-dimensional U-Net, TransUNet, YOLO-seg, and other network structures.

[0071] In some implementations, among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the two-dimensional slice, and the corresponding label is the minimum bounding rectangle of the two-dimensional slice and the coordinates of the center point of the blood vessel in the two-dimensional slice.

[0072] This sample-label pair can be used to train a network for detecting key points in two-dimensional medical images, such as a two-dimensional U-Net or YOLO network structure.

[0073] This invention requires only a very small number of sparse initial key points to be manually selected in three-dimensional space, effectively avoiding the huge workload brought about by manual point-by-point annotation, and supporting the low-cost and rapid construction of massive medical-grade deep learning datasets.

[0074] By annotating the core process once, data adapted to various deep learning tasks can be automatically generated, greatly enhancing the reusability of the data.

[0075] It effectively eliminates the subjective drift caused by manual annotation and the cross-sectional distortion caused by the extreme curvature of blood vessels, ensuring that the extracted three-dimensional coronary vascular tree mask and coronary centerline have high physical accuracy.

[0076] Multi-view slice generation based on refined path points can obtain two-dimensional slices with good center alignment and corresponding vascular cross-sectional contours under multiple projection angles, thereby providing high-quality positional supervision information for subsequent deep learning model training and improving the model's adaptability to complex situations such as stenosis, bifurcation, and motion artifacts.

[0077] Multi-view slice generation based on refined path points can obtain two-dimensional slices with good center alignment and corresponding vascular cross-sectional contours under multiple projection angles, thereby providing high-quality positional supervision information for subsequent deep learning model training and improving the model's adaptability to complex situations such as stenosis, bifurcation, and motion artifacts.

[0078] Based on the same inventive concept, and referring to Figure 3 Embodiments of the present invention also provide a computing device, including at least one memory and at least one processor, wherein the memory stores a program and the processor runs the program to execute the above-described annotation and sample generation method based on coronary CCTA three-dimensional images.

[0079] Based on the same inventive concept, embodiments of the present invention also provide a program product that executes the above-described annotation and sample generation method based on coronary CCTA three-dimensional images when running on a processor.

[0080] It should be noted that the processors mentioned above may include, but are not limited to, one or more processors or microprocessors. Each processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor (MCU), central processing unit (CPU), graphics processing unit (GPU), data processing unit (DPU), or other electronic components to execute the methods in the above embodiments.

[0081] The above-mentioned storage devices can be various media capable of storing program code, such as USB flash drives, hard disk drives, solid-state drives, portable hard drives, read-only memory (ROM), random access memory (RAM), flash memory, EPROM, EEPROM, registers, magnetic disks, or optical disks. The storage device can be volatile, non-volatile, or a combination of both.

[0082] In one example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in a computing device.

[0083] This invention is not limited to the above-described embodiments. Any modifications, improvements, or substitutions that can be conceived by those skilled in the art without departing from the essential content of this invention fall within the scope of this invention.

Claims

1. A method for annotation and sample generation based on coronary CCTA three-dimensional images, characterized in that, include: Initial key point annotation, wherein the initial key point is obtained by initially annotating the center point of the coronary artery in the coronary artery CCTA three-dimensional image. The initial key point coordinates are refined, wherein the initial central axis of the coronary artery is constructed based on the initial key points, the surface is straightened and reconstructed along the initial central axis, the two-dimensional cross-section where the initial key points are located is extracted in the surface reconstruction space, the vascular cross-sectional contour of the coronary artery is segmented in the extracted two-dimensional cross-section, and the geometric center of each segmented independent vascular cross-sectional contour is mapped to the three-dimensional image of the coronary artery CCTA to obtain the refined center point corresponding to the initial key points. Path interpolation, wherein spline interpolation is performed using the refined center point as the control anchor point to obtain a path sampling point set for the coronary artery centerline, the path sampling point set including multiple path sampling points; Multi-view slice generation, wherein the coronary CCTA three-dimensional image is reconstructed along multiple spatial planes through the path sampling points to obtain multiple two-dimensional slices, and the intersection of the spline interpolation curve and the two-dimensional slice is obtained as the vessel center point in the two-dimensional slice; Slice information extraction, wherein the blood vessel cross-sectional contour and the corresponding minimum bounding rectangle are extracted from the two-dimensional slice; Sample-label pair generation, wherein multiple types of sample-label pairs are generated based on one or more of the following: the path sampling point set, the two-dimensional slice, the vascular cross-sectional contour of the two-dimensional slice, the vascular center point of the two-dimensional slice, the two-dimensional cross-section, the vascular cross-sectional contour in the two-dimensional cross-section, and the minimum bounding rectangle.

2. The annotation and sample generation method based on coronary CCTA three-dimensional images according to claim 1, characterized in that, The initial key points include at least: the center point of the coronary artery ostium, the center point of the coronary artery bifurcation, the center point of the free end of the coronary artery, and the center point of the path.

3. The annotation and sample generation method based on coronary CCTA three-dimensional images according to claim 1, characterized in that, The degree of the spline interpolation is an odd number greater than or equal to 3.

4. The annotation and sample generation method based on coronary CCTA three-dimensional images according to claim 1, characterized in that, In the step of generating multi-view slices, the selected spatial plane includes at least one of the following planes: a horizontal plane, a coronal plane, a sagittal plane, and an oblique plane that is different from the horizontal plane, the coronal plane, and the sagittal plane.

5. The annotation and sample generation method based on coronary CCTA three-dimensional images according to claim 1, characterized in that, Among the various types of sample-label pairs, there is one type where the sample is the coronary CCTA 3D image and the corresponding label is the path sampling point set.

6. The annotation and sample generation method based on coronary CCTA three-dimensional images according to claim 1, characterized in that, The method further includes: constructing a three-dimensional coronary artery tree mask based on the vascular cross-sectional contour of the two-dimensional slice; among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the three-dimensional coronary CCTA image and the corresponding label is the three-dimensional coronary artery tree mask constructed based on the vascular cross-sectional contour of the two-dimensional slice.

7. The annotation and sample generation method based on coronary CCTA three-dimensional images according to claim 1, characterized in that, Among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the two-dimensional slice or the two-dimensional cross-section, and the corresponding label is the vascular cross-sectional profile of the two-dimensional slice or the vascular cross-sectional profile of the two-dimensional cross-section.

8. The method for annotation and sample generation based on coronary CCTA three-dimensional images according to claim 1, characterized in that, Among the various types of sample-label pairs, there is one type of sample-label pair where the sample is the two-dimensional slice, and the corresponding label is the minimum bounding rectangle of the two-dimensional slice and / or the coordinates of the center point of the blood vessel in the two-dimensional slice.

9. A computing device, characterized in that, The method includes a memory and a processor, the memory storing a program and the processor running the program to perform the annotation and sample generation method based on coronary CCTA three-dimensional images according to any one of claims 1 to 8.

10. A program product, characterized in that, When the program product is run on the processor, it executes the annotation and sample generation method based on coronary CCTA three-dimensional images according to any one of claims 1 to 8.

Images