Growth inference method and system for coronary artery

Abstract

The invention provides a coronary artery growth inference method and system, and the method comprises the following steps: obtaining heart chamber point cloud data obtained through a CTA technology, and reconstruction point cloud data of a coronary artery; removing an aorta part in the reconstructed point cloud data to obtain coronary artery point cloud data; building a coronary artery inference network based on the heart chamber point cloud data and the coronary artery point cloud data; and obtaining a test sample, and inputting the test point cloud data of the test sample into the coronary artery inference network to output the growth space position of the coronary artery in the test sample. After the technical scheme is adopted, accurate segmentation and observation of the coronary artery are realized based on the point cloud technology.

Description

Technical Field

[0001] The present invention relates to the field of intelligent analysis of medical images, and in particular to a method and system for inferring the growth of coronary arteries. Background Art

[0002] As the main blood supply vessel of the heart muscle, the health of the coronary artery is crucial to the heart function. However, the precise segmentation and observation of the coronary artery usually depends on the coronary angiography (CTA) technology. CTA is the acronym of CTangeography, which refers to the CT angiography of arterial blood vessels. It combines CT enhancement technology with thin-layer, large-range, and rapid scanning technology. Through reasonable post-processing, it can clearly display the details of blood vessels in various parts of the body. When ordinary CT scans are performed, the blood vessels are not specially displayed. When the patient has a special vascular disease and needs to check the blood vessels very carefully, the contrast agent is injected into the artery and scanned at the right time. The CTA examination can clearly observe the distribution of the contrast agent in the blood vessel, thereby observing whether the blood vessel is narrowed, blocked, or dilated.

[0003] CTA is mainly used for vascular imaging, and can be reconstructed in three dimensions under software synthesis, giving an intuitive impression to clinical treatment. It is non-invasive and easy to operate, and is of great value for vascular variation, vascular disease, and the relationship between lesions and blood vessels. However, ordinary CT plain scans are difficult to directly display the structure of coronary arteries due to the fat wrapped around the coronary arteries. Despite this, CT plain scans can still roughly segment the various chambers of the heart, which provides a new idea for indirect observation of coronary arteries. Using the structural information of the heart chambers to infer the position of the coronary arteries may help to achieve preliminary observation of the coronary arteries in plain scan CT, thereby improving the diagnosis of coronary heart disease. In addition, this method also helps to enhance the analysis and understanding of the positional relationship between the heart chambers and the coronary arteries at the structural level, providing a new perspective for in-depth research on heart disease.

[0004] However, CTA technology can only visualize the patient's blood condition in the current state, and cannot judge the possible future growth of the coronary arteries. Therefore, a new method of inferring the growth of coronary arteries is needed to infer and analyze the growth direction and spatial position of the coronary arteries in three-dimensional space. Summary of the invention

[0005] In order to overcome the above technical defects, the purpose of the present invention is to provide a method and system for inferring the growth of coronary arteries of the heart, based on point cloud technology to achieve accurate segmentation and observation of the coronary arteries.

[0006] The present invention discloses a method for inferring the growth of coronary arteries of a heart, comprising the following steps:

[0007] Acquire the heart chamber point cloud data obtained by CTA technology, as well as the reconstructed point cloud data of the coronary arteries;

[0008] The aorta part in the reconstructed point cloud data is removed to obtain the coronary artery point cloud data;

[0009] Build a coronary artery inference network based on heart chamber point cloud data and coronary artery point cloud data;

[0010] A test sample is obtained, and test point cloud data of the test sample is input into a coronary artery inference network to output a growth spatial position of the coronary artery in the test sample.

[0011] Preferably, the step of acquiring the cardiac chamber point cloud data obtained by CTA technology and the reconstructed point cloud data of the coronary arteries comprises:

[0012] Collect CTA cardiac volume data files, CTA clinical data medical record information, CT cardiac volume data files and CT clinical data medical record information from multiple hospital units as basic data, and establish a basic database after preprocessing the basic data;

[0013] Based on the annotation rules and annotation process, the CTA heart chamber data, CTA coronary artery data, and CT heart chamber data in the basic data are structurally reconstructed and exported to obtain the three-dimensional mask data corresponding to the CTA heart chamber data, CTA coronary artery data, and CT heart chamber data respectively;

[0014] The voxel data in the three-dimensional mask data is read and converted into a three-dimensional array. The three-dimensional array is adjusted to a unified coordinate system after affine transformation. For the cardiac chambers and coronary artery structures marked in the three-dimensional mask data, the voxel data in the eight-neighborhood of each marked cardiac chamber and coronary artery structure are iteratively traversed to determine the first boundary point cloud of each cardiac chamber and the second boundary point cloud of the coronary artery structure, and the first boundary information between the cardiac chambers and the second boundary information between the coronary artery structures are extracted. The first boundary point cloud, the second boundary point cloud, the first boundary information and the second boundary information are standardized in size and stored as cardiac chamber point cloud data and reconstructed point cloud data, respectively.

[0015] Preferably, the step of removing the aorta portion from the reconstructed point cloud data to obtain the coronary artery point cloud data comprises:

[0016] Reading the aorta point cloud in the reconstructed point cloud data and the heart chamber point cloud data, and removing the part of the reconstructed point cloud data whose difference with the aorta point cloud is less than a preset threshold, so as to obtain the roughly segmented point cloud data of the coronary artery;

[0017] Abnormal clusters in the roughly segmented point cloud data of the coronary arteries are excluded, and the distance between the centroid of the remaining clusters in the roughly segmented point cloud data of the coronary arteries and the centroid of the aorta point cloud is calculated. The two remaining clusters with the largest distance are retained to obtain the coronary artery point cloud data defined by the left branch of the coronary artery and the right branch of the coronary artery, respectively, and the coronary artery point cloud data and the heart chamber point cloud data are standardized and stored.

[0018] Preferably, the step of building a coronary artery inference network based on the cardiac chamber point cloud data and the coronary artery point cloud data comprises:

[0019] Building a convolutional neural network using an encoder-decoder structure so that the convolutional neural network is used as a point cloud completion network for growth inference of coronary arteries;

[0020] Acquire cardiac chamber point cloud data and coronary artery point cloud data to be processed, and divide the cardiac chamber point cloud data and coronary artery point cloud data into training set data, validation set data, and test set data;

[0021] With the coronary artery point cloud data as the target point cloud data and the heart chamber point cloud data as the input point cloud data, the convolutional neural network is trained to obtain the coronary artery inference network.

[0022] Preferably, the steps of training a convolutional neural network using coronary artery point cloud data as target point cloud data and cardiac chamber point cloud data as input point cloud data to obtain a coronary artery inference network include:

[0023] Resample the target point cloud data and the heart chamber point cloud data and arrange them into the size of input to the convolutional neural network;

[0024] Initialize the convolutional neural network and load the network parameters pre-trained on the ShapeNet dataset into the convolutional neural network;

[0025] Setting a pre-hyperparameter for the convolutional neural network, wherein the pre-hyperparameter includes a pre-weight and a post-weight, the pre-weight increases the weight of the loss function on the coarse segmentation point cloud data of the coronary artery, and the post-weight increases the weight of the loss function on the fine segmentation point cloud data of the coronary artery;

[0026] Use stochastic gradient descent to optimize the convolutional neural network, and select the network parameters with the smallest L1 chamfer distance on the validation set data for storage.

[0027] Preferably, the loss function Loss Multiscale for:

[0028]

[0029] Where S fine and S coarseThey are the initial dense coronary point cloud and the sparse coronary point cloud after sampling. To predict point clouds, λ is a hyperparameter whose value ranges from 1 to 0.01;

[0030] The L1 chamfer distance is calculated by the following formula:

[0031]

[0032] Among them, S1 and S2 are two point clouds for calculating the chamfer distance.

[0033] Preferably, the steps of building a convolutional neural network using a codec structure so that the convolutional neural network is used as a point cloud completion network for growth inference of coronary arteries of the heart include:

[0034] Obtain a convolutional neural network using an encoder-decoder structure;

[0035] Two PointNet layers are used as encoders in the convolutional neural network, and the encoding layer width is set to 512. In the decoding stage, a decoder based on a multi-layer perceptron is used to obtain the output sparse point cloud with local generation blocks, where the local generation blocks are generated based on the following formula:

[0036]

[0037] Among them, Y fine is the dense point cloud matrix, Y coarse is the sparse point cloud matrix, is a matrix that resamples the sparse point cloud matrix to the size of a dense point cloud matrix, seed is the folding seed, and F is a multi-layer perceptron used to transform Y coarse The point cloud matrix after relative seed expansion is converted to fine Matrices of matching dimensions.

[0038] Preferably, the step of acquiring a test sample and inputting the test point cloud data of the test sample into the coronary artery inference network to output the growth spatial position of the coronary artery in the test sample comprises:

[0039] Acquire a test sample, and input the test point cloud data of the test sample into the coronary artery inference network, and locate the coronary artery position of the coronary artery point cloud data of the test sample after the output of the coronary artery inference network converges, so as to obtain a predicted coronary artery point cloud;

[0040] The L1 chamfer distance, L2 chamfer distance and evaluation index FScore of the predicted coronary artery point cloud of the test sample and the real coronary artery point cloud of the test sample are calculated to infer the accuracy of the output coronary artery position of the coronary artery inference network.

[0041] Preferably, the evaluation index FScore is calculated by the following formula:

[0042]

[0043] The precision rate represents the probability that the actual test sample is a positive sample among the test samples predicted to have positive coronary artery point clouds, and the recall rate represents the probability that the actual test sample is a positive sample among the test samples predicted to have positive coronary artery point clouds.

[0044] The present invention also discloses a coronary artery growth inference system, comprising:

[0045] Chamber point cloud acquisition module, which acquires the cardiac chamber point cloud data obtained by CTA technology and the reconstructed point cloud data of the coronary artery;

[0046] A coronary artery point cloud acquisition module removes the aorta portion from the reconstructed point cloud data to obtain coronary artery point cloud data;

[0047] The coronary point cloud inference module builds a coronary artery inference network based on the heart chamber point cloud data and coronary artery point cloud data;

[0048] A drawing module obtains a test sample and inputs the test point cloud data of the test sample into the coronary artery inference network to output the growth spatial position of the coronary artery in the test sample.

[0049] Compared with the prior art, the above technical solution has the following beneficial effects:

[0050] 1. Compared with the existing technology, the present invention innovatively introduces a new point cloud completion network technology to complete the specific coronary artery growth inference function. Based on the powerful ability of deep learning, it focuses on inferring and restoring the complete three-dimensional structure from sparse or partially observed point cloud data, making up for the functional deficiency of the existing generation technology in the perception of coronary artery position;

[0051] 2. For the first time, point cloud is used to generatively infer the coronary artery structure from the heart chambers. The use of point cloud modality makes up for the shortcomings of existing voxel-based and grid-based 3D technologies in terms of speed and storage usage.

[0052] 3. It solves the problem that the coronary arteries are difficult to observe in the existing technology in contrast-free three-dimensional cardiac imaging (including cardiac MRI, cardiac CT, etc.), which is helpful for low-dose and dose-free coronary three-dimensional reconstruction and coronary heart disease diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a flow chart of a method for inferring the growth of coronary arteries in accordance with a preferred embodiment of the present invention;

[0054] Figure 2 A schematic diagram of a local generation block in accordance with a preferred embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the comparison between the predicted coronary artery point cloud and the actual coronary artery point cloud in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0056] The advantages of the present invention are further described below in conjunction with the accompanying drawings and specific embodiments.

[0057] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0058] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0059] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0060] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0061] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0062] In the following description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module" and "component" can be used interchangeably.

[0063] See also Figure 1 , shows a flow chart of a method for inferring the growth of coronary arteries in accordance with a preferred embodiment of the present invention. In this embodiment, the method for inferring the growth of coronary arteries includes the following steps:

[0064] S100: Acquire the heart chamber point cloud data obtained by CTA technology and the reconstructed point cloud data of the coronary artery;

[0065] Point cloud data, as a collection of vectors in a three-dimensional coordinate system, is widely used in fields such as autonomous driving. Its completion technology is an important generation and estimation problem, which has a profound impact on the field of 3D computer vision. Point cloud is a massive point set that expresses the spatial distribution and surface characteristics of the target in the same spatial reference system. The properties of point cloud include spatial resolution, point position accuracy, surface normal vector, etc. Sparse point cloud or dense point cloud are the basis of reverse modeling. Point cloud data is the most common and basic three-dimensional model. Point cloud models are often obtained directly from measurements, and each point corresponds to a measurement point without other processing methods, so it contains the largest amount of information. However, this information is hidden in the point cloud and needs to be extracted by other extraction methods. The process of extracting information from the point cloud is three-dimensional image processing.

[0066] In the existing technology (such as the completion technology for radar data in the field of autonomous driving), it is usually only possible to complete the surface point cloud scanned at a single angle. However, the present invention needs to infer the spatial position and structure of the coronary artery based on the heart chamber information. Taking into account that fat surrounding and myocardial information are difficult to obtain accurately, there are theoretically multiple possibilities for the reasonable spatial position of the coronary artery corresponding to the same heart chamber information. At the same time, for the segmentation of the supervisory information, i.e., the blood vessels, there will also be label variability because the segmentation endpoint branch is difficult to determine. In addition, small strip-shaped objects are objects whose structures are difficult to accurately infer using existing completion technologies. Therefore, existing completion technologies are difficult to directly apply to the coronary artery growth inference work of the present invention, and require a lot of changes to them.

[0067] On the other hand, CTA is expensive and highly radioactive. It can be used for coronary artery reconstruction, but there are also some medical imaging modalities that can obtain cardiac chamber information but have poor coronary artery visibility, such as CT and low-dose CTA. In order to minimize radiation and reduce the cost of coronary artery observation, the use of more easily accessible cardiac chamber point cloud data to infer the more difficult to obtain coronary artery position can help enhance coronary artery images without contrast agents, which has economic and social benefits. The same effect can be achieved using paired volume data to generate adversarial networks, but the raw volume data involved consumes a lot of memory and it is difficult to form a continuous vascular structure output. The point cloud-based method can more quickly and clearly form the output of the coronary artery position, and specific voxel points in non-enhanced images can be observed in a targeted manner.

[0068] Therefore, by collecting and standardizing the three-dimensional heart chamber point cloud data, using the heart chamber positions in three-dimensional cardiac medical images such as CT or CMR, the growth direction and spatial position of the coronary arteries in three-dimensional space are inferred and analyzed, and the reconstructed point cloud data of the coronary arteries can be obtained.

[0069] S200: removing the aorta portion from the reconstructed point cloud data to obtain coronary artery point cloud data;

[0070] Through this step, the parts that do not produce coronary arteries can be removed, and the final result is clearer.

[0071] S300: Building a coronary artery inference network based on cardiac chamber point cloud data and coronary artery point cloud data;

[0072] The coronary artery is the main blood vessel supplying blood to the human heart and is crucial for understanding and diagnosing heart disease. Traditional coronary artery diagnosis methods usually rely on the experience and expertise of doctors, but this method has problems of subjectivity and error. To solve this problem, an automated coronary artery inference network can be established using point cloud data and deep learning technology. Heart chamber point cloud data and coronary artery point cloud data can provide information about the internal structure and vascular distribution of the heart. The first step in building a coronary artery inference network is to preprocess and extract features from the input point cloud data. This can include operations such as removing noise, sampling and reconstructing point clouds to improve the quality and accuracy of the data. Then, deep learning models such as convolutional neural networks (CNN) or graph convolutional networks (GCN) can be used to process and analyze the point cloud data.

[0073] In the design of the network, a multi-level feature extraction and fusion method can be used to effectively capture the shape and structural features of the coronary arteries. For example, multiple convolutional layers and pooling layers can be used to extract features at different levels, and these features can be fused through jump connections or attention mechanisms. In addition, some advanced network structures and technologies, such as residual connections, batch normalization, and self-attention mechanisms, can be introduced to further improve the performance and stability of the network.

[0074] The key to training a coronary artery inference network is to build a high-quality training dataset. This can be achieved through medical imaging data and expert annotation. The annotation process can include manually drawing the outline and branching structure of the coronary artery, or using an automated algorithm for auxiliary annotation. With a large amount of training data and appropriate data augmentation methods, the generalization ability and robustness of the network can be improved.

[0075] After the network training is completed, the new heart chamber point cloud data can be input into the network to obtain the inference results of the coronary arteries. Therefore, the coronary artery inference network can be combined with other medical image analysis algorithms to improve the segmentation and recognition performance of the coronary arteries.

[0076] S400: Acquire a test sample, and input the test point cloud data of the test sample into a coronary artery inference network to output the growth spatial position of the coronary artery in the test sample.

[0077] First, it is crucial to obtain test samples. This can be achieved in a variety of ways, such as obtaining cardiac imaging data from a hospital or medical research institution. These imaging data can be the results from medical imaging technologies such as magnetic resonance imaging (MRI) or computed tomography (CT). These imaging data are usually stored in DICOM format and contain detailed information about the heart structure and coronary arteries.

[0078] Next, the heart image data in the test sample needs to be preprocessed to extract the point cloud data of the coronary arteries. This involves image segmentation algorithms to separate the heart area from other tissues and structures. Then, a 3D reconstruction algorithm can be used to convert the heart's voxel data into point cloud data to describe the 3D shape and structure of the coronary arteries.

[0079] After obtaining the point cloud data of the test sample, it can be input into the coronary artery inference network. Before training the network, it is necessary to prepare labeled training data sets. These data sets should contain the labeled information of the coronary arteries, that is, the growth spatial position of each point. This can be achieved through manual labeling or using automated algorithms for auxiliary labeling. Through a large number of training samples and appropriate data enhancement techniques, the learning ability and generalization ability of the network can be improved. Once the network training is completed, the point cloud data of the test sample can be input into the network to obtain the predicted results of the growth spatial position of the coronary arteries. These results can be the position of each point in three-dimensional coordinates or the segmentation results of the coronary arteries.

[0080] Through the above technical solution, the cost threshold for precise positioning of the coronary arteries is significantly reduced by using an economical, efficient and easy-to-operate three-dimensional spatial positioning solution for the coronary arteries. It can intuitively and accurately display the position and morphology of the coronary arteries and perform directional non-enhanced imaging coronary artery reconstruction.

[0081] In a preferred embodiment, step S100 includes:

[0082] S110: Collect CTA cardiac volume data files, CTA clinical data medical records, CT cardiac volume data files and CT clinical data medical records from multiple hospital units at different levels and in different regions as basic data, pre-process the basic data, and establish a basic database. The pre-processing process includes clearing, renaming and format conversion of the basic data. Exemplarily, in this embodiment, cardiac CTA is used as the volume data to be processed, and its DICOM file is obtained.

[0083] S120: Based on the labeling rules and labeling process, the CTA heart chamber data, CTA coronary artery data, and CT heart chamber data in the basic data are structurally reconstructed and exported to obtain three-dimensional mask data corresponding to the CTA heart chamber data, CTA coronary artery data, and CT heart chamber data, respectively.

[0084] The annotation process includes: clarifying the annotation object, personnel division of labor, annotation steps, result review, unified annotation rules, confirming the meaning, form and output format of the annotation results, and describing the decision-making mechanism. The annotation objects and annotation tasks include: clarifying the annotation objects as the heart chamber structure and the coronary artery of the heart, and outlining the results of the operation of senior physicians recognized by the industry as the gold standard, and all must be reviewed and confirmed by the auditors before entering the next step. After annotating with the outline software, it is exported in nii format; among them, the heart chamber structure needs to have a clear boundary definition. Exemplarily, in CTA, the embodiment of the present invention outlines multiple chambers of the heart and arteriovenous data based on voxels, and the structures of interest include left and right atria, left and right ventricles, left ventricular wall, aortic root, pulmonary artery root, left and right pulmonary veins, left atrial appendage, etc. Based on the AW4.7 workstation configured by GE Revolution APEX CT, the coronary artery is reconstructed by voxel using intensity data and manually checked. AW4.7 configured by GE Revolution Apex CT refers to the advanced software platform equipped with the CT system. AW4.7 is a workstation software developed by GE Medical Systems for the RevolutionCT series. It provides advanced image processing, diagnostic tools and analysis functions, enabling doctors and researchers to interpret and analyze medical images more efficiently. AW4.7 has the following features: Advanced image processing: AW4.7 provides advanced image processing technology, including noise reduction, contrast enhancement and 3D reconstruction. Intelligent diagnostic tools: Built-in algorithms can help doctors identify and locate lesions. Advanced analysis functions: Supports a variety of image analysis technologies, such as energy spectrum analysis. User-friendly interface: The intuitive user interface makes operation easier and allows you to quickly obtain the required information. Data management: Powerful data management functions make it easy to organize, store and retrieve images and diagnostic reports.

[0085] The labeling rules include: During the labeling process, the labeling quality of the labelers should be supervised, and the consistency index and accuracy index should be considered at the same time. When the performance of the labelers drops significantly, the labeling work should be suspended, the labelers should be retrained, and they should be reinstated after passing the assessment. After labeling, the accuracy, consistency, comprehensibility, accessibility, data security, and traceability should be evaluated.

[0086] S130: Read voxel data in the three-dimensional mask data and convert the voxel data into a three-dimensional array, adjust the three-dimensional array to a unified coordinate system after affine transformation, iterate the voxel data in the eight-neighborhood of each labeled heart chamber and coronary artery structure in the three-dimensional mask data to determine the first boundary point cloud of each heart chamber and the second boundary point cloud of the coronary artery structure, extract the first boundary information between the heart chambers and the second boundary information between the coronary artery structures, normalize the sizes of the first boundary point cloud, the second boundary point cloud, the first boundary information and the second boundary information, and store them as heart chamber point cloud data and reconstructed point cloud data, respectively.

[0087] Illustratively, this embodiment samples the following structures based on voxel data annotation: left ventricle, left ventricular wall, right ventricle, right atrium, left atrium, aorta, and coronary arteries.

[0088] In a preferred embodiment, step S200 includes:

[0089] S210: reading the aorta point cloud in the reconstructed point cloud data and the heart chamber point cloud data, and removing the portion of the reconstructed point cloud data whose difference with the aorta point cloud is less than a preset threshold, so as to obtain the roughly segmented point cloud data of the coronary artery;

[0090] Exemplarily, the preset threshold value is taken as 0.01 unit length, which is 2 mm according to the standardized scale.

[0091] S220: excluding abnormal clusters in the roughly segmented coronary artery point cloud data, and calculating the distance between the centroid of the remaining clusters in the roughly segmented coronary artery point cloud data and the centroid of the aorta point cloud, retaining the two remaining clusters with the largest distance to obtain coronary artery point cloud data defined by the left coronary artery branch and the right coronary artery branch, and storing the coronary artery point cloud data and the heart chamber point cloud data in a standardized manner.

[0092] Exemplarily, DBSCAN clustering is used, and clusters with less than 800 points are defined as abnormal clusters to be excluded.

[0093] Further, step S300 includes:

[0094] S310: constructing a convolutional neural network using a codec structure, so that the convolutional neural network is used as a point cloud completion network for growth inference of coronary arteries of the heart;

[0095] S320: Obtain standardized cardiac chamber point cloud data and coronary artery point cloud data to be processed, and divide the cardiac chamber point cloud data and coronary artery point cloud data into training set data, validation set data, and test set data; illustratively, this embodiment uses 7:1:2 as the ratio of the number of cases in the training set, validation set, and test set. The sample size is 140 pairs of cardiac chamber point cloud data and coronary artery point cloud data for the training set, 20 pairs of cardiac chamber point cloud data and coronary artery point cloud data for the validation set, and 40 pairs of cardiac chamber point cloud data and coronary artery point cloud data for the test set.

[0096] S330: Using the coronary artery point cloud data as target point cloud data and the heart chamber point cloud data as input point cloud data, a convolutional neural network is trained to obtain a coronary artery inference network.

[0097] Among them, the point cloud completion network can be an autoencoder or a convolutional neural network such as PF-Net. Exemplarily, the point cloud completion network in the preferred embodiment of the present invention is a convolutional neural network using an encoder-decoder structure. Two PointNet layers are used as encoders, and the encoding layer width is set to 512. In the decoding stage, a decoder based on a multi-layer perceptron is first used to obtain an output sparse point cloud. Since it is difficult to describe the local bending morphology using only a multi-layer perceptron to learn the morphology of morphological seeds, an additional nonlinear setting is used in the process of generating dense point clouds. The improved dense point cloud decoder is called a local generation block. The point cloud completion network with a local generation block can generate the corresponding coronary artery point cloud based on the input chamber point cloud inference, thereby realizing the spatial perception of the coronary arteries of the heart. The multi-layer perceptron is followed by the FoldingNet block in the decoder. It can directly generate a complete point cloud from a partial point cloud by combining local information and global information. The point cloud completion network is substantially different from the processing of radar data in autonomous driving technology by a similar structure. In other words, the process of generating a dense point cloud by outputting a sparse point cloud and fusing local information in the present invention involves a fine depiction of the coronary artery from the trunk to possible branches. Therefore, an additional nonlinear setting is adopted in the process of generating dense point clouds, and the improved dense point cloud decoder is called Figure 2 Compared with the morphology of folded seeds learned only by multi-layer perceptron, the local generation block in this embodiment better completes the local curvature morphology of the coronary artery and the regularity of the branching.

[0098] More specifically, step S330 includes:

[0099] S331: Resample the target point cloud data and the heart chamber point cloud data and arrange them into a size for input to the convolutional neural network; exemplarily, 16384 is used as the resampling number of the input heart chamber point cloud data, and 16384 is used as the resampling number of the output coronary artery point cloud data.

[0100] S332: Initialize the convolutional neural network, and load the network parameters pre-trained on the ShapeNet dataset into the convolutional neural network; wherein the initialization method can be random initialization or pre-trained parameter initialization. Exemplarily, the preferred embodiment of the present invention loads the network parameters pre-trained on the ShapeNet dataset.

[0101] S333: setting a pre-hyperparameter for the convolutional neural network, wherein the pre-hyperparameter includes a pre-weight and a post-weight, the pre-weight increases the weight of the loss function on the roughly segmented point cloud data of the coronary artery, and the post-weight increases the weight of the loss function on the finely segmented point cloud data of the coronary artery;

[0102] S334: Use stochastic gradient descent to optimize the convolutional neural network, and select the network parameters with the smallest L1 chamfer distance (L1 chamfer distance) of the convolutional neural network on the validation set data as the best performing network parameters for storage.

[0103] More specifically: Loss function Multiscale for:

[0104]

[0105] Where S fine and S coarse They are the initial dense coronary point cloud and the sparse coronary point cloud after sampling. To predict point clouds, λ is a hyperparameter whose value ranges from 1 to 0.01. In this embodiment, the training batch size is 64, and the number of training steps is used as the basis for changing λ. When the number of training steps is less than 2000, λ is 1; when the number of training steps is greater than 2000 but less than 5000, λ is 0.5; when the number of training steps is greater than 5000 but less than 8000, λ is 0.1; when the number of training steps is greater than 5000 but less than 8000, λ is 0.01. That is, in the early stage of training, the network pays more attention to the global structure of the point cloud, and in the later stage of training, it pays more attention to the fine point cloud structure.

[0106] The L1 chamfer distance is calculated by the following formula:

[0107]

[0108] Among them, S1 and S2 are two point clouds for calculating the chamfer distance, which are the predicted coronary artery point cloud and the real coronary artery point cloud inferred by the network in the validation dataset in this applicable scenario.

[0109] Optionally, step S310 includes:

[0110] S311: Obtain a convolutional neural network using a codec structure;

[0111] S312: Two PointNet layers are used as encoders in a convolutional neural network, and the encoding layer width is set to 512. In the decoding stage, a decoder based on a multi-layer perceptron is used to obtain an output sparse point cloud with local generation blocks, where refer to Figure 2 , the local generation block is generated based on the following formula:

[0112]

[0113] Among them, Y fine is the dense point cloud matrix, Y coarse is the sparse point cloud matrix, is a matrix that resamples the sparse point cloud matrix to the size of a dense point cloud matrix, seed is the folding seed, and F is a multi-layer perceptron used to transform Y coarse The point cloud matrix after relative seed expansion is converted to fine Matrix of matching size. For example, is a dense coronary point cloud, yes After expansion, λ is a hyperparameter with a value of 0.1.

[0114] In a preferred embodiment, step S400 includes:

[0115] S410: Acquire a test sample and use the trained point cloud completion network to verify the accuracy of the position of the coronary artery inferred from the cardiac chamber point cloud data. Specifically, input the test point cloud data of the test sample into the coronary artery inference network, and locate the coronary artery position of the coronary artery point cloud data of the test sample after the output of the coronary artery inference network converges to obtain a predicted coronary artery point cloud; illustratively, this embodiment includes 40 pairs of test samples.

[0116] S420: Calculate the accuracy index of the predicted coronary artery point cloud and the real coronary artery point cloud. Specifically, calculate the L1 chamfer distance, L2 chamfer distance and evaluation index FScore of the predicted coronary artery point cloud of the test sample and the real coronary artery point cloud of the test sample, and infer the accuracy of the output coronary artery position of the coronary artery inference network. The accuracy index can be an index reflecting the overall distance between point clouds, or it can be an index of the local distance from the key structure of the heart. Exemplarily, in this embodiment, L1 chamfer distance, L2 chamfer distance and FScore are used as accuracy indicators. The L1 chamfer distance is as shown above, and the L2 chamfer distance replaces the 1 norm of the L1 chamfer distance with the square of the 2 norm.

[0117] The evaluation index FScore is calculated by the following formula:

[0118]

[0119] With a certain distance threshold as the limit, when the minimum L1 chamfer distance between the predicted coronary artery point cloud and the real coronary artery point cloud is less than a certain value, it is regarded as a positive sample relative to the real coronary artery point cloud. The precision rate represents the probability that the actual test sample is a positive sample among the test samples where the predicted coronary artery point cloud is positive, and the recall rate represents the probability that the actual test sample is a positive sample among the test samples where the predicted coronary artery point cloud is positive. The distance threshold in this embodiment is 0.01 unit length, i.e., 2 mm.

[0120] Specifically, in the test set consisting of 40 test samples, the average L1 chamfer distance obtained by using the local generation module is 12.56×1e-3 unit lengths, the average L2 chamfer distance is 8.01×1e-4 unit lengths, and the average FScore is 57.18; the average L1 chamfer distance obtained without using the local generation block is 15.19×1e-3 unit lengths, the average L2 chamfer distance is 12.19×1e-4 unit lengths, and the average FScore is 50.82. This shows that the local generation block has a strong implementation necessity and better implementation performance for the coronary artery growth inference method. In actual situations, the main branches of the coronary arteries have more possible directions when they are divided into secondary branches, and manual segmentation is difficult to fully summarize the situation of finer branches. Figure 3 The figure shows a schematic diagram of the heart chamber point cloud data, the predicted coronary artery point cloud and the actual coronary artery point cloud using the local generation block, reflecting that the local generation block can better present this phenomenon.

[0121] The present invention also discloses a coronary artery growth inference system, comprising:

[0122] Chamber point cloud acquisition module, which acquires the cardiac chamber point cloud data obtained by CTA technology and the reconstructed point cloud data of the coronary artery;

[0123] A coronary artery point cloud acquisition module removes the aorta portion from the reconstructed point cloud data to obtain coronary artery point cloud data;

[0124] The coronary point cloud inference module builds a coronary artery inference network based on the heart chamber point cloud data and coronary artery point cloud data;

[0125] A drawing module obtains a test sample and inputs the test point cloud data of the test sample into the coronary artery inference network to output the growth spatial position of the coronary artery in the test sample.

[0126] It should be noted that the embodiments of the present invention have better practicability and do not impose any form of limitation on the present invention. Any technician familiar with the field may use the technical content disclosed above to change or modify it into an equivalent effective embodiment. However, any modification or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for estimating the growth of coronary arteries of a heart, characterized in that: The following steps are involved: Acquire the heart chamber point cloud data obtained by CTA technology, as well as the reconstructed point cloud data of the coronary arteries; The aorta part in the reconstructed point cloud data is removed to obtain the coronary artery point cloud data; Building a coronary artery inference network based on the heart chamber point cloud data and the coronary artery point cloud data; A test sample is acquired, and test point cloud data of the test sample is input into the coronary artery inference network to output the growth spatial position of the coronary artery in the test sample.

2. The growth inference method according to claim 1, characterized in that: The steps of obtaining the heart chamber point cloud data obtained by CTA technology and the reconstructed point cloud data of the coronary arteries include: Collect CTA cardiac volume data files, CTA clinical data medical record information, CT cardiac volume data files and CT clinical data medical record information from multiple hospital units as basic data, and establish a basic database after preprocessing the basic data; Based on the annotation rules and annotation process, the CTA heart chamber data, CTA coronary artery data, and CT heart chamber data in the basic data are structurally reconstructed and exported to obtain the three-dimensional mask data corresponding to the CTA heart chamber data, CTA coronary artery data, and CT heart chamber data respectively; The voxel data in the three-dimensional mask data is read and converted into a three-dimensional array, and the three-dimensional array is adjusted to a unified coordinate system after affine transformation. For the cardiac chambers and coronary artery structures that have been labeled in the three-dimensional mask data, the voxel data in the eight-neighborhood of each labeled cardiac chamber and coronary artery structure are iteratively traversed to determine the first boundary point cloud of each cardiac chamber and the second boundary point cloud of the coronary artery structure, and the first boundary information between the cardiac chambers and the second boundary information between the coronary artery structures are extracted, and the first boundary point cloud, the second boundary point cloud, the first boundary information and the second boundary information are size-standardized and stored as cardiac chamber point cloud data and reconstructed point cloud data, respectively.

3. The growth inference method according to claim 1, characterized in that: The steps of removing the aorta portion from the reconstructed point cloud data to obtain the coronary artery point cloud data include: Reading the aorta point cloud in the reconstructed point cloud data and the heart chamber point cloud data, and removing the portion of the reconstructed point cloud data whose difference with the aorta point cloud is less than a preset threshold, so as to obtain the roughly segmented point cloud data of the coronary artery; Abnormal clusters in the coronary artery coarse segmentation point cloud data are excluded, and the distance between the centroid of the remaining clusters in the coronary artery coarse segmentation point cloud data and the centroid of the aorta point cloud is calculated, and the two remaining clusters with the largest distance are retained to obtain coronary artery point cloud data defined by the left coronary artery branch and the right coronary artery branch, respectively, and the coronary artery point cloud data and the heart chamber point cloud data are standardized and stored.

4. The growth inference method according to claim 3, characterized in that: The steps of building a coronary artery inference network based on the cardiac chamber point cloud data and the coronary artery point cloud data include: Building a convolutional neural network using a codec structure, so that the convolutional neural network is used as a point cloud completion network for growth inference of coronary arteries of the heart; Acquire cardiac chamber point cloud data and coronary artery point cloud data to be processed, and divide the cardiac chamber point cloud data and coronary artery point cloud data into training set data, validation set data and test set data; With the coronary artery point cloud data as the target point cloud data and the heart chamber point cloud data as the input point cloud data, the convolutional neural network is trained to obtain the coronary artery inference network.

5. The growth inference method according to claim 4, characterized in that: The steps of training a convolutional neural network using coronary artery point cloud data as target point cloud data and heart chamber point cloud data as input point cloud data to obtain a coronary artery inference network include: Resampling the target point cloud data and the heart chamber point cloud data to a size that is input to the convolutional neural network; initializing the convolutional neural network and loading the network parameters pre-trained on the ShapeNet dataset into the convolutional neural network; Setting a pre-hyperparameter for the convolutional neural network, wherein the pre-hyperparameter includes a pre-weight and a post-weight, wherein the pre-weight increases the weight of the loss function on the coarse segmentation point cloud data of the coronary artery, and the post-weight increases the weight of the loss function on the fine segmentation point cloud data of the coronary artery; The convolutional neural network is optimized using stochastic gradient descent, and the network parameters of the convolutional neural network with the smallest L1 chamfer distance on the validation set data are selected for storage.

6. The growth inference method according to claim 5, characterized in that: The loss function Loss Multiscale for: Where S fine and S coarse They are the initial dense coronary point cloud and the sparse coronary point cloud after sampling. To predict point clouds, λ is a hyperparameter whose value ranges from 1 to 0.01; The L1 chamfer distance is calculated by the following formula: Among them, S1 and S2 are two point clouds for calculating the chamfer distance.

7. The growth inference method according to claim 4, characterized in that: The steps of building a convolutional neural network using a codec structure so that the convolutional neural network is used as a point cloud completion network for growth inference of coronary arteries of the heart include: Obtain a convolutional neural network using an encoder-decoder structure; Two PointNet layers are used as encoders in the convolutional neural network, and the encoding layer width is set to 512. In the decoding stage, a decoder based on a multi-layer perceptron is used to obtain an output sparse point cloud with local generation blocks, where the local generation blocks are generated based on the following formula: Among them, Y fine is the dense point cloud matrix, Y coarse is the sparse point cloud matrix, is a matrix that resamples the sparse point cloud matrix to the size of a dense point cloud matrix, seed is the folding seed, and F is a multi-layer perceptron used to transform Y coarse The point cloud matrix after relative seed expansion is converted to fine Matrices of matching dimensions.

8. The growth inference method according to claim 1, wherein: The steps of acquiring a test sample and inputting the test point cloud data of the test sample into the coronary artery inference network to output the growth spatial position of the coronary artery in the test sample include: Acquire a test sample, and input the test point cloud data of the test sample into the coronary artery inference network, and locate the coronary artery position of the coronary artery point cloud data of the test sample after the output of the coronary artery inference network converges, so as to obtain a predicted coronary artery point cloud; The L1 chamfer distance, L2 chamfer distance and evaluation index FScore of the predicted coronary artery point cloud of the test sample and the real coronary artery point cloud of the test sample are calculated to infer the accuracy of the output coronary artery position of the coronary artery inference network.

9. The growth inference method according to claim 8, characterized in that: The evaluation index FScore is calculated by the following formula: The precision rate represents the probability that the actual test sample is a positive sample among the test samples predicted to have positive coronary artery point clouds, and the recall rate represents the probability that the actual test sample is a positive sample among the test samples predicted to have positive coronary artery point clouds.

10. A coronary artery growth inference system, characterized in that: include: Chamber point cloud acquisition module, which acquires the cardiac chamber point cloud data obtained by CTA technology and the reconstructed point cloud data of the coronary artery; The coronary artery point cloud acquisition module removes the aorta part from the reconstructed point cloud data to obtain the coronary artery point cloud data; The coronary point cloud inference module builds a coronary artery inference network based on the cardiac chamber point cloud data and the coronary artery point cloud data; The drawing module obtains a test sample and inputs the test point cloud data of the test sample into the coronary artery inference network to output the growth spatial position of the coronary artery in the test sample.

Images