A method for extracting coronary artery centerline

Abstract

This invention discloses a method for extracting the coronary artery centerline, comprising the following steps: loading a centerline reference file and a CT image file from the CAT08 dataset and performing preprocessing to obtain a preprocessed dataset class; constructing a convolutional neural network model, wherein the convolutional neural network model includes a radius regression model, a direction multi-classification model, and a deviation correction binary classification model; reading data from the preprocessed dataset class and inputting the read data into the convolutional neural network model for training and validation to obtain a trained model; and constructing a tracker based on the trained model, starting from an initial reference point and iteratively advancing a distance predicting the radius of the blood vessel in the predicted direction to extract the coronary artery centerline. This invention provides a tracker with extremely strong centerline extraction capabilities.

Description

Technical Field

[0001] This invention belongs to the field of coronary artery centerline extraction technology, and particularly relates to a method for coronary artery centerline extraction. Background Technology

[0002] Numerous scholars have conducted research on the problem of coronary artery centerline extraction and proposed various methods. These mainly include traditional segmentation methods and deep learning-based methods.

[0003] Traditional segmentation methods can be summarized into methods based on centerline tracing, topology refinement, minimum cost path, and distance transformation.

[0004] Existing technology includes a centerline extraction method based on a tubular tissue inscribed sphere model (MIBTT). This method performs two distance transformations on the original graphic to search for the graphic boundary and remove unnecessary voxels, and constructs a maximum inscribed sphere model to find the skeleton reflecting the shape of the original graphic. Finally, it uses a principle similar to bisection to process the skeleton lines to obtain the centerline. However, this method involves two distance transformations and the construction of a maximum inscribed sphere model, resulting in high computational complexity. Furthermore, the algorithm is sensitive to noise, which can cause the skeleton lines to deviate from the true centerline.

[0005] Another existing technology is the centerline extraction method based on twelve-directional topology refinement (TDTT). This method introduces the concept of three-dimensional spatial criticality and refines the reconstructed blood vessels in parallel along the twelve sides of the cube according to three-dimensional morphological principles. That is, it deletes pixels that do not contain target points while preserving the coronary artery topology as much as possible. The algorithm also uses Dijkstra's algorithm to prune small branches in the blood vessels, which positively impacts the accuracy of topology refinement. However, this method considers less global information. This may lead to an inability to accurately determine the overall direction and topological structure of the centerline during extraction, thus affecting the accuracy of the final result.

[0006] For deep learning-based centerline extraction methods, a discriminative coronary artery tracking method exists. The neural network model designed in this study includes a backbone network composed of multiple convolutional layers, a tracker, and a discriminator. The discriminator model designed in this study can exclude other tubular tissues near the coronary arteries while the tracker extracts the vessel centerline, preventing the tracker from getting trapped in them, and can terminate the tracking process in time when the tracker reaches the vessel terminal. This model contains multiple convolutional layers, a tracker, and a discriminator, resulting in a relatively complex overall structure and high computational resource requirements.

[0007] In summary, all methods for extracting the centerline of blood vessels have certain limitations. For example, centerline tracing methods require setting the blood vessel as an ideal model to achieve good tracking results; centerline extraction methods based on minimum cost paths are prone to problems such as shortcuts, deviation from the center, and premature termination of the search; extraction methods based on topology refinement require a large amount of computation and have a long computation time; extraction methods based on distance transform cannot guarantee the single-pixel nature, continuity, and topological properties of the center pixel; deep learning methods require training on large datasets, standard datasets are difficult to establish, and their generalization ability may be affected when faced with medical images from different sources and of different qualities. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a method for extracting the coronary artery centerline, thereby resolving the issues present in the prior art.

[0009] To achieve the above objectives, the present invention provides a method for extracting the coronary artery centerline, comprising the following steps:

[0010] Load the centerline reference file and CT image file from the CAT08 dataset, and perform preprocessing to obtain the preprocessed dataset class;

[0011] Construct a convolutional neural network model, which includes a radius regression model, an orientation multi-classification model, and a bias correction binary classification model;

[0012] Data is read from the preprocessed dataset, and the read data is input into the convolutional neural network model for training and validation to obtain a trained model;

[0013] A tracker is built based on the trained model. Starting from the initial reference point, it iteratively moves the distance in the predicted direction to predict the radius of the blood vessel and extracts the center line of the coronary artery.

[0014] Preferably, the preprocessing method includes:

[0015] The centerline reference file is preprocessed and converted into a preprocessed file for training and testing convolutional neural network models;

[0016] Read CT images, obtain the metadata and three-dimensional array of the CT images, and obtain local CT images near the reference point based on spline interpolation.

[0017] A dataset class, inheriting from torch.utils.data.Dataset, is created using the preprocessed file and local CT images near the reference point.

[0018] Preferably, the method for preprocessing the centerline reference file includes:

[0019] The three-dimensional coordinates and vessel radius information of each reference point on the coronary artery centerline are extracted from the centerline reference file. The centerline direction at each reference point is determined according to the spatial relationship between the reference points. The centerline direction vector is converted into a point index on the Fibonacci sphere. New reference points outside the centerline are generated as negative samples based on the existing reference points. All reference point information is integrated and stored as a CSV file.

[0020] Preferably, the method for preprocessing CT images includes:

[0021] The process involves acquiring metadata and a 3D array from the CT image, generating a 19×19×19 3D grid centered at the origin, and moving the grid center to the reference point. Based on the metadata of the CT image, coordinate transformations are performed on all grid points to obtain their coordinates within the coordinate system of the CT image. The transformed grid coordinates are then used to interpolate the 3D array of the CT image to obtain a local image near the reference point, which is then stored in Nifti format.

[0022] Preferably, the first 6 layers of the convolutional neural network model consist of 6 convolutional blocks, and the 7th layer is a single 3D convolutional layer. Each convolutional block includes a 3D convolution operation, a batch normalization operation, and an activation operation. The construction parameters of the convolutional block are the number of input channels, the number of output channels, the kernel size, and the degree of dilation, respectively. The number of input and output channels increases, the kernel size decreases, and dilated convolution is only used in the third and fourth layers.

[0023] Preferably, the method for training the convolutional neural network model includes:

[0024] AdamW was selected as the optimizer for the model, and MultiStepLR was selected as the scheduler. The optimizer is used to automatically update the model parameters based on the loss and learning rate during training, and to add an L2 regularization term to the loss function. The scheduler is used to dynamically adjust the learning rate during training.

[0025] Preferably, the tracking method of the tracker includes:

[0026] The radius regression model predicts the vessel radius value based on the image, providing the tracker with tracking step length information; the direction multi-classification model lists the probability that each vector on the Fibonacci sphere is equal to the direction vector of the current position centerline, providing a reference for the tracker's forward direction; the deviation correction binary classification model analyzes whether the current point is on the centerline based on the image, determines whether the tracker deviates from the centerline, and terminates the tracker's tracking behavior when the tracker reaches the end of the vessel or the coronary ostium.

[0027] Preferably, the determination of whether to terminate the tracker is based on the termination entropy. The determination method includes: calculating the average termination entropy of the last 3 steps, and terminating the tracking when the average value exceeds a predetermined threshold.

[0028] The expression for calculating the termination entropy is:

[0029]

[0030] In the formula, D represents the number of direction classes, p i This represents the probability of each directional class in the output of the directional multi-classification model.

[0031] Preferably, after extracting the coronary artery centerline, the accuracy of the extracted coronary artery centerline is determined by the overlap, the overlap before the first error, the overlap of clinically relevant blood vessels, and the average distance index.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] This invention discloses a method for extracting the coronary artery centerline, comprising the following steps: loading a centerline reference file and a CT image file from the CAT08 dataset and preprocessing them to obtain a preprocessed dataset class; constructing a convolutional neural network model, which includes a radius regression model, a direction multi-classification model, and a deviation correction binary classification model; reading data from the preprocessed dataset class and inputting the read data into the convolutional neural network model for training and validation to obtain a trained model; and constructing a tracker based on the trained model, which iteratively moves the predicted distance of the predicted vessel radius from an initial reference point in the predicted direction to extract the coronary artery centerline. This invention utilizes a radius regression model, a direction multi-classification model, and a deviation correction binary classification model to construct the tracker, giving it a highly powerful centerline extraction capability. Attached Figure Description

[0034] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0035] Figure 1 This is a flowchart of the extraction method according to an embodiment of the present invention;

[0036] Figure 2 This is a flowchart illustrating the centerline reference file processing of an embodiment of the present invention.

[0037] Figure 3 This is a schematic diagram of a reference point outside the generation centerline in an embodiment of the present invention;

[0038] Figure 4 This is a structural diagram of a convolutional neural network model according to an embodiment of the present invention;

[0039] Figure 5 This is a function graph of the ReLU activation function according to an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the learning rate update plan for r_net and d_net in an embodiment of the present invention;

[0041] Figure 7 This is a diagram of the Sigmoid function according to an embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram of the tracking initialization in an embodiment of the present invention;

[0043] Figure 9 This is a schematic diagram illustrating the use of d_net to obtain the probabilities of all directional classes in x1 according to an embodiment of the present invention;

[0044] Figure 10 This is a schematic diagram illustrating the selection of the forward direction according to the angle with d1 in an embodiment of the present invention;

[0045] Figure 11 This is a schematic diagram illustrating the use of o_net to prevent the tracker from deviating from the center line in an embodiment of the present invention. Detailed Implementation

[0046] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0047] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0048] Example 1

[0049] like Figure 1 As shown, this embodiment provides a method for extracting the coronary artery centerline, including the following steps:

[0050] Load the centerline reference file and CT image file from the CAT08 dataset, and perform preprocessing to obtain the preprocessed dataset class;

[0051] Convolutional neural network models are constructed, including radius regression models, orientation multi-classification models, and bias correction binary classification models.

[0052] Read data from the preprocessed dataset, input the read data into the convolutional neural network model for training and validation, and obtain the trained model;

[0053] A tracker is built based on the trained model. Starting from the initial reference point, it iteratively moves the distance in the predicted direction to predict the radius of the blood vessel and extracts the center line of the coronary artery.

[0054] Furthermore, the preprocessing methods include:

[0055] The centerline reference file is preprocessed and converted into a preprocessed file for training and testing convolutional neural network models;

[0056] Read CT images, obtain the metadata and three-dimensional array of the CT images, and obtain local CT images near the reference point based on spline interpolation.

[0057] A dataset class, inheriting from torch.utils.data.Dataset, is created using the preprocessed file and local CT images near the reference point.

[0058] The specific data preprocessing methods are as follows:

[0059] After acquiring the dataset, the centerline reference file and CT image file are loaded from the CAT08 dataset. The data is then filtered, extracted, processed, and integrated to ultimately transform it into a dataset class that can be directly called by PyTorch.

[0060] This embodiment requires training three CNN models. As shown in Table 1, the input data required for each of the three models is different. The preprocessing process needs to provide the sample and label information required for each model.

[0061] Table 1

[0062]

[0063] The centerline reference file is processed as follows:

[0064] The centerline reference file processing procedure receives all 32 centerline reference files (reference.txt) from the CAT08 dataset, extracts the text data from them, performs a series of processing steps, and finally integrates the data into a new table and saves it to a CSV file.

[0065] The process first extracts reference point information from the centerline reference file, filtering out the 3D coordinates and vessel radius information at each reference point. Then, it iterates through all reference points and determines the centerline direction at each reference point based on their spatial relationships.

[0066] Because the centerline direction at a reference point is highly random, it's almost impossible to find a reference point with exactly the same direction. Directly using the centerline direction vector as the direction classification label for the d_net model leads to too many categories, too few samples per category, low model training accuracy, and poor training performance. Therefore, the direction vector must be converted into the index value of a point on a Fibonacci sphere to limit the number of direction classes.

[0067] In addition, since the correction model o_net requires reference points not on the centerline as negative samples during preprocessing, a new set of reference points not on the centerline must be generated based on the existing reference points. Finally, the coordinates, vessel radius, and orientation class index of all reference points are stored in a new sample table file.

[0068] Specific implementation as follows Figure 2 As shown:

[0069] (1) Extract reference point information from text file

[0070] The program uses numpy.loadtxt() to read the reference file, stores the 3D coordinates and blood vessel radius of each reference point into a special tuple point_tup, and then summarizes them into a list vessel for easy access to the reference point information in subsequent operations.

[0071] (2) Determine the direction of the centerline at the reference point

[0072] For each reference point on the centerline, obtain two points, pre and next, such that the distance from the two points to the reference point is approximately equal to the reference value of the blood vessel radius of that reference point. The vector from the reference point to the two points pre and next can represent the direction of the centerline at the reference point.

[0073] (3) Convert the direction vector into a direction class index

[0074] Use the Fibonacci grid sampling method to obtain a Fibonacci spherical shell with a sampling number of points_num. For each obtained centerline direction vector v, find the spherical shell vector vi that makes the smallest angle with it on the sphere, and obtain the index of vi on the shell.

[0075] (4) Generate a new reference point outside the centerline

[0076] The program receives information from a reference point on the centerline, obtains the coordinates of two directional vectors corresponding to the reference point, pre_coord and next_coord, calculates the cross product of the two vectors to obtain a new vector vect that is orthogonal to both directional vectors, adjusts the magnitude of vect to the radius of the blood vessel at the reference point, and adjusts the starting point of vect to the original reference point. At this time, the coordinates of vect are the coordinates of the reference point outside the centerline.

[0077] The positional relationship between the original reference point, the two direction vectors, and the new reference point is as follows: Figure 3 As shown.

[0078] (5) Save the reference information to a new file

[0079] The program extracts and processes reference points from different files, then converts them into a special tuple, sample_tup, and adds several key information items to identify which subset of the dataset the reference point comes from, which blood vessel it comes from, and whether it is a point on the centerline.

[0080] CT image preprocessing is as follows:

[0081] This process reads CT images and extracts their metadata and 3D arrays. A 3D mesh centered on the reference point is created based on the coordinates of the reference point in the CSV table. Then, the mesh point coordinates are mapped to the image's coordinate system using the CT image metadata. Spline interpolation is used to obtain the value of the 3D array corresponding to each mesh point in the CT image, thereby acquiring local CT images near the reference point and providing input samples for model training.

[0082] (1) Obtaining the metadata and three-dimensional array of CT images

[0083] The SimpleITK medical image processing library is used to read the .mhd and .raw files that store CT image information, and extract the metadata img_meta (coordinate origin img_origin, voxel size img_spacing, and direction matrix img_direction) and the three-dimensional array img_arr.

[0084] (2) Create a grid centered on the reference point.

[0085] The program uses numpy.meshgrid to generate a 19×19×19 three-dimensional grid centered at the origin, and then moves the grid center to the reference point.

[0086] (3) Grid coordinate transformation and interpolation

[0087] The program performs coordinate transformation on all grid points in the CT image based on the origin, spacing, and direction matrix to obtain the coordinates of the grid points in the coordinate system of the CT image. After transformation, `map_coordinates` is used to interpolate the 3D array of the CT image based on the grid coordinates to obtain local image patches near the reference point. Finally, the Nibabel medical image processing library is used to store the local images in Nifti format.

[0088] The dataset class is created as follows:

[0089] After the first two steps of preprocessing the tabular file and images, the original dataset is transformed into a CSV table and a set of local CT images in Nifty format. To facilitate the use of the dataset in the model training program, a dataset class inheriting from torch.utils.data.Dataset needs to be created in the final stage of data preprocessing.

[0090] The dataset object is automatically partitioned upon creation. In the constructor, the program accesses a CSV table to obtain reference point information, divides the reference point information into 10 equal parts, and retains 9 parts for the training set and 1 part for the test set. The parameter `is_valid` determines whether the current dataset is a training set or a validation set, and the parameter `valid_set_ndx` determines which of the 10 reference point information parts is retained in the validation set.

[0091] Since the three models solve different problems, their labels are also different. Therefore, the DSet class only has an empty get_tag() interface. The specific label acquisition methods are implemented by the three dataset subclasses RDSet, DDSet and ODset.

[0092] Furthermore, the method for building a neural network is as follows:

[0093] In this embodiment, the structures of models r_net, d_net, and o_net are basically the same, such as... Figure 4 As shown, all models have a 7-layer structure. The first 6 layers consist of 6 convolutional blocks (Conv_block), and the 7th layer consists of a single 3D convolutional layer.

[0094] Of the three models, d_net outputs the probability of each directional class, so the number of output channels for d_net is equal to the number of directional classes; r_net outputs the blood vessel radius value, and has 1 output channel; o_net is a binary classification model, which outputs a single value to represent the probability that the input sample is located on the center line, so it also has 1 output channel.

[0095] The convolutional block is set as follows:

[0096] The convolutional neural network used in the example consists of multiple similar convolutional blocks (conv_block). Each convolutional block includes a 3D convolution operation (nn.Conv3d()), a batch normalization operation (BatchNorm3d()), and an activation operation (nn.ReLU()). The construction parameters of the convolutional block are the number of input channels (chann_in), the number of output channels (chann_out), the kernel size (k_size), and the degree of dilation (the default is 1, indicating no dilation).

[0097] nn.Conv3d() takes five-dimensional data (batch_size, channel, Depth, Height, Width), uses a specified convolution kernel to move along the three dimensions of Depth, Height, and Width of the input image, and calculates the convolution once for each step, that is, multiplies the convolution kernel with the corresponding position in the image, and the product results are accumulated to obtain the output image.

[0098] Batch-Normalization is widely used in convolutional neural network models that use mini-batches. nn.BatchNorm3d() calculates the mean and variance parameters of each mini-batch and normalizes all batches to a standard normal distribution with a mean of 0 and a variance of 1.

[0099] Activation functions are a class of nonlinear functions that help models effectively retain trained features, avoiding the situation where each layer of a neural network performs only linear operations, becoming a primitive perceptron. ReLU stands for Rectified Linear Unit, and its mathematical expression is shown in equation (1). It is a simple and commonly used activation function, and its graph is shown in the figure. Figure 5 .

[0100] ReLU(x) = max(0,x) (1)

[0101] The model is built as follows:

[0102] The convolutional neural network model Net consists of seven convolutional layers. Input data passes through each of these seven layers sequentially before outputting. The first six layers each have six convolutional blocks, while the seventh layer contains only one convolutional operation. The number of input channels, the number of output channels, the kernel size, and the dilation parameter vary from layer one to layer seven. The number of input and output channels increases, while the kernel size decreases. Dilated convolution is only used in the third and fourth layers. The number of output channels in the final layer is determined during object construction because different model objects perform different tasks, resulting in a different final number of output channels. The parameters of the convolutional neural network are shown in Table 2.

[0103] Table 2

[0104]

[0105]

[0106] Furthermore, the model training method is as follows:

[0107] Training the convolutional neural network model is the core part of this embodiment. During this process, the program uses DataLoader to read data from the dataset and input it into the model. The model is trained and validated in multiple iterations, and the loss and other training metrics are calculated. The learning rate parameter is dynamically adjusted during the iteration to improve the training effect as much as possible.

[0108] The training data is read as follows:

[0109] After data preprocessing, the data in the CAT08 dataset is processed and re-stored in files, and can be easily accessed by the program through the DSet class and its subclasses. The program uses the torch.utils.data.DataLoader tool in PyTorch, which is responsible for processing model input, to read data from the dataset object in batches and input it into the model. The DataLoader parameter batch_size represents the batch size, shuffle=True indicates that the input data is shuffled, and num_workers represents the number of threads processing the data input.

[0110] The learning rate initialization method is as follows:

[0111] In this embodiment, AdamW is selected as the model optimizer, and MultiStepLR is selected as the scheduler. The optimizer is responsible for automatically updating the model parameters based on the loss and learning rate during training, and adding an L2 regularization term to the loss function; while the scheduler is responsible for dynamically adjusting the learning rate during training. The adjustment strategy of MultiStepLR is to multiply the learning rate by a specific parameter gamma based on the milestones array when the number of iterations reaches a specified number.

[0112] For example, Figure 6 Set the learning rate update schedule for r_net and d_net, where milestones = [50, 65, 80] and gamma = 0.3.

[0113] The model iteration method is as follows:

[0114] The program undergoes several iterations, each of which requires training and validating the model, outputting logs, and updating the learning rate.

[0115] The training process is as follows:

[0116] During training, the program transfers samples and labels to the GPU input model and obtains the output. The model loss and other training metrics are calculated from the model output. Then, the gradient is zeroed out, the gradient value is calculated through backpropagation of the model loss, and the model parameters are updated.

[0117] This embodiment trains three CNN models: a radius regression model (r_net), a direction multi-classification model (d_net), and a bias correction binary classification model (o_net). During training, each of the three models is trained separately, and different model metrics are output.

[0118] In addition to the multi-class cross-entropy loss, accuracy is another training metric for directional multi-class classification models. The program selects the two largest values ​​from the model's output and compares the corresponding directional class indices `output_top2_ndx` with the indices `tag_top2_ndx` of the two largest directional classes (with a value of 0.5) in the labels. If they are identical, the number of positive samples `correct_n` is incremented by 1. After training with all samples, the accuracy value is obtained by calculating the ratio of `correct_n` to the total number of samples `all_n`.

[0119] In addition to the binary cross-entropy loss, the training metrics for the bias correction binary classification model include accuracy, precision, recall, and F1 score. The program first uses the Sigmoid function to convert the model output into probability, then calculates the four basic metrics TP, FP, FN, and TN, and calculates the accuracy, precision, recall, and F1 score based on the four basic metrics. The Sigmoid function can limit the model output from (-∞,+∞) to the range of (0,1)

[20] . The calculation formula of the function is shown in Equation (2), and the graph of the function is shown in Equation (2). Figure 7 .

[0120]

[0121] The verification process is as follows:

[0122] The validation process is similar to the training process, requiring data to be transferred to the GPU and input into the model to obtain output, from which loss and other metrics are calculated. However, the validation process is performed on a validation set and does not require gradient calculation or optimization of training metrics.

[0123] The log output method is as follows:

[0124] In each iteration, logs are output, sending the loss and other metrics calculated during the training and validation process of the current iteration to the screen, a CSV file, and TensorBoard. The function of sending logs to TensorBoard is particularly important because it transforms the model's training metrics into images, allowing trainers to observe in real time how various metrics change with each iteration.

[0125] During this process, the program uses SummaryWriter to write evaluation metrics to files in the / tf-logs / directory. TensorBoard can automatically read these metrics and display them as images in real time. TensorBoard displays the training loss and validation loss in the same image.

[0126] The model testing method is as follows:

[0127] The model testing and results display section uses three trained CNN models to build a tracker that extracts the coronary artery centerline from user-uploaded coronary CT images based on a tracking algorithm, and uses visualization tools to display the extracted centerline.

[0128] The general idea behind the tracking algorithm is as follows: The tracker starts from a known initial reference point and iteratively moves a certain distance in a certain direction. The direction and distance of each movement are predicted by three trained CNN models. Upon reaching the end of the blood vessel (or the coronary ostium), the tracker immediately returns to the initial reference point and begins tracking in another direction. The final path traveled by the tracker can be considered as the centerline extracted from the coronary artery.

[0129] The tracker consists of three models: r_net, d_net, and o_net. During tracking, the program extracts local CT images near the tracker and inputs them into the three models. The radius regression model r_net predicts the radius of the blood vessel at that point based on the image, providing the tracker with tracking step length information. The direction multi-classification model d_net lists the probability that each vector on the Fibonacci sphere is equal to the direction vector of the current position's centerline, providing a reference for the tracker's direction of movement. The deviation correction binary classification model o_net analyzes whether the current point is on the centerline based on the image, determining whether the tracker has deviated from the centerline.

[0130] (1) Trace Initialization

[0131] Before tracking initialization, the only known information is the coordinates of the user-input CT image and the initial reference point. For example... Figure 8 As shown, during the initialization process, the program first acquires local CT images near the initial reference point x0, and inputs them into the radius regression model r_net and the orientation multi-classification model d_net, respectively, to obtain the output r0 of r_net and the outputs d1 and d2 of d_net. Subsequently, the two trackers start from x0 and move along directions d1 and d2 respectively, reaching points x1 and x2, and recording the coordinates of x, x1, and x2. After the initialization operation is completed, the number of initial points with known coordinates increases from one to three.

[0132] (2) Iterative tracing

[0133] During the iterative tracing process, two trackers start from x1 and x2 respectively and trace in opposite directions d1 and d2. One tracker moves towards the origin of the coronary artery, i.e., the coronary ostium, while the other tracker moves towards the distal end of the vessel. After the iterative tracing process is completed, each tracker should extract half of the coronary artery centerline.

[0134] like Figure 9 The tracker starts from point x1, inputs the local CT image near x1 into r_net, and obtains the output r1 of r_net. Similarly, the local CT image at x1 is input into d_net, but instead of obtaining only one forward direction, it obtains the probability of all direction categories and sorts all possible forward directions in descending order of probability.

[0135] like Figure 10 After sorting the directions, the program then filters all forward directions d. First, it excludes directions that form an angle greater than 60 degrees with direction d1, resulting in a list of remaining directions, vector_list.

[0136] like Figure 11 The tracker starts from x1 and moves a distance r1 along the direction d with the highest probability in vector_list to a new point x*. It then obtains the local image near the point and inputs it into the correction model o_net, which determines whether the new point is located on the center line of the blood vessel (whether the output of o_net is greater than 0.5).

[0137] If o_net determines that x* is not on the center line, it corrects the direction of travel by discarding point x* and direction d, causing the tracker to backtrack to x1 and move along the direction with the second highest probability in vector_list. If the direction with the second highest probability is also discarded, it moves along the direction with the third highest probability, and so on.

[0138] The reason why the direction with the highest probability in d_net is not directly taken as the direction of movement is that the training effect of d_net in this embodiment is not good enough. Relying solely on d_net to determine the direction of movement of the tracker will cause the tracker to easily deviate from the reference center line, leave the coronary artery and enter other nearby human tissues.

[0139] (3) Tracking terminated

[0140] When the tracker reaches the distal end of a blood vessel or the ostium of a coronary artery, its tracking behavior must be terminated. Therefore, a criterion is needed to determine whether tracking should be terminated. The termination criterion is determined based on termination entropy. The tracker needs to calculate the termination entropy for each step it takes and calculate the average termination entropy of the most recent three steps. If this value exceeds a predetermined threshold, tracking is considered terminated.

[0141] The formula for calculating the termination entropy is shown in equation (3), where D represents the number of directional classes and p represents the probability of each directional class in the output of d_net.

[0142]

[0143] The test metrics are implemented as follows:

[0144] After the tracker terminates, further processing is required on the centerline generated by the tracker. First, the centerline is interpolated to reduce the distance between adjacent points to approximately 0.03 mm. Then, a heap-optimized Dijkstra's minimum path algorithm is used to establish a point-to-point connection between the centerline generated by the tracker and the reference centerline. Based on this connection, four test metrics—OV, OF, OT, and AI—are calculated.

[0145] (1) Interpolation

[0146] The program uses the numpy.linspace() function to perform linear interpolation on every two adjacent points on the generated centerline to ensure that the distance between adjacent points on the generated centerline after interpolation is about 0.03mm, which is the same as the distance between adjacent points on the reference centerline.

[0147] (2) Establish connections between points

[0148] Let the center line generated by the tracker be denoted as infer, and the reference center line as referer. The connection relationship between points created by the program satisfies the following conditions: all points on infer can only be connected to points on referer, and points on referer can only be connected to points on infer; the first points on the two center lines are connected to each other, and the last points are also connected to each other; and if there is a connection between infer[a] and infer[b], then infer[a] is connected to infer[b+1], or infer[a+1] is connected to infer[b].

[0149] The above connection method ensures that each point on the `infer` graph is connected to at least one point on the `refer` graph, and vice versa. The program employs an optimized Dijkstra's graph search algorithm to establish these connections, minimizing the distance between any two points in each connection.

[0150] (3) Statistical test indicators

[0151] After establishing the point connections between the two centerlines, this embodiment uses these connections to statistically analyze test metrics. While traversing all connections, the program determines whether the point on the referrer or the point on the infer has changed between the current connection and the previous connection. Based on this determination, the program modifies the values ​​of metrics such as TPM, TPR, FN, and FP, and after completing the traversal, it statistically analyzes OV, OF, OT, and AI based on these metrics.

[0152] This invention also includes a platform front-end design, the details of which are as follows:

[0153] This embodiment uses Flask to maintain an online test page for a model. Flask is a micro web framework that can be used to quickly build websites and web services. The test page allows users to upload their own CT image files and reference centerline files. The page calls backend code to extract the coronary artery centerlines from the CT images and uses visualization tools to display the extracted centerlines and the reference centerlines in a 3D scatter plot to the user, allowing the user to compare the two centerlines and determine the effectiveness of the centerline extraction test.

[0154] The file upload method is as follows:

[0155] The file upload functionality is implemented using the Flask-Dropzone extension. Flask-Dropzone allows users to upload CT image files and centerline reference files to the server via drag-and-drop operations.

[0156] The model testing method is as follows:

[0157] In this section, the JavaScript program uses AJAX to call the backend's predict() and progress() functions to extract the center line and obtain the extraction progress, respectively.

[0158] The method for visualizing the center line is as follows:

[0159] This section adds an ECharts 3D scatter plot (Scatter3D) to the page using the pyecharts library. The plot simultaneously displays the reference center line and the center lines extracted by the tracker (predict_coords). It also uses a toolbox to add image saving and data view tools, allowing users to obtain the center line image and the coordinates of each point on the center line.

[0160] This embodiment also included experiments, and the results were analyzed as follows:

[0161] In this embodiment, three different sets of experimental data were generated during the preprocessing stage, based on different values ​​for the number of orientation classes (100, 200, and 500). During the model training stage, these three sets of data were sequentially input into the neural network to train three sets of model parameters. During the model testing stage, the embodiment allows switching the model parameters used in the testing process by selecting different options in the "Number of Orientation Classes" section of the testing interface, thus enabling separate testing of the three sets of model parameters.

[0162] Preprocessing stage: In the preprocessing stage, the example generates experimental data online100, online200, and online500 for the radius regression model and the orientation multi-classification model, and generates experimental data onoff100, onoff200, and onoff500 for the bias correction binary classification model.

[0163] The model training phase is as follows:

[0164] In this embodiment, the CNN model is trained using multiple sets of experimental data provided during the preprocessing stage. During training, the program calculates the loss and other metrics for each model on the training and validation sets. Experiments are repeated with adjusted conditions to optimize the model multiple times. Table 3 shows the statistical values ​​of various model metrics for the three models when the number of directional classes is 100, 200, and 500, respectively.

[0165] Table 3

[0166]

[0167] The training metrics of r_net and o_net are both excellent. Regardless of the value of the number of classes in the direction, the loss of r_net and o_net is very small. Other metrics of o_net, such as accuracy, precision, recall and F1 score are also very high (the original data is extremely close to 1).

[0168] However, the loss of the multi-classification model is still somewhat high. Although the accuracy has reached a maximum of 0.977, it still falls short of the ideal value. This often leads to tracking direction prediction errors when the implementation initially uses only r_net and d_net for tracking experiments. As a result, the tracker deviates from the reference centerline shortly after starting from the initial reference point. This is the direct reason for introducing o_net in this implementation.

[0169] The testing phase is as follows:

[0170] In this embodiment, during the testing phase, the number of orientation classes is set to 100, 200, and 500 respectively. A point is selected from the reference file of the centerline of the four blood vessels in the subset dataset 00 as a known initial point. The centerlines of the four blood vessels are extracted from the CT image. The values ​​of evaluation indicators OV, OF, OT, and AI are calculated based on the extracted centerlines and the reference centerlines, thereby comparing the impact of the difference in the number of orientation classes on the centerline extraction effect.

[0171] Of the four evaluation metrics, higher values ​​for OV, OF, and OT indicate a higher degree of overlap between the two center lines, meaning better center line extraction. Conversely, lower values ​​for AI indicate a smaller average distance deviation between the two center lines, meaning better center line extraction.

[0172] As shown in Table 4, overall, when the number of direction classes is 500, the values ​​of the OV, OF, and OT indices are slightly higher than when the number of direction classes is 100 or 200, while the value of the AI ​​index is slightly lower than when the number of direction classes is 100 or 200. Only in vessel0, when the number of direction classes is 500, are the OV and OT indices slightly lower. Therefore, it can be concluded that the centerline extraction effect is better when the number of direction classes is 500.

[0173] Table 4

[0174]

[0175] Meanwhile, the embodiment tested centerline extraction on all subsets of data, assuming a directional class count of 500. OV, OF, OT, and AI metrics were calculated, and their average values ​​were statistically analyzed. As shown in Table 5, the average values ​​of OV, OF, and OT metrics reached over 90%, while the AI ​​metric was controlled below 0.35mm.

[0176] Table 5

[0177]

[0178]

[0179] In addition, the embodiment also conducted a centerline extraction test on the subset dataset dataset03 using only r_net and d_net, and calculated four indicators: OV, OF, OT and AI, to evaluate the impact of the introduction of the correction model o_net on the centerline extraction effect.

[0180] As shown in Table 6, the OV, OF, and OT indices all increased after the introduction of o_net, while the AI ​​index decreased. Specifically, OV increased by an average of 25.73%, OF by an average of 51.82%, OT by an average of 25.24%, and AI decreased by an average of 11.82%, indicating that the introduction of o_net significantly improved the centerline extraction effect.

[0181] Table 6

[0182]

[0183] The data shows that the average values ​​of the centerline overlap rate measurement indicators, namely, overlap (OV), overlap before the first error (OF), and clinically relevant vascular overlap (OT), can reach over 90%, while the average internal mean (AI) is successfully controlled below 0.35 mm, which can meet clinical needs.

[0184] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for extracting the centerline of coronary arteries, characterized in that, Includes the following steps: Load the centerline reference file and CT image file from the CAT08 dataset, and perform preprocessing to obtain the preprocessed dataset class; The preprocessing method includes: The centerline reference file is preprocessed and converted into a preprocessed file for training and testing convolutional neural network models; Read CT images, obtain the metadata and three-dimensional array of the CT images, and obtain local CT images near the reference point based on spline interpolation. A dataset class inheriting from torch.utils.data.Dataset is created using the preprocessed file and local CT images near the reference point; The method for preprocessing the centerline reference file includes: The three-dimensional coordinates and vessel radius information of each reference point on the coronary artery centerline are extracted from the centerline reference file. The centerline direction at each reference point is determined according to the spatial relationship between the reference points. The centerline direction vector is converted into a point index on the Fibonacci sphere. New reference points outside the centerline are generated as negative samples based on the existing reference points. All reference point information is integrated and stored as a CSV file. Methods for preprocessing CT images include: The process involves acquiring metadata and a 3D array from a CT image, generating a 19×19×19 3D grid centered at the origin, and moving the grid center to a reference point. Based on the metadata of the CT image, coordinate transformations are performed on all grid points to obtain their coordinates within the coordinate system of the CT image. The transformed grid coordinates are then used to interpolate the 3D array of the CT image to obtain a local image near the reference point, which is then stored in Nifti format. Construct a convolutional neural network model, which includes a radius regression model, an orientation multi-classification model, and a bias correction binary classification model; The first six layers of the convolutional neural network model consist of six convolutional blocks, and the seventh layer is a single three-dimensional convolutional layer. Each convolutional block includes a three-dimensional convolution operation, a batch normalization operation, and an activation operation. The construction parameters of the convolutional block are the number of input channels, the number of output channels, the kernel size, and the degree of dilation. The number of input and output channels increases, the kernel size decreases, and dilated convolution is only used in the third and fourth layers. Data is read from the preprocessed dataset, and the read data is input into the convolutional neural network model for training and validation to obtain a trained model; A tracker is built based on the trained model. Starting from the initial reference point, it iteratively moves the distance in the predicted direction to predict the radius of the blood vessel and extracts the center line of the coronary artery. The tracking method of the tracker includes: The radius regression model predicts the radius of the blood vessel based on the image, providing the tracker with tracking step length information; the direction multi-classification model lists the probability that each vector on the Fibonacci sphere is equal to the direction vector of the current position centerline, providing a reference for the tracker's forward direction; the deviation correction binary classification model analyzes whether the current point is on the centerline based on the image, determines whether the tracker deviates from the centerline, and terminates the tracker's tracking behavior when the tracker reaches the end of the blood vessel or the coronary ostium. The method for determining whether to terminate the tracker is based on the termination entropy includes: calculating the average termination entropy of the last 3 steps, and terminating the tracking when the average value exceeds a predetermined threshold. The expression for calculating the termination entropy is: In the formula, D represents the number of direction classes, p i This represents the probability of each directional class in the output of the directional multi-class classification model. The tracker employs an iterative tracking method, specifically including: during the iterative tracking process, two trackers start from x1 and x2 respectively and track in opposite directions d1 and d2. One tracker moves towards the origin of the coronary artery, i.e., the coronary ostium, while the other tracker moves towards the distal end of the vessel. After the iterative tracking process is completed, each tracker should extract half of the coronary artery centerline. Starting from point x1, the tracker inputs the local CT image near x1 into r_net to obtain the output r1 of r_net. Similarly, the local CT image at x1 is input into d_net to obtain the probability of all directional categories. All possible directions are then sorted from highest to lowest probability. After sorting the directional categories, all directions are filtered. First... Directions with an angle greater than 60 degrees to direction d1 are excluded, resulting in a list of remaining directions, vector_list. The tracker starts from x1 and moves a distance r1 along the direction d with the highest probability in vector_list, reaching a new point x*. The local image near this point is obtained and input into the correction model o_net, which determines whether the new point is located on the center line of the blood vessel. If o_net determines that x* is not on the center line, it corrects the direction of travel, i.e., discards point x* and direction d, and causes the tracker to retreat to x1 and move along the direction with the second highest probability in vector_list. If the direction with the second highest probability is also discarded, it moves along the direction with the third highest probability, and so on, until the correct direction is found.

2. The method for extracting the coronary artery centerline according to claim 1, characterized in that, The method for training the convolutional neural network model includes: AdamW was selected as the optimizer for the model, and MultiStepLR was selected as the scheduler. The optimizer is used to automatically update the model parameters based on the loss and learning rate during training, and to add an L2 regularization term to the loss function. The scheduler is used to dynamically adjust the learning rate during training.

3. The method for extracting the coronary artery centerline according to claim 1, characterized in that, After extracting the coronary artery centerline, the accuracy of the extracted coronary artery centerline is determined by the overlap, the overlap before the first error, the overlap of clinically relevant blood vessels, and the average distance.

Images