Coronary artery center line extraction method

Abstract

The invention discloses a coronary artery center line extraction method, which comprises the following steps of: loading a center line reference file and a CT (Computed Tomography) image file from a CAT08 data set, and preprocessing to obtain a preprocessed data set class; constructing a convolutional neural network model, wherein the convolutional neural network model comprises a radius regression model, a direction multi-classification model and a correction dichotomy model; reading data from the preprocessed data set class, and inputting the read data into the convolutional neural network model for training and verification to obtain a trained model; and constructing a tracker based on the trained model, iteratively advancing to the prediction direction from the initial reference point to predict the distance of the radius of the blood vessel, and extracting the center line of the coronary artery. According to the method, the constructed tracker has extremely high center line extraction capability.

Description

Technical Field

[0001] The invention belongs to the technical field of coronary artery centerline extraction, and in particular relates to a coronary artery centerline extraction method. Background Art

[0002] Many scholars have conducted research on the problem of coronary artery centerline extraction and proposed various methods, including traditional segmentation methods and deep learning-based methods.

[0003] Traditional segmentation methods can be summarized as methods based on centerline tracking, topology refinement, minimum cost path, and distance transformation.

[0004] The prior art is a centerline extraction method based on the tubular tissue inscribed ball model (MIBTT). This method performs two distance transformations on the original image to complete the search for the image boundary and delete unnecessary voxels, and constructs a maximum inscribed ball model to find the skeleton that reflects the shape of the original image. Finally, the skeleton line is processed using a principle similar to the dichotomy method to obtain the centerline. However, this method involves two distance transformations and the construction of a maximum inscribed ball model, resulting in a high computational complexity of the algorithm, and the algorithm is more sensitive to noise, which will cause the skeleton line to deviate from the true centerline.

[0005] There is another centerline extraction method based on twelve-direction topological refinement (TDTT) in the prior art. This method introduces the concept of three-dimensional space criticality, and performs refinement operations on the three-dimensionally reconstructed blood vessels in parallel along the twelve sides of the cube according to the three-dimensional morphological principles, that is, pixels that do not contain target points are deleted on the basis of retaining the topological structure of the coronary artery as much as possible. The algorithm also uses the Dijkstra algorithm to prune small branches in the blood vessels, and this pruning operation has a positive effect on improving the accuracy of topological refinement. However, this method takes less global information into consideration. This may lead to the inability to accurately judge the overall direction and topological structure of the centerline during the extraction process, thereby affecting the accuracy of the final result.

[0006] For the centerline extraction method based on deep learning, there is a discriminative coronary artery tracking method in the prior art. The neural network model designed by the institute includes a backbone network composed of multiple convolutional layers, a tracker, and a discriminator. The discriminator model designed by the study can exclude other tubular tissues near the coronary vessels while the tracker extracts the centerline of the blood vessel, preventing the tracker from being trapped in it, and can terminate the tracking process in time when the tracker reaches the end of the blood vessel. The model contains multiple convolutional layers, trackers, and discriminators. The overall structure is relatively complex and has high requirements for computing resources.

[0007] In summary, all kinds of blood vessel centerline extraction methods have certain limitations. For example, the centerline tracking method sets the blood vessel as an ideal model to obtain a good tracking effect; the centerline extraction method based on the minimum cost path is prone to problems such as shortcuts, deviation from the center, and premature termination of the search; the extraction method based on topological refinement requires a large amount of calculation and a long calculation time; the extraction method based on distance transformation cannot guarantee the single pixel, continuity and topology of the central pixel; the deep learning method requires a large number of data sets for training, and it is difficult to establish a standard data set. In addition, when facing medical images of different sources and different qualities, its generalization ability may be affected. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes a method for extracting the centerline of a coronary artery to solve the above problems existing in the prior art.

[0009] To achieve the above object, the present invention provides a method for extracting the centerline of a coronary artery, comprising the following steps:

[0010] Load the centerline reference file and CT image file from the CAT08 dataset and perform preprocessing to obtain the preprocessed dataset class;

[0011] Constructing a convolutional neural network model, wherein the convolutional neural network model includes a radius regression model, a direction multi-classification model, and a correction binary classification model;

[0012] Reading data from the preprocessed data set class, inputting the read data into the convolutional neural network model for training and verification, and obtaining a trained model;

[0013] A tracker is constructed based on the trained model. Starting from the initial reference point, it iteratively moves in the predicted direction by the distance of the predicted vessel radius to extract the centerline of the coronary artery.

[0014] Preferably, the method for pretreatment comprises:

[0015] Preprocessing the centerline reference file to convert it into a preprocessed file for convolutional neural network model training and testing;

[0016] Read the CT image, obtain the meta-information and three-dimensional array of the CT image, and obtain the local CT image near the reference point based on the spline interpolation method;

[0017] Create a dataset class that inherits the torch.utils.data.Dataset class through the preprocessed file and the local CT image near the reference point.

[0018] Preferably, the method for preprocessing the centerline reference file comprises:

[0019] The three-dimensional coordinates and vessel radius information of each reference point on the coronary artery centerline are extracted from the centerline reference file, the centerline direction at each reference point is determined according to the spatial position relationship between the reference points, the centerline direction vector is converted into a point index on the Fibonacci sphere, new reference points outside the centerline are generated as negative samples according to the existing reference points, and all reference point information is integrated and stored as a CSV file.

[0020] Preferably, the method for preprocessing the CT image includes:

[0021] The meta-information and three-dimensional array of the CT image are obtained, and a three-dimensional grid of size 19×19×19 centered at the origin is generated, and the center of the grid is moved to the reference point; the coordinate transformation operation is performed on all the grid points on the grid according to the meta-information of the CT image to obtain the coordinates of the grid points in the coordinate system of the CT image; the three-dimensional array of the CT image is interpolated according to the transformed grid coordinates to obtain the local image near the reference point, and the local image is stored in Nifti format.

[0022] Preferably, the first 6 layers of the convolutional neural network model are 6 convolution blocks, and the 7th layer is a separate three-dimensional convolution layer. The convolution block includes a three-dimensional convolution operation, a batch normalization operation and an activation operation. The construction parameters of the convolution block are the number of input channels, the number of output channels, the size of the convolution kernel and the degree of voids, respectively. The number of input and output channels shows an increasing trend, the size of the convolution kernel shows a decreasing trend, and void convolution is only used for the third and fourth layers.

[0023] Preferably, the method for training the convolutional neural network model includes:

[0024] AdamW is selected as the optimizer of the model, and MultiStepLR is selected as the scheduler; the optimizer is used to automatically update the model parameters according to the loss and learning rate during training, and add an L2 regularization term to the loss function; the scheduler is used to dynamically adjust the learning rate during training.

[0025] Preferably, the tracking method of the tracker includes:

[0026] The radius regression model predicts the blood vessel radius value based on the image and provides tracking step information for the tracker. The direction multi-classification model lists the probability that each vector on the Fibonacci sphere is equal to the centerline direction vector of the current position, providing a reference for the tracker's forward direction. The deviation correction binary classification model determines whether the tracker deviates from the centerline based on image analysis to determine whether the current point is on the centerline. When the tracker reaches the end of the blood vessel or the coronary artery orifice, the tracking behavior of the tracker is terminated.

[0027] Preferably, whether to terminate the tracker is determined by the termination entropy, and the determination method includes: calculating the average of the termination entropies of the most recent three steps, and terminating the tracking when the average exceeds a predetermined threshold;

[0028] The calculation expression of the termination entropy is:

[0029]

[0030] In the formula, D represents the number of direction classes, p i Represents the probability corresponding to each direction class in the output of the direction multi-classification model.

[0031] Preferably, after extracting the coronary artery centerline, whether the extracted coronary artery centerline is accurate is judged by using the coincidence degree, coincidence degree before the first error, coincidence degree of clinically relevant vascular parts and average distance indicators.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] The present invention discloses a method for extracting the centerline of a coronary artery, comprising the following steps: loading a centerline reference file and a CT image file from a CAT08 data set, and performing preprocessing to obtain a preprocessed data set class; constructing a convolutional neural network model, the convolutional neural network model comprising a radius regression model, a direction multi-classification model, and a correction binary classification model; reading data from the preprocessed data set class, inputting the read data into the convolutional neural network model for training and verification, and obtaining a trained model; constructing a tracker based on the trained model, starting from an initial reference point, iteratively advancing in a predicted direction by a distance of a predicted blood vessel radius, and extracting the centerline of the coronary artery. The present invention constructs a tracker through a radius regression model, a direction multi-classification model, and a correction binary classification model, so that the tracker has a very strong centerline extraction capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0035] Figure 1 is a flow chart of the extraction method according to an embodiment of the present invention;

[0036] Figure 2 A flowchart of centerline reference file processing according to an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of reference points outside the generating center line according to an embodiment of the present invention;

[0038] Figure 4 This is a structural diagram of a convolutional neural network model according to an embodiment of the present invention;

[0039] Figure 5 is a function diagram of the ReLU activation function of an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of a learning rate update plan for r_net and d_net according to an embodiment of the present invention;

[0041] Figure 7 It is a Sigmoid function diagram of an embodiment of the present invention;

[0042] Figure 8 This is a schematic diagram of tracking initialization according to an embodiment of the present invention;

[0043] Fig. 9 This is a schematic diagram of using d_net to obtain all direction class probabilities in x1 according to an embodiment of the present invention;

[0044] Fig.10 This is a schematic diagram of selecting the forward direction according to the angle with d1 according to an embodiment of the present invention;

[0045] Fig.11 Schematic diagram of using o_net to prevent a tracker from deviating from a center line according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Embodiment 1

[0049] like Figure 1 As shown, this embodiment provides a method for extracting the centerline of a coronary artery, comprising the following steps:

[0050] Load the centerline reference file and CT image file from the CAT08 dataset and perform preprocessing to obtain the preprocessed dataset class;

[0051] Construct a convolutional neural network model, which includes a radius regression model, a direction multi-classification model, and a correction binary classification model;

[0052] Read data from the preprocessing dataset class, input the read data into the convolutional neural network model for training and verification, and obtain a trained model;

[0053] A tracker is constructed based on the trained model. Starting from the initial reference point, it iteratively moves in the predicted direction by the distance of the predicted vessel radius to extract the centerline of the coronary artery.

[0054] Further, the method for preprocessing includes:

[0055] Preprocessing the centerline reference file to convert it into a preprocessed file for convolutional neural network model training and testing;

[0056] Read the CT image, obtain the meta-information and three-dimensional array of the CT image, and obtain the local CT image near the reference point based on the spline interpolation method;

[0057] Create a dataset class that inherits the torch.utils.data.Dataset class through the preprocessed file and the local CT image near the reference point.

[0058] The specific data preprocessing method is as follows:

[0059] After acquiring the dataset, the centerline reference file and CT image file are loaded from the CAT08 dataset, and the data are screened, extracted, processed, and integrated, and finally converted into a dataset class that can be directly called by PyTorch.

[0060] In this embodiment, a total of three CNN models need to be trained. As shown in Table 1, the input data required by the three models are different. The preprocessing process needs to provide the samples and label information required by each model.

[0061] Table 1

[0062]

[0063] Centerline reference files are processed as follows:

[0064] The centerline reference file processing process receives all the centerline reference files reference.txt (32 in total) in the CAT08 data set, extracts the text data and performs a series of processing, and finally integrates the data into a new table and saves it in a CSV file.

[0065] The process first extracts the reference point related information from the centerline reference file, and screens out the three-dimensional coordinate information of the reference point and the vascular radius information at the reference point. Then, all reference points are traversed, and the centerline direction at each reference point is determined according to the spatial position relationship between the reference points.

[0066] Since the centerline direction at the reference point is highly random, it is almost impossible to find a reference point with exactly the same reference direction. Directly using the centerline direction vector as the direction classification label of the d_net model will result in too many categories, too few samples in each category, low model training accuracy, and poor training effect. Therefore, the direction vector must be converted into the index value of the point on the Fibonacci ball to limit the number of direction classes.

[0067] In addition, since the correction model o_net requires the preprocessing process to provide reference points that are not on the center line as negative samples, a set of new reference points that are not on the center line must be generated based on the existing reference points. Finally, the coordinates, vascular radius, and direction index of all reference points are stored in a new sample table file.

[0068] Specific implementation as Figure 2 As shown:

[0069] (1) Extracting reference point information from text files

[0070] The program uses numpy.loadtxt() to read the reference file, stores the 3D coordinates and vessel radius of each reference point into a special tuple point_tup, and then summarizes them into the list vessel to facilitate subsequent operations to call the reference point information.

[0071] (2) Determine the centerline direction at the reference point

[0072] For each reference point on the centerline, obtain the pre and next points so that the distance between the two points and the reference point is approximately equal to the reference value of the vessel radius of the reference point. The vector from the reference point to the pre and next points can represent the direction of the centerline at the reference point.

[0073] (3) Convert the direction vector into a direction index

[0074] Use the Fibonacci grid sampling method to obtain a Fibonacci spherical shell with a sampling number of point_num. For each centerline direction vector v obtained, find the spherical shell vector vi with the smallest angle on the sphere and obtain the index of vi on the shell.

[0075] (4) Generate new reference points outside the center line

[0076] The program receives the information of a reference point on the center line, obtains the coordinates of the two directional vectors pre_coord and next_coord corresponding to the reference point, calculates the outer product of the two vectors to obtain a new vector vect that is orthogonal to the two directional vectors at the same time, and adjusts the modulus value of vect to the blood vessel radius value at the reference point, and adjusts the starting point of vect to the original reference point. At this time, the coordinates of vect are the coordinates of the reference point outside the center line.

[0077] The positional relationship between the original reference point, the two direction vectors and the new reference point is as follows: Figure 3 shown.

[0078] (5) Store the reference information in a new file

[0079] The program extracts and processes the reference points from different files and converts them into a special tuple sample_tup. It also adds several keyword information to it to identify which sub-dataset the reference point comes from, which blood vessel it comes from, and whether it is a point on the centerline.

[0080] The CT image preprocessing is as follows:

[0081] This process reads the CT image and obtains the meta-information and three-dimensional array. A three-dimensional grid centered on the reference point is created according to the coordinates of the reference point in the CSV table, and then the grid point coordinates are mapped to the image coordinate system according to the CT image meta-information. The value of the CT image three-dimensional array corresponding to each grid point is obtained through spline interpolation, so as to obtain the local CT image near the reference point and provide input samples for model training.

[0082] (1) Obtaining meta-information and three-dimensional array of CT images

[0083] Use the medical image processing library SimpleITK to read the .mhd file and .raw file storing CT image information, and extract the meta information img_meta (coordinate origin img_origin, voxel size img_spacing and direction matrix img_direction) and the three-dimensional array img_arr contained therein.

[0084] (2) Create a grid centered on the reference point

[0085] The program uses numpy.meshgrid to generate a 3D grid of size 19×19×19 centered at the origin, and then moves the center of the grid to the reference point.

[0086] (3) Grid coordinate transformation and interpolation

[0087] The program performs coordinate transformation operations on all grid points on the grid according to the origin coordinates (origin), voxel size (spacing) and direction matrix (direction) of the CT image to obtain the coordinates of the grid points in the coordinate system of the CT image. After the transformation, map_coordinates is used to interpolate the 3D array of the CT image according to the grid coordinates to obtain the local image patch near the reference point, and the medical image processing library nibabel is used to store the local image in Nifti format.

[0088] The dataset class is created as follows:

[0089] After the preprocessing of the table files and images in the first two steps, the original data set is transformed into a CSV table and a set of local CT images in Nifti format. In order to conveniently call the data set in the model training program, a data set class that inherits the torch.utils.data.Dataset class needs to be created in the final stage of data preprocessing.

[0090] The dataset object is automatically divided when it is created. In the constructor, the program accesses the CSV table to obtain the reference point information, divides the reference point information into 10 equal parts, and retains 9 and 1 of them as the training set and the test set respectively. The parameter is_valid determines whether the current dataset is a training set or a validation set, and the parameter valid_set_ndx determines which of the 10 reference point information is retained in the validation set.

[0091] Since the three models solve different problems, their labels are also different, so only an empty get_tag() interface is left in the DSet class. The specific label acquisition method is implemented by the three dataset subclasses RDSet, DDSet and ODset.

[0092] Furthermore, the neural network is constructed as follows:

[0093] The structures of the models r_net, d_net and o_net in this embodiment are basically the same. Figure 4 As shown in the figure, all of them are 7-layer structures. The first 6 layers of the model are 6 convolution blocks (Conv_block), and the 7th layer consists of 1 separate 3D convolution layer.

[0094] Among the three models, the output of the d_net model is the probability of each direction class, so the number of d_net output channels is equal to the number of direction classes; the output of r_net is the vessel radius value, and the number of output channels is 1; o_net is a binary classification model, which outputs a numerical value to represent the probability that the input sample is located on the center line, so the number of output channels is also 1.

[0095] The convolutional block is set up as follows:

[0096] The convolutional neural network used in the embodiment is composed of multiple similar convolution blocks (conv_block), each convolution block includes a three-dimensional convolution operation (nn.Conv3d()), a batch normalization operation (BatchNorm3d()) and an activation operation (nn.ReLU()), and the construction parameters of the convolution block are the number of input channels (chann_in), the number of output channels (chann_out), the convolution kernel size (k_size) and the degree of dilation (dilation, the default value is 1, indicating no holes).

[0097] nn.Conv3d() receives five-dimensional data (batch_size, channel, Depth, Height, Width), uses a specified convolution kernel to move along the three dimensions of Depth, Height, and Width of the input image, and calculates the convolution once for each step, that is, the convolution kernel is multiplied by the corresponding position of the image, and the product results are accumulated to obtain the output image.

[0098] Batch-Normalization is widely used in convolutional neural network models that use mini batches. nn.BatchNorm3d() calculates the mean and variance parameters of each mini batch and unifies all batches to a standard normal distribution with a mean of 0 and a variance of 1.

[0099] Activation functions are a type of nonlinear function that can help the model effectively retain the trained features and avoid each layer in the neural network performing only linear operations and becoming a primitive perceptron. ReLU is the full name of the Rectified Linear Unit. Its mathematical expression is shown in formula (1). It is a simple and commonly used activation function. The function graph is shown in Figure 5 .

[0100] ReLU(x)=max(0,x)(1)

[0101] The model is built as follows:

[0102] The convolutional neural network model Net consists of seven convolutional layers, and the input data is output after passing through seven convolutional layers in sequence. The first six layers are six convolutional blocks, while the seventh layer has only one single convolution operation. The number of input channels, the number of output channels, the size of the convolution kernel, and the degree of dilation are different from the first layer to the seventh layer. The number of input and output channels increases, the size of the convolution kernel decreases, and the dilation convolution is only used in the third and fourth layers. The number of output channels of the last layer is determined when the object is constructed. This is because the types of tasks performed by different model objects are different, and the final number of output channels of the model is also different. The convolutional neural network parameters are shown in Table 2.

[0103] Table 2

[0104]

[0105]

[0106] Furthermore, the model training method is as follows:

[0107] The training of the convolutional neural network model is the core part of this embodiment. In this process, the program uses DataLoader to read data from the data set and input the model, trains and verifies the model in multiple iterations, calculates the loss and other training indicators, and dynamically adjusts the learning rate parameters in the iteration to try its best to improve the training effect.

[0108] The training data is read as follows:

[0109] After data preprocessing, the data in the CAT08 dataset is processed and stored again in a file, and can be easily called by the program through the DSet class and its subclasses. The program uses the torch.utils.data.DataLoader tool in PyTorch that is responsible for processing model input to read data from the dataset object in batches and input it into the model. The batch_size parameter of DataLoader indicates the number of batches, shuffle=True indicates that the input data is shuffled, and num_workers indicates the number of threads that process data input.

[0110] The learning rate initialization method is as follows:

[0111] In this example, AdamW is selected as the optimizer of the model, and MultiStepLR is selected as the scheduler. The optimizer is responsible for automatically updating the model parameters according to the loss and learning rate during training, and adding an L2 regularization term to the loss function; while the scheduler is responsible for dynamically adjusting the learning rate during training, where the adjustment strategy of MultiStepLR is to multiply the learning rate by a specific parameter gamma when the number of iterations reaches a specified number according to the milestones array.

[0112] For example, Figure 6 The learning rate update plan for r_net and d_net, where milestones = [50, 65, 80] and gamma = 0.3.

[0113] The model iteration method is as follows:

[0114] The program performs several iterations, and each iteration requires completing model training and verification, log output, and learning rate update.

[0115] The training process is as follows:

[0116] During the training process, the program transfers samples and labels to the GPU input model and obtains the output, and calculates the model loss and other training indicators from the model output. Then the gradient is reset to zero, the gradient value is calculated through the model loss back propagation, and the model parameters are updated.

[0117] In this embodiment, three CNN models are trained, namely, the radius regression model r_net, the direction multi-classification model d_net and the deviation correction binary classification model o_net. During the training process, the three models are trained separately and different model indicators are output.

[0118] In addition to the multi-classification cross entropy loss, the training indicators of the directional multi-classification model also include accuracy. The program selects the two largest values ​​in the output of the model, and compares the direction class index output_top2_ndx corresponding to the two values ​​with the direction class index tag_top2_ndx with the largest two values ​​in the label (value is 0.5). If they are exactly the same, the number of positive samples correct_n is increased by 1. After all samples are trained, the ratio of correct_n to the total number of samples all_n is calculated to get the accuracy value.

[0119] In addition to the binary cross entropy loss, the training indicators of the correction binary classification model also include accuracy, precision, recall and F1 score. The program first uses the Sigmoid function to convert the model output into probability, then calculates the four basic indicators of TP, FP, FN, and TN, and calculates the accuracy, precision, recall and F1 score based on the four basic indicators. The Sigmoid function can limit the output of the model from (-∞, +∞) to the range of (0, 1)

[20] . The calculation formula of the function is shown in formula (2), and the image of the function is shown in Figure 7 .

[0120]

[0121] The verification process is as follows:

[0122] The validation process is similar to the training process, which also requires transferring data to the GPU and inputting it into the model to obtain output, and calculating loss and other indicators based on the output. However, the validation process is performed on the validation set, and does not require gradient calculation or optimizer update of training indicators.

[0123] The log output method is as follows:

[0124] In each iteration, logs are output, and the loss and other indicators calculated during the training and validation process of the current iteration are sent to the screen, CSV file, and TensorBoard. Among them, sending logs to TensorBoard is particularly important because it converts the training indicators of the model into images, allowing trainers to observe the changes of various indicators in real time as the number of iterations increases.

[0125] In this process, the program uses SummaryWriter to write evaluation indicators in files in the / tf-logs / directory. TensorBoard can automatically read these indicators and display them in real-time as images. TensorBoard displays the training loss and validation loss in the same image.

[0126] The model testing method is as follows:

[0127] In the model testing and result presentation part, three trained CNN models are used to build a tracker to extract the coronary artery centerline from the coronary CT images uploaded by users according to the tracking algorithm, and a visualization tool is used to display the extracted centerline.

[0128] The general idea of ​​the tracking algorithm is as follows: the tracker starts from a known initial reference point and iteratively moves forward in a certain direction and a certain distance. The direction and distance of each advance are predicted by three trained CNN models. After reaching the end of the vessel (or the coronary artery ostium), the tracker immediately returns to the initial reference point and starts tracking in another direction. The final route of the tracker can be regarded as the centerline extracted from the coronary vessel.

[0129] The tracker consists of three models: r_net, d_net, and o_net. During the tracking process, the program extracts the local CT image near the tracker and inputs it into the three models. The radius regression model r_net predicts the blood vessel radius value at that point based on the image and provides the tracker with tracking step information; the direction multi-classification model d_net lists the probability that each vector on the Fibonacci sphere is equal to the direction vector of the center line of the current position, providing a reference for the tracker's forward direction; the correction binary classification model o_net analyzes whether the current point is on the center line based on the image and determines whether the tracker has deviated from the center line.

[0130] (1) Tracking initialization

[0131] Before tracking is initialized, the only known information is the CT image input by the user and the coordinates of the initial reference point. Figure 8 As shown in the figure, during the initialization process, the program first obtains the local CT image near the initial reference point x0, and inputs it into the radius regression model r_net and the direction multi-classification model d_net respectively, and obtains the output r0 of r_net and the output d1 and d2 of d_net. Subsequently, the two trackers start from x0 and move forward r0 along the directions d1 and d2 to reach points x1 and x2 respectively, and record the coordinates of x, x1 and x2. After the initialization operation is completed, the number of initial points with known coordinates increases from the original 1 to 3.

[0132] (2) Iterative tracking

[0133] During the iterative tracking process, the two trackers start from x1 and x2 respectively and track in opposite directions d1 and d2. One tracker moves toward the starting point of the coronary artery, i.e., the coronary artery ostium, and the other tracker moves toward the end of the artery. After the iterative tracking process is completed, the two trackers should each extract half of the centerline of the coronary artery.

[0134] like Fig. 9 , the tracker starts from point x1, inputs the local CT image near x1 into r_net, and obtains the output r1 of r_net. Similarly, the local CT image at x1 is input into d_net, but instead of obtaining only one direction, the probabilities of all direction categories are obtained, and all possible directions are sorted from large to small according to the probability.

[0135] like Fig.10 After sorting the direction classes, the program screens all the forward directions d. First, exclude the directions that are greater than 60 degrees from the direction d1, and get the remaining direction list vector_list.

[0136] like Fig.11 , the tracker starts from x1, moves forward a distance r1 along the direction d with the highest probability in vector_list, reaches a new point x*, obtains the local image near the point and inputs it into the correction model o_net, allowing o_net to determine whether the new point is located on the centerline of the blood vessel (whether the o_net output is greater than 0.5).

[0137] If o_net believes that x* is not on the center line, it corrects the forward direction, that is, discards point x* and direction d, lets the tracker fall back to x1 and moves forward along the direction with the second highest probability in vector_list. If the direction with the second highest probability is still discarded, it moves forward along the direction with the third highest probability, and so on.

[0138] The reason why the direction with the highest probability in d_net is not directly used as the forward direction is that the training effect of d_net in this embodiment is not good enough. Relying solely on d_net to determine the forward direction of the tracker will cause the tracker to easily deviate from the reference center line, leave the coronary blood vessels and enter other nearby human tissues.

[0139] (3) Tracking termination

[0140] When the tracker reaches the end of the blood vessel or the coronary artery, the tracking behavior of the tracker needs to be terminated, so a standard is needed to determine whether the tracking should be terminated. The termination standard is determined based on the termination entropy. The tracker needs to calculate the termination entropy once for each step, and calculate the average of the termination entropy of the last three steps. If the value exceeds the predetermined threshold, the tracking is considered terminated.

[0141] The calculation formula of the termination entropy is as shown in formula (3), where D represents the number of direction classes and p represents the probability corresponding to each direction class in the output of d_net.

[0142]

[0143] The test indicators are implemented as follows:

[0144] After the tracker is terminated, further operations need to be performed on the centerline generated by the tracker. First, the centerline is interpolated so that the spacing between adjacent points on the centerline is reduced to about 0.03mm. Then, the heap-optimized Dijkstra minimum path algorithm is used to establish a point-to-point connection relationship between the centerline generated by the tracker and the reference centerline, and the four test indicators OV, OF, OT and AI are calculated based on this connection.

[0145] (1) Interpolation

[0146] The program uses the numpy.linspace() function to perform linear interpolation on every two adjacent points on the generated center line to ensure that the distance between adjacent points on the generated center line after interpolation reaches about 0.03 mm, which is the same as the distance between adjacent points on the reference center line.

[0147] (2) Establishing connection relationships between points

[0148] Let the centerline generated by the tracker be infer and the reference centerline be refer. The connection relationship between the points created by the program satisfies the following conditions: all points on infer can only be connected to points on refer, and points on refer can only be connected to points on infer; the first points on the two centerlines are connected to each other, and the last points are also connected to each other; and if there is a connection between infer[a] and infer[b], then infer[a] is connected to infer[b+1], or infer[a+1] is connected to infer[b].

[0149] The above connection method can ensure that each point on infer is connected to at least one point on refer, and each point on refer is also connected to at least one point on infer. The program uses the optimized Dijkstra graph search algorithm to establish this connection, making the distance between the two points in each connection as small as possible.

[0150] (3) Statistical test indicators

[0151] After establishing the connection relationship between the points of the two center lines, this embodiment counts the test indicators based on this connection relationship. While traversing all the connections, the program determines which of the points on refer and infer has changed between the current connection and the previous connection. Based on this judgment, the program changes the values ​​of indicators such as TPM, TPR, FN, FP, and counts OV, OF, OT, and AI based on these indicators after the traversal is completed.

[0152] The present invention also carries out the platform front-end design, and the design contents are as follows:

[0153] This embodiment uses Flask to maintain a model online test page. Flask is a micro web framework that can be used to quickly build websites and web services. The test page allows users to upload their own CT image files and reference centerline files. The page calls the backend code to extract the coronary centerline from the CT image, and uses a visualization tool to display the centerline extracted from the centerline reference file to the user in the form of a three-dimensional scatter plot, so that the user can compare the two centerlines to determine the effect of the centerline extraction test.

[0154] The file upload method is:

[0155] The file upload function is implemented using Flask's extension Flask-Dropzone. Flask-Dropzone allows users to upload CT image files and centerline reference files to the server by dragging and dropping.

[0156] The model testing method is:

[0157] In this part, the JavaScript program uses ajax to call the predict() and progress() functions of the backend to complete the extraction of the centerline and the acquisition of the extraction progress respectively.

[0158] The centerline visualization method is:

[0159] This part adds an ECharts 3D scatter plot (Scatter3D) to the page through the library pyecharts. The plot displays both the reference centerline (reference) and the centerline extracted by the tracker (predict_coords). The toolbox is used to add image saving tools and data viewing tools, allowing users to obtain the centerline image and the coordinates of each point on the centerline.

[0160] This embodiment also conducted experiments and analyzed the results as follows:

[0161] In the preprocessing stage, this embodiment generates three different sets of experimental data according to different values ​​of the number of direction classes (100, 200, and 500). In the model training stage, the three different sets of data are sequentially input into the neural network to train three sets of model parameters. In the model testing stage, the embodiment switches the model parameters used in the test process by selecting different options in the "Number of Direction Classes" column on the test interface, thereby testing the three sets of model parameters separately.

[0162] Preprocessing stage: In the preprocessing stage, the embodiment generates experimental data online100, online200 and online500 for the radius regression model and the direction multi-classification model, and generates experimental data onoff100, onoff200 and onoff500 for the deviation correction binary classification model.

[0163] The model training phase is as follows:

[0164] In this embodiment, the CNN model is trained according to the multiple sets of experimental data provided by the preprocessing process during the model training stage. During the training process, the program calculates the loss and other indicators of each model on the training set and the validation set. The experiment is repeated after adjusting the experimental conditions for many times, and the model is optimized many times. Table 3 shows the statistical values ​​of various model indicators for the three models when the number of direction classes is 100, 200, and 500 respectively.

[0165] Table 3

[0166]

[0167] Among them, the training indicators of r_net and o_net are relatively excellent. No matter what value the direction classification number takes, the loss of r_net and o_net is very small; other indicators of o_net such as accuracy, precision, recall rate and F1 score are also very high (the original data is extremely close to 1).

[0168] The loss of the directional multi-classification model is still a bit high, and although the highest accuracy has reached 0.977, it is still far from the ideal value. This leads to frequent misprediction of the tracking direction when only r_net and d_net are used for tracking experiments in the embodiment at the beginning, causing the tracker to deviate from the reference center line shortly after starting from the initial reference point. This is also the direct reason for introducing o_net in this embodiment.

[0169] The testing phases are as follows:

[0170] In the test phase, this embodiment sets the number of direction classes to 100, 200 and 500 respectively, selects a point from the centerline reference file of the four blood vessels in the sub-dataset dataset00 as a known initial point, extracts the four blood vessel centerlines from the CT image, and calculates the values ​​of the evaluation indicators OV, OF, OT and AI according to the extracted centerlines and the reference centerlines, so as to compare the impact of the difference in the number of direction classes on the centerline extraction effect.

[0171] Among the four evaluation indicators, the higher the OV, OF and OT, the higher the overlap between the two center lines, that is, the better the center line extraction effect; and the lower the AI, the smaller the average distance deviation between the two center lines, that is, the better the center line extraction effect.

[0172] As shown in Table 4, in general, the values ​​of OV, OF and OT indicators when the number of direction classes is 500 are slightly higher than the values ​​of these three indicators when the number of direction classes is 100 or 200, and the value of AI indicator is slightly lower than the AI ​​value measured when the number of direction classes is 100 or 200. Only in vessel0, the OV and OT indicators are slightly lower when the number of direction classes is 500. Therefore, it can be considered that the centerline extraction effect is better when the number of direction classes is 500.

[0173] Table 4

[0174]

[0175] At the same time, the embodiment performs centerline extraction tests on all sub-datasets with the number of direction classes equal to 500, calculates OV, OF, OT and AI indicators, and calculates the average values ​​of the indicators. As shown in Table 5, the average values ​​of OV, OF, and OT indicators reach more than 90%, and the AI ​​indicator is controlled below 0.35 mm.

[0176] Table 5

[0177]

[0178]

[0179] In addition, the embodiment also conducted a round of centerline extraction test on the sub-dataset dataset03 using only r_net and d_net, and calculated four indicators, OV, OF, OT and AI, to evaluate the impact of the introduction of the correction model o_net on the centerline extraction effect.

[0180] As shown in Table 6, after the introduction of o_net, the OV, OF and OT indicators all increased, and the AI ​​indicator decreased. Among them, OV increased by an average of 25.73%, OF increased by an average of 51.82%, OT increased by an average of 25.24%, and AI decreased by an average of 11.82%, which shows that the introduction of o_net has a significant improvement in the centerline extraction effect.

[0181] Table 6

[0182]

[0183] It can be seen from the data that the average values ​​of the centerline overlap rate measurement indicators overlap (OV), overlap before the first error (OF), and overlap of the clinically relevant vascular part (OT) in this embodiment can reach more than 90%, and the average value of the internal mean (AI) is successfully controlled below 0.35mm, which can meet clinical needs.

[0184] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for extracting the centerline of a coronary artery, characterized in that: The following steps are involved: Load the centerline reference file and CT image file from the CAT08 dataset and perform preprocessing to obtain the preprocessed dataset class; Constructing a convolutional neural network model, wherein the convolutional neural network model includes a radius regression model, a direction multi-classification model, and a correction binary classification model; Reading data from the preprocessed data set class, inputting the read data into the convolutional neural network model for training and verification, and obtaining a trained model; A tracker is constructed based on the trained model. Starting from the initial reference point, it iteratively moves in the predicted direction by the distance of the predicted vessel radius to extract the centerline of the coronary artery.

2. The method for extracting the coronary artery centerline according to claim 1, characterized in that: The method for pre-processing comprises: Preprocessing the centerline reference file to convert it into a preprocessed file for convolutional neural network model training and testing; Read the CT image, obtain the meta-information and three-dimensional array of the CT image, and obtain the local CT image near the reference point based on the spline interpolation method; Create a dataset class that inherits the torch.utils.data.Dataset class through the preprocessed file and the local CT image near the reference point.

3. The method for extracting the coronary artery centerline according to claim 2, characterized in that: The method for preprocessing the centerline reference file includes: The three-dimensional coordinates and vessel radius information of each reference point on the coronary artery centerline are extracted from the centerline reference file, the centerline direction at each reference point is determined according to the spatial position relationship between the reference points, the centerline direction vector is converted into a point index on the Fibonacci sphere, new reference points outside the centerline are generated as negative samples according to the existing reference points, and all reference point information is integrated and stored as a CSV file.

4. The method for extracting the coronary artery centerline according to claim 2, characterized in that: The methods for preprocessing CT images include: The meta-information and three-dimensional array of the CT image are obtained, and a three-dimensional grid of size 19×19×19 centered at the origin is generated, and the center of the grid is moved to the reference point; the coordinate transformation operation is performed on all the grid points on the grid according to the meta-information of the CT image to obtain the coordinates of the grid points in the coordinate system of the CT image; the three-dimensional array of the CT image is interpolated according to the transformed grid coordinates to obtain the local image near the reference point, and the local image is stored in Nifti format.

5. The method for extracting the coronary artery centerline according to claim 1, characterized in that: The first six layers of the convolutional neural network model are six convolution blocks, and the seventh layer is a separate three-dimensional convolution layer. The convolution block includes a three-dimensional convolution operation, a batch normalization operation and an activation operation. The construction parameters of the convolution block are the number of input channels, the number of output channels, the size of the convolution kernel and the degree of voids, respectively. The number of input and output channels shows an increasing trend, the size of the convolution kernel shows a decreasing trend, and void convolution is only used for the third and fourth layers.

6. The method for extracting the coronary artery centerline according to claim 1, characterized in that: The method for training the convolutional neural network model includes: AdamW is selected as the optimizer of the model, and MultiStepLR is selected as the scheduler; the optimizer is used to automatically update the model parameters according to the loss and learning rate during training, and add an L2 regularization term to the loss function; the scheduler is used to dynamically adjust the learning rate during training.

7. The method for extracting the coronary artery centerline according to claim 1, characterized in that: The tracking method of the tracker includes: The radius regression model predicts the blood vessel radius value based on the image and provides tracking step information for the tracker. The direction multi-classification model lists the probability that each vector on the Fibonacci sphere is equal to the centerline direction vector of the current position, providing a reference for the tracker's forward direction. The deviation correction binary classification model determines whether the tracker deviates from the centerline based on image analysis to determine whether the current point is on the centerline. When the tracker reaches the end of the blood vessel or the coronary artery orifice, the tracking behavior of the tracker is terminated.

8. The method for extracting the coronary artery centerline according to claim 7, characterized in that: Determining whether to terminate the tracker by using the termination entropy, the determination method includes: calculating the average of the termination entropies of the most recent three steps, and terminating the tracking when the average exceeds a predetermined threshold; The calculation expression of the termination entropy is: In the formula, D represents the number of direction classes, p i Represents the probability corresponding to each direction class in the output of the direction multi-classification model.

9. The method for extracting the coronary artery centerline according to claim 7, characterized in that: After extracting the coronary artery centerline, the accuracy of the extracted coronary artery centerline was judged by the coincidence, coincidence before the first error, coincidence of clinically relevant vascular parts and average distance indicators.

Images