Method, device and equipment for extracting coronary vessel center line and storage medium

Abstract

The application belongs to the technical field of medical image processing, and discloses a coronary vessel center line extraction method, device, equipment and storage medium. The method comprises the following steps: determining a first center line of a first coronary vessel according to a coronary vessel starting point, a coronary vessel terminal end point and a center distance intensity graph of the first coronary vessel; dividing the first center line into multiple sub-center lines, and then sampling each sub-center line to obtain N sampling points, and determining a corresponding direction vector of each sampling point on a sub-center line tangent; determining an image cutting graph of the N sampling points based on the N sampling points and the corresponding direction vectors; inputting the image cutting graph of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data; inputting the high-dimensional feature data into a target Transformer network to obtain a corresponding offset of each sampling point; and obtaining a second center line based on the first center line and the offset. The first center line can be optimized to obtain the second center line.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and in particular to a coronary vessel centerline extraction method, device and equipment and a storage medium. BACKGROUND

[0002] Cardiovascular disease has become one of the major diseases threatening human life safety. In clinical practice, doctors need to reconstruct coronary vessels to diagnose various vascular diseases. One important step to achieve automatic and intelligent technical means is to accurately extract the coronary vessel centerline, which is also the basis for coronary analysis. However, the existing method has poor effect in extracting the centerline at relatively special lumen positions such as bifurcation and lesions. SUMMARY

[0003] The main purpose of the present application is to provide a coronary vessel centerline extraction method, device, equipment and storage medium, which aims to solve the technical problem of poor effect in extracting the centerline at relatively special lumen positions in the prior art.

[0004] To achieve the above purpose, the present application provides a coronary vessel centerline extraction method, which comprises the following steps:

[0005] Based on the three-dimensional blood vessel image of the first coronary artery, the coronary artery starting point, the coronary artery end point and the center distance intensity map of the first coronary artery are determined;

[0006] According to the coronary artery starting point, the coronary artery end point and the center distance intensity map, the first centerline of the first coronary artery is determined;

[0007] The first centerline is divided into multiple sub-centerlines, and each sub-centerline is sampled to obtain N sampling points, and the corresponding direction vectors of each sampling point on the sub-centerline tangent are determined;

[0008] Based on the N sampling points and the corresponding direction vectors, an image cutting map of the N sampling points is determined;

[0009] The image cutting map of the N sampling points is input into a target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents the size of the image cutting map, and C represents the number of features;

[0010] The high-dimensional feature data with a size of N*H*W*C is input into a target Transformer network to obtain the offset corresponding to each sampling point;

[0011] Based on the first centerline and the offset corresponding to each sampling point, a second centerline is obtained.

[0012] Optionally, the first coronary artery based on the three-dimensional blood vessel image, determining the coronary artery starting point, coronary artery end point and center distance intensity map of the first coronary artery, comprising:

[0013] According to the three-dimensional coronary image, the coronary rough segmentation image of the first coronary artery is determined;

[0014] The coronary rough segmentation image is input into the first convolutional neural network to obtain the coronary starting point of the first coronary artery;

[0015] The coronary rough segmentation image is input into the second convolutional neural network to obtain the coronary end point of the first coronary artery;

[0016] The coronary rough segmentation image is input into the third convolutional neural network to obtain the center distance intensity map of the first coronary artery.

[0017] Optionally, the three-dimensional coronary image is determined according to the three-dimensional coronary image, and the coronary rough segmentation image of the first coronary artery comprises:

[0018] The three-dimensional coronary image is input into the fourth convolutional neural network to obtain the coronary rough segmentation image of the first coronary artery; or,

[0019] The three-dimensional coronary image is processed by using the image processing method of Frangi filtering to obtain the coronary rough segmentation image of the first coronary artery.

[0020] Optionally, the N sampling points and the corresponding direction vectors are determined based on the N sampling points and the corresponding direction vectors, and the image cutting map of the N sampling points is determined.

[0021] The target plane of each sampling point is determined, wherein the normal vector of the target plane is the direction vector corresponding to the sampling point on the sub-center line tangent, and the sampling point is on the target plane;

[0022] The three-dimensional blood vessel image is cut by using the target plane of each sampling point respectively to obtain the image cutting map of the N sampling points.

[0023] Optionally, before the image cutting map of the N sampling points is input into the target CNN convolutional neural network, it further comprises:

[0024] Based on the three-dimensional blood vessel image of the second coronary artery, the image cutting map of the first target center point is determined, wherein the first target center line point is a point on the first center line of the second coronary artery;

[0025] The first target center point is input into the CNN convolutional neural network and the Transformer network to obtain the predicted offset of the first target center point;

[0026] Determine the offset between the first target center point and the annotation center point, wherein the annotation center point is a point on the annotation center line;

[0027] Based on the predicted offset and the labeled offset, the CNN convolutional neural network and the Transformer network are updated to obtain the target CNN convolutional neural network and the target Transformer network.

[0028] Optionally, the step of inputting the high-dimensional feature data of size N*H*W*C into the target Transformer network to obtain the offset corresponding to each sampling point includes:

[0029] The high-dimensional feature data of size N*H*W*C is transformed into target data, wherein the target data is the feature data of the high-dimensional feature data transformed into an image crop of N sampling points;

[0030] The target data is input into the target Transformer network to obtain the offset corresponding to each sampling point.

[0031] Optionally, a second centerline is obtained based on the first centerline and the offsets corresponding to each sampling point, including:

[0032] Determine the first center point on the first center line, wherein the first center point is the coordinate of each sampling point on the first center line;

[0033] The second center point is obtained based on the first center point and the offsets corresponding to each sampling point;

[0034] The second center point is processed by spline interpolation to obtain the second center line.

[0035] Furthermore, to achieve the above objectives, the present invention also proposes a device for extracting the coronary artery centerline, the device comprising:

[0036] The determination module is used to determine the coronary origin, coronary terminal end point, and center distance intensity map of the first coronary artery based on the three-dimensional vascular image of the first coronary artery.

[0037] The determining module is further configured to determine the first centerline of the first coronary artery based on the coronary artery origin, the coronary artery terminal point, and the center distance intensity map;

[0038] The sampling module is used to divide the first centerline into multiple sub-centerlines, and then sample each sub-centerline to obtain N sampling points, and determine the direction vector corresponding to each sampling point on the tangent of the sub-centerline.

[0039] The determination module is further configured to determine an image clipping diagram of the N sampling points based on the N sampling points and the corresponding direction vectors.

[0040] The determination module is further configured to input the image clipping diagram of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents the size of the image clipping diagram, and C represents the number of features.

[0041] The determination module is further configured to input the high-dimensional feature data with the size of N*H*W*C into a target Transformer network to obtain the offset corresponding to each sampling point.

[0042] The determination module is further configured to obtain a second center line based on the first center line and the offset corresponding to each sampling point.

[0043] In addition, to achieve the above object, the present application further provides a coronary vessel center line extraction device, which comprises a memory, a processor and a coronary vessel center line extraction program stored in the memory and executable on the processor, and the coronary vessel center line extraction program is configured to implement the steps of the coronary vessel center line extraction method as described above.

[0044] In addition, to achieve the above object, the present application further provides a storage medium having a coronary vessel center line extraction program stored thereon, and the coronary vessel center line extraction program implements the steps of the coronary vessel center line extraction method as described above when executed by a processor.

[0045] The present invention proposes a method, apparatus, device, and storage medium for extracting the coronary artery centerline. Based on a three-dimensional vascular image of a first coronary artery, the method determines the coronary artery origin, coronary artery terminal, and center distance intensity map of the first coronary artery. Based on the coronary artery origin, coronary artery terminal, and center distance intensity map, a first centerline of the first coronary artery is determined. The first centerline is divided into multiple sub-centerlines, and each sub-centerline is sampled to obtain N sampling points. The direction vector corresponding to each sampling point on the tangent of the sub-centerline is determined. Based on the N sampling points and the corresponding direction vectors, an image crop of the N sampling points is determined. The image crop of the N sampling points is input into a target CNN convolutional neural network to obtain high-dimensional feature data of size N*H*W*C, where H*W represents the size of the image crop and C represents the number of features. The high-dimensional feature data of size N*H*W*C is input into a target Transformer network to obtain the offset corresponding to each sampling point. Based on the first centerline and the offsets corresponding to each sampling point, a second centerline is obtained. By using the above method, the offset of the center point can be predicted by using a sequence network based on the continuity of the images before and after the difficult-to-locate areas in the coronary artery centerline. By combining the first centerline and the predicted offset of the center point, the first centerline can be optimized to obtain a more accurate coronary artery centerline. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the structure of the coronary artery centerline extraction device in the hardware operating environment involved in the embodiments of the present invention;

[0047] Figure 2 This is a flowchart illustrating the first embodiment of the method for extracting the coronary artery centerline of the present invention;

[0048] Figure 3 This is a schematic diagram of the process for generating offset in the first embodiment of the method for extracting the coronary artery centerline of the present invention;

[0049] Figure 4a This is a schematic diagram showing the first centerline in the first embodiment of the coronary artery centerline extraction method of the present invention;

[0050] Figure 4b This is a schematic diagram showing the second centerline in the first embodiment of the method for extracting the coronary artery centerline of the present invention;

[0051] Figure 5 This is a flowchart illustrating the second embodiment of the method for extracting the coronary artery centerline of the present invention.

[0052] Figure 6 This is a structural block diagram of the first embodiment of the coronary artery centerline extraction device of the present invention.

[0053] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0055] Reference Figure 1 , Figure 1 This is a schematic diagram of the coronary artery centerline extraction device in the hardware operating environment of the embodiment of the present invention.

[0056] like Figure 1 As shown, the coronary artery centerline extraction device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0057] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the device for extracting the coronary artery centerline and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0058] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a program for extracting the coronary artery centerline.

[0059] exist Figure 1The network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the coronary vessel center line extraction device can be arranged in the coronary vessel center line extraction device, the coronary vessel center line extraction device calls the coronary vessel center line extraction program stored in the memory 1005 through the processor 1001, and executes the coronary vessel center line extraction method provided in the embodiment of the application.

[0060] Based on the above hardware structure, the coronary vessel center line extraction method embodiment of the application is provided.

[0061] Reference Figure 2 , Figure 2 The flowchart of the first embodiment of the coronary vessel center line extraction method of the application is shown.

[0062] In this embodiment, the coronary vessel center line extraction method comprises the following steps:

[0063] Step S10: Based on the three-dimensional vessel image of the first coronary artery, the coronary artery starting point, the coronary artery end point and the center distance intensity map of the first coronary artery are determined.

[0064] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a mobile phone, a tablet computer, a personal computer, etc., or an electronic device or a coronary vessel center line extraction device capable of realizing the above functions. The embodiment and the following embodiments will be described below taking the coronary vessel center line extraction device as an example.

[0065] It should be noted that the three-dimensional vessel image includes CT and MR images; the distance intensity map refers to the distance intensity map of the first coronary artery segmentation, and the feature of the distance intensity map is that the value of the position closer to the vessel center is lower.

[0066] Step S20: The first center line of the first coronary artery is determined according to the coronary artery starting point, the coronary artery end point and the center distance intensity map.

[0067] In a specific implementation, the first center line of the first coronary artery can be obtained based on the minimal-path algorithm.

[0068] It can be understood that the first center line is a preliminary extracted coronary vessel center line, and when the coronary artery has adhesion at the bifurcation position or the segmentation position, the effect of the first center line is poor (i.e., the deviation between the first center line and the actual coronary center line is large).

[0069] In an embodiment, the first coronary artery based on the three-dimensional blood vessel image, the coronary artery starting point, the coronary artery end point and the center distance intensity diagram of the first coronary artery are determined, comprising:

[0070] According to the three-dimensional coronary image, the coronary rough segmentation image of the first coronary artery is determined;

[0071] The coronary rough segmentation image is input into the first convolutional neural network to obtain the coronary starting point of the first coronary artery;

[0072] The coronary rough segmentation image is input into the second convolutional neural network to obtain the coronary end point of the first coronary artery;

[0073] The coronary rough segmentation image is input into the third convolutional neural network to obtain the center distance intensity diagram of the first coronary artery.

[0074] It should be noted that the coronary rough segmentation image is an image obtained by performing coronary segmentation processing on the three-dimensional coronary image, and the coronary rough segmentation image contains all the coronary arteries in the three-dimensional coronary image.

[0075] It can be understood that the first convolutional neural network, the second convolutional neural network and the third convolutional neural network are all trained convolutional neural networks.

[0076] In a specific implementation, the training process of the trained first convolutional neural network is specifically: a plurality of coronary rough segmentation images with manually labeled coronary starting points are input into an initial convolutional neural network to obtain predicted coronary starting points, the manually labeled coronary starting points are compared with the predicted coronary starting points, and the differences obtained by comparison are continuously fed back to the initial convolutional neural network, so as to iteratively update the convolutional neural network, thereby obtaining the trained convolutional neural network, i.e. the first convolutional neural network, which can be used to detect the coronary starting point in the coronary rough segmentation image.

[0077] In a specific implementation, the training process of the trained second convolutional neural network is specifically: a plurality of coronary rough segmentation images with manually labeled coronary end points are input into an initial convolutional neural network to obtain predicted coronary end points, the manually labeled coronary end points are compared with the predicted coronary end points, and the differences obtained by comparison are continuously fed back to the initial convolutional neural network, so as to iteratively update the convolutional neural network, thereby obtaining the trained convolutional neural network, i.e. the second convolutional neural network, which can be used to detect the coronary end point in the coronary rough segmentation image.

[0078] In a specific implementation, the coronary artery starting point in the three-dimensional blood vessel image can also be obtained by an image processing method, and the coronary artery end point in the three-dimensional blood vessel image can also be obtained by a region grow method.

[0079] In an embodiment, the determining, according to the three-dimensional coronary artery image, of the coronary artery rough segmentation image of the first coronary artery comprises:

[0080] inputting the three-dimensional coronary artery image into a fourth convolutional neural network to obtain the coronary artery rough segmentation image of the first coronary artery; or

[0081] processing the three-dimensional coronary artery image by using an image processing method of Frangi filtering to obtain the coronary artery rough segmentation image of the first coronary artery.

[0082] It should be noted that the fourth convolutional neural network is a trained convolutional neural network.

[0083] Step S30: dividing the first centerline into a plurality of sub-centerlines, sampling each sub-centerline to obtain N sampling points, and determining a direction vector corresponding to each sampling point on a tangent line of the sub-centerline.

[0084] Step S40: determining an image cutout map of the N sampling points based on the N sampling points and the corresponding direction vectors.

[0085] In an embodiment, the determining, based on the N sampling points and the corresponding direction vectors, of the image cutout map of the N sampling points comprises:

[0086] determining a target plane of each sampling point, wherein a normal vector of the target plane is the direction vector corresponding to the sampling point on the tangent line of the sub-centerline, and the sampling point is on the target plane;

[0087] cutting the three-dimensional blood vessel image by using the target plane of each sampling point to obtain the image cutout map of the N sampling points.

[0088] It can be understood that the target plane of each sampling point is different, and therefore the image cutout map of each sampling point is different.

[0089] Step S50: inputting the image cutout map of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents the size of the image cutout map, and C represents the number of features.

[0090] It should be noted that the target CNN convolutional neural network is a trained CNN convolutional neural network, and the CNN convolutional neural network can be a resnet network or a densnet network.

[0091] It should be noted that N>1.

[0092] It should be noted that CNN convolutional neural networks can extract local lumen features from image cropping, thus achieving automated local feature extraction of coronary artery lumens.

[0093] Step S60: Input the high-dimensional feature data of size N*H*W*C into the target Transformer network to obtain the offset corresponding to each sampling point.

[0094] It should be noted that the target Transformer network is a trained Transformer network, which can be a bidirectional LSTM network or a sequence network such as an RNN.

[0095] It should be noted that the Transformer network has the characteristic of parallelism. By simultaneously inputting the high-dimensional features corresponding to N sampling points into the Transformer network, it can learn the relationship between the front and back of the coronary arteries, especially the locations of lesions, stenosis, and bifurcation in the coronary arteries. The Transformer network can fully learn the continuity of the anatomy of the blood vessels before and after the lesions. When there are lesions or bifurcation locations in the coronary arteries in a single frame image, it is difficult to identify the center point from a single frame image. Based on the sequence network (i.e., the Transformer network), it can better predict the accurate location of the center point by using the continuity of the front and back images of the difficult-to-locate areas of the blood vessel centerline.

[0096] It should be noted that since the CNN features of each point are unrelated, by inputting the high-dimensional features of N points into the Transformer network, the spatial continuity of the cross-sectional image of the N sampling points can be learned by taking advantage of the correlation of global features of the Transformer.

[0097] In specific implementations, such as Figure 3 As shown, the image crop of the sampling point is first input into the target CNN convolutional neural network to obtain high-dimensional features, and then the high-dimensional features are input into the target Transformer network to obtain the offset of the sampling point.

[0098] In one embodiment, the step of inputting the high-dimensional feature data of size N*H*W*C into the target Transformer network to obtain the offset corresponding to each sampling point includes:

[0099] The high-dimensional feature data of size N*H*W*C is transformed into target data, wherein the target data is the feature data of the high-dimensional feature data transformed into an image crop of N sampling points;

[0100] Input the target data into the target Transformer network to obtain the offset corresponding to each sampling point.

[0101] It should be noted that the target data refers to (N, H*W*C) data.

[0102] In a specific implementation, the high-dimensional feature data N*H*W*C output by the target CNN convolutional neural network needs to be dimensionally transformed to obtain (N, H*W*C) data, and then the (N, H*W*C) data is input into the target Transformer network to obtain the offset corresponding to each sampling point.

[0103] Step S70: obtaining a second center line based on the first center line and the offset corresponding to each sampling point.

[0104] In a specific implementation, Figure 4a denotes the first center line, Figure 4b denotes the second center line, and Figure 4a It can be seen from equations (1) and (2) that the second center line is more accurate than the first center line at the bifurcation position of the coronary artery. Figure 4b

[0105] In an embodiment, obtaining the second center line based on the first center line and the offset corresponding to each sampling point comprises:

[0106] determining a first center point on the first center line, wherein the first center point is the coordinate of each sampling point on the first center line;

[0107] obtaining a second center point based on the first center point and the offset corresponding to each sampling point;

[0108] processing the second center point by a spline interpolation method to obtain the second center line.

[0109] It should be noted that processing the second center point by the spline interpolation method refers to interpolating a continuous function based on the second center point, so that this continuous curve passes through all the second center points, and also estimates the approximate value of the function at other points, thereby obtaining the second center line.

[0110] ​The embodiment determines the coronary artery starting point, the coronary artery end point and the center distance intensity map of the first coronary artery based on the three-dimensional blood vessel image of the first coronary artery, determines the first center line of the first coronary artery according to the coronary artery starting point, the coronary artery end point and the center distance intensity map, divides the first center line into multiple sub-center lines, samples each sub-center line to obtain N sampling points, and determines the corresponding direction vector of each sampling point on the sub-center line tangent, determines the image interception map of the N sampling points based on the N sampling points and the corresponding direction vector, inputs the image interception map of the N sampling points into the target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents the size of the image interception map, and C represents the feature number, inputs the high-dimensional feature data with the size of N*H*W*C into the target Transformer network to obtain the offset of each sampling point, and obtains the second center line based on the first center line and the offset of each sampling point. In the foregoing manner, the offset of the center point is predicted according to the continuity of the images before and after the difficult positioning place in the coronary artery center line by using the sequence network, and the first center line is optimized in combination with the offset of the predicted center point, so that a more accurate coronary artery center line is obtained.

[0111] Reference Figure 5 , Figure 5 The flowchart of the second embodiment of the coronary artery center line extraction method is shown.

[0112] Based on the first embodiment, the coronary artery center line extraction method further includes the following steps before step S50.

[0113] Step S401: Determine the image interception map of the first target center point based on the three-dimensional blood vessel image of the second coronary artery, wherein the first target center point is a point on the first center line of the second coronary artery.

[0114] It should be noted that the three-dimensional blood vessel image of the second coronary artery is not limited to one, and the three-dimensional blood vessel image of the second coronary artery refers to image data used for training the CNN convolutional neural network and the Transformer network.

[0115] Step S402: Input the image interception map of the first target center point into the CNN convolutional neural network and the Transformer network to obtain the predicted offset of the first target center point.

[0116] Step S403: Determine the labeled offset of the first target center point and the labeled center point, wherein the labeled center point is a point on the labeled center line.

[0117] It should be noted that the center line marked is a blood vessel center line marked in advance on the three-dimensional blood vessel image of the second coronary artery.

[0118] Step S404: updating the CNN convolutional neural network and the Transformer network based on the predicted offset and the marked offset to obtain a target CNN convolutional neural network and a target Transformer network.

[0119] In a specific implementation, a loss value can be determined according to the predicted offset and the actual offset, and then the parameters of the CNN convolutional neural network and the Transformer network are updated according to the loss value to obtain a target CNN convolutional neural network and a target Transformer network. The loss value is calculated as follows:

[0120] Loss = sqrt((x-x0)^2+(y-y0)^2)

[0121] In the embodiment, the image cutting graph of the first target center point is input into the CNN convolutional neural network and the Transformer network to obtain a predicted offset of the first target center point. A marked offset of the first target center point and a marked center point is determined, wherein the marked center point is a point on the marked center line. The CNN convolutional neural network and the Transformer network are updated based on the predicted offset and the marked offset to obtain a target CNN convolutional neural network and a target Transformer network. In this way, the loss value can be determined by the predicted offset and the marked offset, and then the target CNN convolutional neural network and the target Transformer network are updated according to the loss value.

[0122] In addition, the embodiment of the present application also provides a storage medium, and the storage medium stores a coronary blood vessel center line extraction program. When the coronary blood vessel center line extraction program is executed by a processor, the steps of the coronary blood vessel center line extraction method described above are implemented.

[0123] Reference Figure 6 , Figure 6 FIG. 1 is a structural block diagram of a coronary blood vessel center line extraction device according to a first embodiment of the present application.

[0124] As shown in FIG. 1, the coronary blood vessel center line extraction device according to the first embodiment of the present application comprises: Figure 6

[0125] ​A determining module 10 is configured to determine a coronary artery starting point, a coronary artery ending point and a center distance intensity map of a first coronary artery based on a three-dimensional blood vessel image of the first coronary artery.

[0126] The determining module 10 is further configured to determine a first center line of the first coronary artery according to the coronary artery starting point, the coronary artery ending point and the center distance intensity map.

[0127] A sampling module 20 is configured to divide the first center line into a plurality of sub-center lines, sample each sub-center line to obtain N sampling points, and determine a corresponding direction vector of each sampling point on a tangent line of the sub-center line.

[0128] The determining module 10 is further configured to determine an image clipping map of the N sampling points based on the N sampling points and the corresponding direction vectors.

[0129] The determining module 10 is further configured to input the image clipping map of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents a size of the image clipping map, and C represents a feature number.

[0130] The determining module 10 is further configured to input the high-dimensional feature data with the size of N*H*W*C into a target Transformer network to obtain a corresponding offset of each sampling point.

[0131] The determining module 10 is further configured to obtain a second center line based on the first center line and the corresponding offset of each sampling point.

[0132] It should be understood that the above is only an example, and the technical solutions of the present application are not limited in any way. In specific applications, those skilled in the art can make settings according to needs, and the present application does not limit this.

[0133] The embodiment determines a coronary artery starting point, a coronary artery end point and a center distance intensity map of the first coronary artery based on a three-dimensional blood vessel image of the first coronary artery; determines a first center line of the first coronary artery according to the coronary artery starting point, the coronary artery end point and the center distance intensity map; divides the first center line into multiple sub-center lines, samples each sub-center line to obtain N sampling points, and determines a corresponding direction vector of each sampling point on a sub-center line tangent; determines an image interception map of the N sampling points based on the N sampling points and the corresponding direction vectors; inputs the image interception map of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents the size of the image interception map, and C represents the number of features; inputs the high-dimensional feature data with the size of N*H*W*C into a target Transformer network to obtain a corresponding offset of each sampling point; and obtains a second center line based on the first center line and the corresponding offset of each sampling point. In this way, the offset of the center point is predicted according to the continuity of the images before and after the difficult positioning place in the coronary artery center line by using the sequence network, and the first center line is optimized in combination with the offset of the predicted center point, so that a more accurate coronary artery center line is obtained.

[0134] In an embodiment, the determining module 10 is further configured to:

[0135] determine a coronary artery coarse segmentation image of the first coronary artery according to the three-dimensional coronary image;

[0136] input the coronary artery coarse segmentation image into a first convolutional neural network to obtain a coronary artery starting point of the first coronary artery;

[0137] input the coronary artery coarse segmentation image into a second convolutional neural network to obtain a coronary artery end point of the first coronary artery;

[0138] input the coronary artery coarse segmentation image into a third convolutional neural network to obtain a center distance intensity map of the first coronary artery.

[0139] In an embodiment, the determining module 10 is further configured to:

[0140] input the three-dimensional coronary image into a fourth convolutional neural network to obtain a coronary artery coarse segmentation image of the first coronary artery; or

[0141] perform image processing on the three-dimensional coronary image by using a Frangi filtering method to obtain a coronary artery coarse segmentation image of the first coronary artery.

[0142] In an embodiment, the sampling module 20 is further configured to:

[0143] determining a target plane of each sampling point, wherein a normal vector of the target plane is a direction vector corresponding to the sampling point on a sub-center line tangent, and the sampling point is on the target plane;

[0144] performing image interception on the three-dimensional blood vessel image respectively by using the target plane of each sampling point to obtain an image interception image of N sampling points.

[0145] In an embodiment, the determining module 10 is further configured to:

[0146] determining an image interception image of a first target center point based on the three-dimensional blood vessel image of the second coronary artery, wherein the first target center line point is a point on a first center line of the second coronary artery;

[0147] inputting the first target center point into a CNN convolutional neural network and a Transformer network to obtain a predicted offset of the first target center point;

[0148] determining a labeled offset of the first target center point and a labeled center point, wherein the labeled center point is a point on a labeled center line;

[0149] updating the CNN convolutional neural network and the Transformer network based on the predicted offset and the labeled offset to obtain a target CNN convolutional neural network and a target Transformer network.

[0150] In an embodiment, the determining module 10 is further configured to:

[0151] converting the high-dimensional feature data with the size of N*H*W*C into target data, wherein the target data is feature data converted from the high-dimensional feature data into an image interception image of N sampling points;

[0152] inputting the target data into a target Transformer network to obtain an offset corresponding to each sampling point.

[0153] In an embodiment, the determining module 10 is further configured to:

[0154] determining a first center point on the first center line, wherein the first center point is a coordinate of each sampling point on the first center line;

[0155] obtaining a second center point based on the first center point and the offset corresponding to each sampling point;

[0156] processing the second center point by a spline interpolation method to obtain a second center line.

[0157] It should be noted that the above-described workflow is merely illustrative and does not limit the scope of protection of the present application. In actual applications, a person skilled in the art can select part or all of the above-described workflow to achieve the purpose of the embodiment according to actual needs, which is not limited herein.

[0158] In addition, technical details not described in detail in the embodiment can be referred to the extraction method of the coronary vessel center line provided by any embodiment of the present application, which will not be described here.

[0159] In addition, it should be noted that in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system that includes the element.

[0160] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0161] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0162] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A method for extracting a centerline of a coronary vessel, characterized by, The extraction method of the coronary vessel center line comprises: determining a coronary artery starting point, a coronary artery end point and a center distance intensity map of the first coronary artery based on a three-dimensional blood vessel image of the first coronary artery; determining a first center line of the first coronary artery according to the coronary artery starting point, the coronary artery end point and the center distance intensity map; dividing the first center line into multiple sub-center lines, and then sampling each sub-center line to obtain N sampling points, and determining a corresponding direction vector of each sampling point on a sub-center line tangent; determining an image interception map of the N sampling points based on the N sampling points and the corresponding direction vectors; inputting the image interception map of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data with a size of N*H*W*C, wherein H*W represents the size of the image interception map, and C represents the number of features; inputting the high-dimensional feature data with the size of N*H*W*C into a target Transformer network to obtain a corresponding offset of each sampling point; obtaining a second center line based on the first center line and the corresponding offset of each sampling point; before the image interception map of the N sampling points is input into the target CNN convolutional neural network, the method further comprises: determining an image interception map of a first target center point based on a three-dimensional blood vessel image of a second coronary artery, wherein the first target center point is a point on a first center line of the second coronary artery; inputting the first target center point into the CNN convolutional neural network and the Transformer network to obtain a predicted offset of the first target center point; determining a labeled offset of a labeled center point from a first target center point, wherein the labeled center point is a point on a labeled center line; updating the CNN convolutional neural network and the Transformer network based on the predicted offset and the labeled offset to obtain a target CNN convolutional neural network and a target Transformer network; obtaining a second center line based on the first center line and the corresponding offset of each sampling point, comprising: determining a first center point on the first center line, wherein the first center point is a coordinate of each sampling point on the first center line; obtaining a second center point based on the first center point and the corresponding offset of each sampling point; processing the second center point by a spline interpolation method to obtain a second center line.

2. The method of claim 1, wherein, The method comprises: determining a coronary artery rough segmentation image of the first coronary artery according to the three-dimensional blood vessel image; inputting the coronary artery rough segmentation image into a first convolutional neural network to obtain a coronary artery starting point of the first coronary artery; inputting the coronary artery rough segmentation image into a second convolutional neural network to obtain a coronary artery end point of the first coronary artery; inputting the coronary artery rough segmentation image into a third convolutional neural network to obtain a center distance intensity map of the first coronary artery.

3. The method of claim 2, wherein, The determining the coronary rough segmentation image of the first coronary artery according to the three-dimensional blood vessel image comprises: inputting the three-dimensional blood vessel image into a fourth convolutional neural network to obtain the coronary rough segmentation image of the first coronary artery; or, processing the three-dimensional blood vessel image by using an image processing method of Frangi filtering to obtain the coronary rough segmentation image of the first coronary artery.

4. The method of claim 1, wherein, The determining the image cutout of the N sampling points based on the N sampling points and the corresponding direction vectors comprises: determining a target plane of each sampling point, wherein a normal vector of the target plane is a direction vector corresponding to the sampling point on a sub-centerline tangent, and the sampling point is on the target plane; performing cutting on the three-dimensional blood vessel image by using the target plane of each sampling point to obtain the image cutout of the N sampling points.

5. The method of claim 1, wherein, The inputting the high-dimensional feature data with the size of N*H*W*C into the target Transformer network to obtain the offset corresponding to each sampling point comprises: transforming the high-dimensional feature data with the size of N*H*W*C into target data, wherein the target data is feature data of transforming the high-dimensional feature data into the image cutout of the N sampling points; inputting the target data into the target Transformer network to obtain the offset corresponding to each sampling point.

6. An apparatus for extracting a centerline of a coronary vessel, characterized by, The coronary vessel centerline extraction device comprises: a determining module configured to determine a coronary artery starting point, a coronary artery end point and a center distance intensity map of a first coronary artery based on a three-dimensional blood vessel image of the first coronary artery; the determining module is further configured to determine a first centerline of the first coronary artery according to the coronary artery starting point, the coronary artery end point and the center distance intensity map; a sampling module configured to divide the first centerline into a plurality of sub-centerlines, sample each sub-centerline to obtain N sampling points, and determine a direction vector corresponding to each sampling point on a sub-centerline tangent; the determining module is further configured to determine an image cutout of the N sampling points based on the N sampling points and the corresponding direction vectors; the determining module is further configured to input the image cutout of the N sampling points into a target CNN convolutional neural network to obtain high-dimensional feature data with the size of N*H*W*C, wherein H*W represents the size of the image cutout, and C represents the number of features; the determining module is further configured to input the high-dimensional feature data with the size of N*H*W*C into a target Transformer network to obtain the offset corresponding to each sampling point; the determining module is further configured to obtain a second centerline based on the first centerline and the offset corresponding to each sampling point; the determining module is further configured to: determine an image cutout of a first target center point based on a three-dimensional blood vessel image of a second coronary artery, wherein the first target center point is a point on a first centerline of the second coronary artery; input the first target center point into a CNN convolutional neural network and a Transformer network to obtain a predicted offset of the first target center point. determining a labeling offset between the first target center point and a labeling center point, wherein the labeling center point is a point on a labeling center line; updating the CNN convolutional neural network and the Transformer network based on the predicted offset and the labeling offset, to obtain a target CNN convolutional neural network and a target Transformer network; The determination module is further configured to: determine a first center point on the first center line, wherein the first center point is a coordinate of each sampling point on the first center line; obtain a second center point based on the first center point and the offset corresponding to each sampling point; obtain a second center line by processing the second center point through a spline interpolation method.

7. An apparatus for extracting a centerline of a coronary vessel, characterized by, The device comprises a memory, a processor, and a coronary vessel center line extraction program stored on the memory and executable on the processor, and the coronary vessel center line extraction program is configured to implement the steps of the coronary vessel center line extraction method according to any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium stores a coronary vessel center line extraction program, and the coronary vessel center line extraction program, when executed by a processor, implements the steps of the coronary vessel center line extraction method according to any one of claims 1 to 5.

Images