Method, device, storage medium and electronic device for segmenting organ blood vessels

Abstract

The present invention provides an organ blood vessel segmentation method, device, storage medium and electronic device. An organ blood vessel segmentation method includes: acquiring a target organ image; using a pre-trained neural network model to preliminarily segment the organ blood vessels in the target organ image to obtain a first organ blood vessel segmentation result; using each pixel point of the preliminarily segmented organ blood vessels as a foreground pixel seed point to produce a background pixel seed point, and performing graph segmentation based on the foreground pixel seed point and the background pixel seed point, removing the graph segmentation result outside the target organ area, and obtaining a second organ blood vessel segmentation result; merging the first organ blood vessel segmentation result with the second organ blood vessel segmentation result to generate a final organ blood vessel segmentation result. The vascular image reconstructed by the present invention has a strong correlation with the blood vessels in the original image, and can provide accurate needle insertion planning for cryoablation.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an organ blood vessel segmentation method, device, storage medium and electronic equipment. Background Art

[0002] In recent years, the rapid development of minimally invasive surgery has brought hope to patients with poor tolerance. With the advancement of artificial intelligence technology, the demand for automatic ablation needle planning systems in the field of multimodal cold and hot ablation is also increasing.

[0003] In the planning of ablation surgery for organ lesions such as liver tumors, vascular reconstruction is particularly important. Accurate vascular reconstruction can minimize the damage to the patient's important blood vessels and organs caused by cryoablation.

[0004] In related vascular reconstruction technologies, the connectivity of vascular segmentation is poor, the correlation between the reconstructed vascular image and the blood vessels in the original image is not strong, and the continuity of the blood vessels themselves is poor, which cannot provide accurate needle insertion planning for cryoablation. Summary of the Invention

[0005] To address the above issues, the present invention provides an organ vascular segmentation method, device, storage medium and electronic device. The reconstructed vascular image has a strong correlation with the blood vessels in the original image, and the blood vessels have good continuity, which can provide accurate needle insertion planning for cryoablation.

[0006] In a first aspect, an embodiment of the present invention provides a method for segmenting blood vessels of an organ, comprising:

[0007] Acquire target organ images;

[0008] Preliminarily segmenting the organ blood vessels in the target organ image using a pre-trained neural network model to obtain a first organ blood vessel segmentation result;

[0009] Using each pixel point of the initially segmented organ blood vessels as a foreground pixel seed point to generate background pixel seed points, and performing graph cut segmentation based on the foreground pixel seed point and the background pixel seed point, removing the graph cut segmentation results outside the target organ region to obtain a second organ blood vessel segmentation result;

[0010] The first organ and blood vessel segmentation result and the second organ and blood vessel segmentation result are combined to generate a final organ and blood vessel segmentation result.

[0011] In some implementations, the neural network model includes a Res-UNet Plus model formed by replacing the convolutional layer of the U-Net model with a residual neural network module and inserting a jump connection portion into the residual neural network module.

[0012] In some implementations, the organ blood vessel segmentation method further includes: training the neural network model until the loss function drops to a preset threshold, thereby obtaining a trained neural network model;

[0013] The loss function is as follows:

[0014] loss = W1 × loss 背景 +W2×loss 血管

[0015] Among them, loss 背景 Represents the loss function value of the background, loss 血管 represents the loss function value of the organ and blood vessels, W1 represents the loss function weight value of the background, W2 represents the loss function weight value of the organ and blood vessels, and W1 is less than W2.

[0016] In some implementations, the background loss function and / or the organ blood vessel loss function includes a cross entropy loss function.

[0017] In some implementations, the value of W2 is 5 to 30 times the value of W1.

[0018] In some implementations, using the pixels of the initially segmented organ blood vessels as foreground pixel seed points to generate background pixel seed points includes:

[0019] Using each pixel point of the initially segmented organ blood vessels as a foreground pixel seed point, the foreground pixel seed point is dilated;

[0020] The target organ segmentation result obtained based on the target organ image is subtracted from the dilation result to obtain background pixel seed points.

[0021] In some implementations, the graph cut segmentation includes three-dimensional graph cut segmentation, and performing the graph cut segmentation based on the foreground pixel seed point and the background pixel seed point includes:

[0022] Marking each pixel of the target organ image as a foreground pixel seed point and a background pixel seed point;

[0023] Abstracting the target organ image into a graph on a three-dimensional level, connecting each pixel as a node in the graph to form an edge, and the weight of the edge between two adjacent nodes is the difference between the pixel values ​​of the two adjacent nodes;

[0024] Determine a maximum weight value in the graph, and subtract the weight of each edge from the maximum weight value to obtain an updated weight value of each edge;

[0025] Connecting all nodes in the graph to source nodes and sink nodes to obtain multiple edges, and adding weights to the multiple edges according to the labels of foreground pixel seed points and background pixel seed points;

[0026] determining a non-saturated shortest path from the source node to the sink node and adjusting the shortest path to a saturated path until no non-saturated path exists, thereby obtaining a residual graph;

[0027] Segmenting the saturated graph arcs in the residual graph to obtain a graph cut segmentation result;

[0028] The area outside the target organ in the graph cut segmentation result is removed to obtain the blood vessel segmentation result of the second organ.

[0029] In a second aspect, an embodiment of the present invention provides an organ and blood vessel segmentation device, comprising:

[0030] An image acquisition module, used to acquire images of target organs;

[0031] a first segmentation module, configured to preliminarily segment the organ blood vessels in the target organ image using a pre-trained neural network model to obtain a first organ blood vessel segmentation result;

[0032] a second segmentation module, configured to generate background pixel seed points using each pixel point of the initially segmented organ blood vessels as a foreground pixel seed point, and perform graph segmentation based on the foreground pixel seed point and the background pixel seed point, removing the graph segmentation results outside the target organ region to obtain a second organ blood vessel segmentation result;

[0033] The result generating module is configured to merge the first organ and blood vessel segmentation result with the second organ and blood vessel segmentation result to generate a final organ and blood vessel segmentation result.

[0034] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by one or more processors, the method described in the first aspect is implemented.

[0035] In a fourth aspect, an embodiment of the present invention provides an electronic device comprising a memory and one or more processors, wherein the memory stores a computer program, and when the computer program is executed by the one or more processors, the method described in the first aspect is implemented.

[0036] Compared with the prior art, one or more embodiments of the present invention can bring at least the following beneficial effects:

[0037] First, the organ blood vessels are preliminarily segmented using a neural network model, and then graph segmentation is performed using background pixel seed points and foreground pixel seed points to strengthen the correlation between blood vessels in the original image and improve the connectivity of blood vessel segmentation. After the two segmentation results are merged, the reconstructed blood vessel image has a strong correlation with the blood vessels in the original image, which can provide accurate needle insertion planning for cryoablation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is a flow chart of a method for segmenting blood vessels of an organ provided by an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of a neural network model provided by an embodiment of the present invention;

[0041] Figure 3 Schematic diagram of a residual neural network module provided by an embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of a three-dimensional reconstruction of a blood vessel segmentation result of a first organ provided by an embodiment of the present invention;

[0043] Figure 5 is a two-dimensional image provided by an embodiment of the present invention;

[0044] Figure 6 yes Figure 5 A graph abstractly constructed from a two-dimensional image;

[0045] Figure 7 This is a graph obtained by calculating using a cost calculation formula provided by an embodiment of the present invention;

[0046] Figure 8 is a schematic diagram of the remaining diagram provided by an embodiment of the present invention;

[0047] Figure 9 Schematic diagram of foreground and background pixel segmentation provided by an embodiment of the present invention;

[0048] Figure 10 is the graph cut segmentation result provided by the embodiment of the present invention;

[0049] Figure 11a This is the final organ and blood vessel segmentation result provided by the embodiment of the present invention;

[0050] Figure 11b is the first organ blood vessel segmentation result provided by an embodiment of the present invention;

[0051] Figure 12 Schematic diagram of an organ and blood vessel segmentation device provided by an embodiment of the present invention;

[0052] Figure 13 This is a schematic diagram of a liver blood vessel segmentation process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0054] Example 1

[0055] Figure 1 A flow chart of an organ blood vessel segmentation method is shown in FIG. Figure 1 As shown, the organ blood vessel segmentation method provided in this embodiment includes steps S101 to S104:

[0056] Step S101: Acquire a target organ image.

[0057] In some implementations, the target organ may be a liver, the target organ image may be a liver enhanced CT image, and the organ blood vessels include the portal vein of the liver.

[0058] Step S102: Preliminarily segment the organ blood vessels in the target organ image using the pre-trained neural network model to obtain a first organ blood vessel segmentation result.

[0059] In some implementations, the neural network model includes replacing the convolutional layer of the U-Net model with a residual neural network module and inserting the skip connection part into the residual neural network module. Figure 2 The Res-UNet Plus model shown in the figure takes organ images as input and outputs organ vascular segmentation results.

[0060] The Res-UNet Plus model 120 includes a downsampling section 1201, a skip connection section 1202, and an upsampling section 1203. All three sections incorporate residual neural network modules to increase model complexity and more accurately calculate segmentation results. The original organ image 110 is input into the Res-UNet Plus model 120, resulting in an image output 140 (predicted label). The downsampling section 1201 includes four pairs of residual neural network modules and pooling layers connected to the residual neural network modules, as well as a residual neural network module connected to the organ image 110. The skip connection section 1201 includes five residual neural network modules, one connected to the residual neural network module connected to the organ image 110 and the four residual neural network modules connected to the pooling layers in the downsampling section 1201. The upsampling section 1203 includes four pairs of residual neural network modules and deconvolution layers, one residual neural network module, and a convolution layer. This model structure improves the degradation and gradient vanishing problems of the U-shaped neural network model, enhances the Dice accuracy of the Res-UNet Plus model and the segmentation accuracy of organs and blood vessels. At the same time, the residual neural network module is added to the jump connection of the U-shaped neural network, which deepens the memory of the segmented image for shallow features and further improves the segmentation accuracy of organs and blood vessels.

[0061] In some implementations, the residual neural network module can be implemented as follows: Figure 3 The BasicBlock structure shown uses two 3*3 convolutional layers, but is not limited to the BasicBlock structure.

[0062] In some implementations, the organ blood vessel segmentation method of this embodiment further includes:

[0063] Step S100: training the neural network model until the loss function drops to a preset threshold, thereby obtaining a trained neural network model;

[0064] In some implementations, the loss function is as follows:

[0065] loss = W1 × loss 背景 +W2×loss 血管

[0066] Among them, loss 背景 Represents the loss function value of the background, loss 血管 represents the loss function value of the organ and blood vessels, W1 represents the loss function weight value of the background, W2 represents the loss function weight value of the organ and blood vessels, and W1 is less than W2.

[0067] In some implementations, the background loss function and / or the organ vessel loss function includes a cross-entropy loss function.

[0068]

[0069] Among them, loss CE Represents the cross entropy loss function, n represents the total number of samples, x represents the sample, set A(x) is the true label, and set B(x) is the predicted label obtained by segmentation.

[0070] Because the proportion of organ blood vessels in CT image slices (organ images) containing them is too small relative to the background, we first perform one-hot encoding on the background and organ blood vessels in the training labels (input images, i.e., background + organ blood vessel images). Then, based on the training labels corresponding to the one-hot encoding, we adjust the loss function weights for the background and the organ blood vessels to increase the loss weight of the organ blood vessels and reduce the weight of the background.

[0071] In some implementations, the value of W2 can be 5 to 30 times the value of W1, for example, 20 times, to maximize the prominence of organ and blood vessels in the image. Furthermore, to prevent excessively large loss functions in the neural network model, which can lead to unstable and continuously fluctuating model training, the value of W1 can range from 0.05 to 0.5, for example, 0.1. Adjusting this weight relationship can effectively improve model accuracy and training efficiency.

[0072] When using the trained Res-UNet Plus model to extract blood vessels from the liver region (organ image) of the original enhanced CT image, three consecutive CT images are obtained in the order of the CT image sequence and input into the Res-UNetPlus model for segmentation. The segmentation results of the middle image of the three consecutive CT images are spliced ​​to obtain the first organ blood vessel segmentation result, and the first organ blood vessel segmentation result is 3D reconstructed to obtain Figure 4 The reconstruction result is shown in the figure. Because the Res-UNet Plus model cannot accurately calculate all vascular sections in two dimensions, the generated organ and vessel segmentation results contain many discontinuous breakpoints. Therefore, to truly restore the organs and vessels, it is necessary to use the first organ and vessel segmentation result as the foreground pixel seed points for the subsequent graph segmentation and generate background pixel seed points for further graph segmentation. This leads to the second organ and vessel segmentation result. By fusing the two organ and vessel segmentation results, the organs and vessels can be accurately segmented and their morphology can be well restored.

[0073] Step S103: Use the pixel points of the initially segmented organ blood vessels as foreground pixel seed points to generate background pixel seed points, and perform graph segmentation based on the foreground pixel seed points and the background pixel seed points. Remove the graph segmentation results outside the target organ area to obtain a second organ blood vessel segmentation result.

[0074] In some implementations, the above step S103 of using the pixel points of the initially segmented organ blood vessels as foreground pixel seed points to generate background pixel seed points may further include:

[0075] Step S103a: using the pixel points of the initially segmented organ blood vessels as foreground pixel seed points, and performing dilation processing on the foreground pixel seed points.

[0076] In some cases, the foreground pixel seed point can be expanded by a preset number of pixels to achieve the desired expansion effect and truly represent the organ vascular morphology. For example, the foreground pixel seed point can be expanded by 15 pixels.

[0077] Step S103b: Subtract the dilation result from the target organ segmentation result obtained based on the target organ image to obtain background pixel seed points. The target organ segmentation result can be obtained by performing organ segmentation on the target organ image using a pre-trained organ segmentation model.

[0078] The background pixel seed points and foreground pixel seed points obtained in this way can be used to calculate the graph segmentation result.

[0079] In some implementations, the graph cut segmentation of this embodiment includes three-dimensional graph cut segmentation. The graph cut segmentation based on the foreground pixel seed points and the background pixel seed points in step S103 may further include:

[0080] Step S103c: Mark each pixel of the target organ image as a foreground pixel seed point and a background pixel seed point.

[0081] It should be understood that the target organ image here is the CT image after adjusting the window width and window level to highlight the organ blood vessels in the image. Based on the foreground pixel seed points and background pixel seed points determined in the previous step, each pixel in the target organ image is marked as foreground or background. This marking does not change the original pixel value; it only adds a label for subsequent segmentation.

[0082] Step S103d: abstract the target organ image into a graph on a three-dimensional level, connect each pixel as a node in the graph in pairs to form an edge, and the weight of the edge between two adjacent nodes is the difference between the pixel values ​​of the two adjacent nodes.

[0083] Step S103e: Determine the maximum weight value in the graph, and subtract the weight of each edge from the maximum weight value to obtain the updated weight value of each edge.

[0084] Step S103f: Connect all nodes in the graph to the source node and the sink node respectively to obtain multiple edges, and add weights to the multiple edges according to the labels of the foreground pixel seed points and the background pixel seed points.

[0085] Step S103g: Determine the non-saturated shortest path in the path from the source node to the sink node and adjust it to a saturated path until no non-saturated path exists, thereby obtaining a residual graph.

[0086] Step S103h: Segment the saturated graph arcs in the remaining graph to obtain a graph cut segmentation result.

[0087] Step S103i: remove the region other than the target organ in the graph segmentation result to obtain a second organ blood vessel segmentation result.

[0088] It should be understood that the target organ image for the three-dimensional graph segmentation is a three-dimensional image, and the graph abstracted in the three-dimensional graph segmentation is a three-dimensional graph.

[0089] In this embodiment, for the sake of illustration, a two-dimensional image is used for illustration:

[0090] First, Figure 5 For the pixels of a two-dimensional image shown in FIG, the numerical values ​​in the figure represent the pixel values ​​corresponding to each pixel. According to the background pixel seed points and foreground pixel seed points determined previously, each pixel in this image is marked as foreground and background.

[0091] Secondly, Figure 5 The two-dimensional image shown is abstracted as Figure 6 In the graph shown, each pixel is connected to form an edge as a node (black dot) in the abstract graph. The weight of the edge between two adjacent nodes is the difference between the pixel values ​​of the two adjacent nodes. For example, Figure 5 The pixel value in the upper left corner is 10, and the pixel value to the right is 9, then Figure 6 In the graph constructed in , the weight of the edge connecting these two pixels is 10-9=1, and so on.

[0092] Next, we use the cost calculation formula to determine the maximum weight in the graph. We subtract the weight of each edge from the maximum weight to get the updated weight of each edge. We connect all nodes in the graph to source nodes and sink nodes to obtain multiple edges, and we add weights to these edges based on the labels of the foreground and background pixel seed points.

[0093] The fee calculation formula is as follows:

[0094]

[0095] Among them, x k represents the kth boundary element (or edge), x i represents the i-th boundary element, Represents the cost (or weight) of the boundary elements, that is, finds the maximum value of the weight of each boundary element in the graph.

[0096] by Figure 6 For example, the maximum weight in this abstract graph is 9. Subtract the weight of each edge from 9 to update the weight of each edge, and we get Figure 7 , the weight of the edge with the original weight of 9 is updated to 0. On the basis of updating the weights, all nodes in the graph are further connected to the source node S and the sink node T respectively, thereby obtaining multiple edges, and adding weights to the multiple edges connected to the source node S and the sink node T. In practical applications, weight values ​​can be added according to the possibility that the pixels connected to the source node S belong to the organs and blood vessels, and the weight values ​​added to each edge may be the same or different. For example, when it is determined that a certain node belongs to the background, the weight of the edge connected to the sink node T can be assigned to 22 or other values ​​that are significantly larger than 22. Figure 6 When a node is determined to be a foreground node, the weight of the edge connecting it with the sink node T can be assigned to 0, and the weight of the edge connecting it with the source node S can be assigned to 22 or other values ​​significantly greater than Figure 6 When a node is very likely to belong to the foreground, the weight of the edge connecting it to the sink node T can be assigned a value less than 22, such as 20; conversely, when a node is very unlikely to belong to the foreground, the weight of the edge connecting it to the sink node T can be assigned a value less than 22, such as 5. The situation of belonging to the background is similar and will not be given as an example here. In other words, the corresponding edge can be assigned a value based on the probability of the node belonging to the background or foreground, so that accurate segmentation results can be obtained in subsequent segmentation, effectively avoiding the omission of pixels belonging to the foreground.

[0097] Thirdly, the non-saturated shortest path is determined from the path from the source node to the sink node and adjusted to a saturated path until no non-saturated path exists, and the residual graph is obtained.

[0098] Find the non-saturated shortest path from the source node S to the sink node T, that is, the path with the least number of edges and no edges with weight 0, and adjust the non-saturated shortest path to a saturated path, that is, the path with weight 0. The adjustment method includes: subtracting the minimum weight of all edges of the path from the weight of each edge of the non-saturated shortest path, stopping when the weight of one of the edges of the path is 0, and then continue to determine the non-saturated shortest path from the source node to the sink node and adjust it to a saturated path, and so on, until there are no non-saturated paths, that is, each path has an edge with weight 0, thereby calculating the residual graph G. f .

[0099] by Figure 7For example, the non-saturated shortest path from source node S to sink node T is a path with weights of 8, 8, 2, and 8 in the graph. This path does not have an edge with weight 0, and it passes through 4 edges, which is the path with the fewest edges from source node S to sink node T. The minimum weight of the 4 edges in this path is 2. Subtract 2 from 8, 8, 2, and 8, respectively, to get 6, 6, 0, and 6, to adjust the path to a saturated path. Next, search for the non-saturated shortest path in the remaining paths from node S to sink node T in the current graph until there are no more non-saturated paths. At this time, the remaining graph is obtained, as shown in Figure 8 shown.

[0100] Finally, the remaining graph G is segmented f The saturated graph arcs (i.e., edges with a weight of 0) in the graph cut segmentation result image are obtained, in which the pixels connected to the source node S are foreground pixels, and the pixels connected to the sink node T are background pixels. For example, the segmentation Figure 8 For the edge with weight 0, we get Figure 9 Graph cut segmentation result. Because some regions may be over-segmented after the graph cut segmentation calculation, this embodiment also removes redundant segmented regions other than the target organ in the graph cut segmentation result, such as the inferior vena cava, aorta, part of the kidney, and ribs, and finds and removes small connected domains to obtain the second organ vascular segmentation result.

[0101] In one example, if Figure 10 In the graph cut segmentation result shown, redundant segmentations can be seen outside the liver region and should be removed.

[0102] Step S104: Merge the first organ and blood vessel segmentation result with the second organ and blood vessel segmentation result to generate a final organ and blood vessel segmentation result.

[0103] The blood vessel segmentation results of the first organ and the blood vessel segmentation results of the second organ are merged, that is, the union is calculated, and small connected domains are found and removed to obtain the final segmentation result.

[0104] Compared to Figure 11b The blood vessel segmentation result of the first organ is shown as Figure 11a In the final organ and blood vessel segmentation result shown, the organ and blood vessels are clear and continuous, and some discontinuous blood vessel areas are repaired in the second organ and blood vessel segmentation result, with a high degree of restoration.

[0105] The method in this embodiment uses 3D graph segmentation to strengthen the correlation between blood vessels in the original image, improving the connectivity of the segmented vessels and providing an accurate basis for subsequent needle planning for lesion ablation. Furthermore, compared to traditional 3D neural networks, such as V-Net, this method achieves superior results using less video memory and computing resources.

[0106] In some implementations, the method may further include:

[0107] Step S105: classify the organ blood vessels based on the final organ blood vessel segmentation result.

[0108] For example, the liver has two types of blood vessels: the hepatic vein and the portal vein. Based on the final organ blood vessel segmentation results, the classification of organ blood vessels can include:

[0109] Step S105a, extracting any connected domain from the final organ and blood vessel segmentation result;

[0110] Step S105b: determining whether the extracted connected domain meets a preset condition; wherein the preset condition includes that the number of pixels distributed outside the liver region is not less than a preset proportion, and the preset proportion may be, for example, 35%.

[0111] Step S105c: determining that the liver blood vessel that meets the preset conditions is the portal vein, and determining that the liver blood vessel that does not meet the preset conditions is the hepatic vein.

[0112] Based on the above classification process, the segmented liver vessels can be further accurately classified based on the organ vessel segmentation results, which is conducive to subsequent more fine-grained ablation planning for different types of liver vessels and accurate and efficient removal of lesions.

[0113] Figure 13 A schematic diagram of the liver vessel segmentation process using the present method is shown. In practical applications, when performing liver vessel segmentation, an enhanced CT sequence image of the liver image is first input. On the one hand, the liver is segmented using a pre-trained liver segmentation model. On the other hand, the liver vessels are preliminarily segmented based on the enhanced CT sequence image using a pre-trained Res-UNet Plus model. The preliminarily segmented liver vessels are used as foreground seed points. After dilation processing, they are subtracted from the liver segmented by the liver segmentation model to obtain background seed points. 3D graph segmentation processing is then performed based on the foreground pixel seed points and the background pixel seed points. The 3D graph segmentation results outside the target organ area are removed, and redundant segmentations outside the liver are removed. The obtained segmentation result is then unioned with the preliminarily segmented liver vessels. Small connected domains are then removed to obtain the final segmentation result.

[0114] In order to compare and verify the segmentation effect of this method, an RTX3090 graphics card is used to compare the three indicators of the V-Net segmentation method and this method, as shown in Table 1.

[0115] Table 1 Comparison between traditional 3D neural network liver vessel segmentation and this method

[0116] Segmentation method Calculation time Video memory usage CUDA usage Segmentation sensitivity V-Net 134S 17.3GB 92.537％ 0.4571 This method 153S 12.4GB 39.216％ 0.6621

[0117] The above sensitivity index is used to determine the proportion of correctly predicted vascular areas in the annotated vascular areas. The sensitivity calculation formula is:

[0118] TPR=TP / (TP+FN)

[0119] Among them, TP stands for true positive, which refers to the annotation range of the correct predicted real organ blood vessels; FN stands for false negative, which refers to the annotation range outside the real organ blood vessels that are incorrectly predicted.

[0120] It can be seen that this method has obvious advantages over the V-Net segmentation method.

[0121] Example 2

[0122] Figure 12 A schematic diagram of an organ blood vessel segmentation device is shown in FIG. Figure 12 As shown, this embodiment provides an organ and blood vessel segmentation device, comprising:

[0123] An image acquisition module 201 is used to acquire an image of a target organ;

[0124] A first segmentation module 202 is configured to preliminarily segment the organ blood vessels in the target organ image using a pre-trained neural network model to obtain a first organ blood vessel segmentation result;

[0125] The second segmentation module 203 is configured to generate background pixel seed points using the pixels of the initially segmented organ blood vessels as foreground pixel seed points, perform graph segmentation based on the foreground pixel seed points and the background pixel seed points, remove the graph segmentation results outside the target organ region, and obtain a second organ blood vessel segmentation result;

[0126] The result generating module 204 is configured to merge the first organ and blood vessel segmentation result with the second organ and blood vessel segmentation result to generate a final organ and blood vessel segmentation result.

[0127] In some implementations, the target organ may be a liver, the target organ image may be a liver enhanced CT image, and the organ blood vessels include the portal vein of the liver.

[0128] In some implementations, the neural network model includes replacing the convolutional layer of the U-Net model with a residual neural network module and inserting the skip connection part into the residual neural network module. Figure 2 The Res-UNet Plus model shown in the figure takes organ images as input and outputs organ vascular segmentation results.

[0129] In some implementations, the residual neural network module may adopt a BasicBlock structure and use two 3*3 convolutional layers, but is not limited to the BasicBlock structure.

[0130] In some implementations, the apparatus further includes:

[0131] The model training module is used to train the neural network model until the loss function drops to a preset threshold, thereby obtaining a trained neural network model;

[0132] In some implementations, the loss function is as follows:

[0133] loss = W1 × loss 背景 +W2×loss 血管

[0134] Among them, loss 背景 Represents the loss function value of the background, loss 血管 represents the loss function value of the organ and blood vessels, W1 represents the loss function weight value of the background, W2 represents the loss function weight value of the organ and blood vessels, and W1 is less than W2.

[0135] In some implementations, the background loss function and / or the organ vessel loss function includes a cross-entropy loss function.

[0136] In some implementations, the value of W2 can be 5 to 30 times the value of W1, for example, 20 times, to maximize the prominence of organ and blood vessels in the image. Furthermore, to prevent excessively large loss functions in the neural network model, which can lead to unstable and continuously fluctuating model training, the value of W1 can range from 0.05 to 0.5, for example, 0.1. Adjusting this weight relationship can effectively improve model accuracy and training efficiency.

[0137] In some implementations, using the pixels of the initially segmented organ blood vessels as foreground pixel seed points to generate background pixel seed points may further include:

[0138] Each pixel point of the initially segmented organ blood vessels is used as the foreground pixel seed point, and the foreground pixel seed point is dilated; the result of the dilation processing is subtracted from the target organ image to obtain the background pixel seed point.

[0139] In some implementations, the graph cut segmentation of this embodiment includes three-dimensional graph cut segmentation, and the graph cut segmentation based on foreground pixel seed points and background pixel seed points may further include:

[0140] Mark each pixel of the target organ image as a foreground pixel seed point and a background pixel seed point; abstract the target organ image into a graph on a three-dimensional level, and connect each pixel as a node in the graph to form an edge, and the weight of the edge between two adjacent nodes is the difference between the pixel values ​​of the two adjacent nodes; determine the maximum weight in the graph, subtract the weight of each edge from the maximum weight to obtain the updated weight value of each edge; connect all nodes in the graph to the source node and the sink node to obtain multiple edges, and add weights to the multiple edges according to the markings of the foreground pixel seed point and the background pixel seed point; determine the non-saturated shortest path in the path from the source node to the sink node and adjust it to a saturated path until no non-saturated path exists, thereby obtaining a residual graph; segment the saturated graph arc in the residual graph to obtain a graph cut segmentation result; remove the area outside the target organ in the graph cut segmentation result to obtain a second organ blood vessel segmentation result.

[0141] In some implementations, the apparatus may further include:

[0142] The classification module classifies the organ and blood vessels based on the final organ and blood vessel segmentation results.

[0143] Taking the liver as an example, the categories of liver blood vessels include hepatic vein and portal vein; classifying the organ blood vessels based on the final organ blood vessel segmentation result may include: extracting any connected domain in the final organ blood vessel segmentation result; determining whether the extracted connected domain meets a preset condition; wherein the preset condition includes that the number of pixels distributed outside the liver area is not less than a preset proportion; determining that the liver blood vessel that meets the preset condition is the portal vein, and determining that the liver blood vessel that does not meet the preset adjustment is the hepatic vein, wherein the preset proportion may be, for example, 35%.

[0144] It should be understood that the target organ image for the three-dimensional graph segmentation is a three-dimensional image, and the graph abstracted in the three-dimensional graph segmentation is a three-dimensional graph.

[0145] The device of this embodiment has all the beneficial effects of the method provided in the first embodiment, which will not be described in detail in this embodiment.

[0146] Those skilled in the art will appreciate that the above modules or steps can be implemented using a general-purpose computing device, centralized on a single computing device, or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0147] Example 3

[0148] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by one or more processors, the organ blood vessel segmentation method of the first embodiment is implemented.

[0149] In this embodiment, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The content of the method is detailed in Example 1 and will not be repeated here.

[0150] Example 4

[0151] This embodiment provides an electronic device including a memory and one or more processors. The memory stores a computer program. When the computer program is executed by the one or more processors, the organ blood vessel segmentation method of the first embodiment is implemented.

[0152] In this embodiment, the processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components to implement the methods in the above embodiments. The methods implemented when the computer program running on the processor is executed can refer to the specific embodiments of the methods provided in the aforementioned embodiments of the present invention, and will not be repeated here.

[0153] In the several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed system and method can also be implemented in other ways. The above-described system and method embodiments are merely illustrative.

[0154] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0155] Although the embodiments disclosed herein are as described above, the contents described herein are merely embodiments for facilitating understanding of the present invention and are not intended to limit the present invention. Any person skilled in the art may make any modifications and variations in the form and details of the embodiments without departing from the spirit and scope of the present invention. However, the scope of patent protection of the present invention shall remain subject to the scope defined by the appended claims.

Claims

1. A method for segmenting blood vessels of an organ, characterized in that: include: Acquire target organ images; Preliminarily segmenting the organ blood vessels in the target organ image using a pre-trained neural network model to obtain a first organ blood vessel segmentation result; Using each pixel point of the initially segmented organ blood vessels as a foreground pixel seed point, the foreground pixel seed point is dilated by a preset number of pixels to truly represent the organ blood vessel morphology; subtracting the dilation result from the target organ segmentation result obtained based on the target organ image to obtain background pixel seed points, and performing graph cut segmentation based on the foreground pixel seed points and the background pixel seed points, removing the graph cut segmentation result outside the target organ region to obtain a second organ blood vessel segmentation result; The first organ and blood vessel segmentation result and the second organ and blood vessel segmentation result are combined to find and remove small connected domains to generate a final organ and blood vessel segmentation result.

2. The organ blood vessel segmentation method according to claim 1, characterized in that: The neural network model includes a Res-UNet Plus model formed by replacing the convolutional layer of the U-Net model with a residual neural network module and inserting the jump connection part into the residual neural network module.

3. The organ blood vessel segmentation method according to claim 2, characterized in that: It also includes: training the neural network model until the loss function drops to a preset threshold, thereby obtaining a trained neural network model; The loss function is as follows: loss = W 1× loss 背景 + W 2× loss 血管 in, loss 背景 Represents the loss function value of the background, loss 血管 Represents the loss function value of the organ blood vessels, W 1 represents the weight value of the loss function of the background, W 2 represents the weight value of the loss function of the organ blood vessels, and W 1 is less than W 2.

4. The organ blood vessel segmentation method according to claim 3, characterized in that: The background loss function and / or the organ blood vessel loss function include a cross entropy loss function.

5. The organ blood vessel segmentation method according to claim 3, characterized in that: W The value of 2 is W 5 to 30 times the value of 1.

6. The organ blood vessel segmentation method according to claim 1, characterized in that: The graph cut segmentation includes three-dimensional graph cut segmentation, and the graph cut segmentation based on the foreground pixel seed point and the background pixel seed point includes: Marking each pixel of the target organ image as a foreground pixel seed point and a background pixel seed point; Abstracting the target organ image into a graph on a three-dimensional level, connecting each pixel as a node in the graph to form an edge, and the weight of the edge between two adjacent nodes is the difference between the pixel values ​​of the two adjacent nodes; Determine a maximum weight value in the graph, and subtract the weight of each edge from the maximum weight value to obtain an updated weight value of each edge; Connecting all nodes in the graph to source nodes and sink nodes to obtain multiple edges, and adding weights to the multiple edges according to the labels of foreground pixel seed points and background pixel seed points; determining a non-saturated shortest path from the source node to the sink node and adjusting the shortest path to a saturated path until no non-saturated path exists, thereby obtaining a residual graph; Segmenting the saturated graph arcs in the residual graph to obtain a graph cut segmentation result; The area outside the target organ in the graph cut segmentation result is removed to obtain the blood vessel segmentation result of the second organ.

7. An organ blood vessel segmentation device, characterized in that: include: An image acquisition module, used to acquire images of target organs; a first segmentation module, configured to preliminarily segment the organ blood vessels in the target organ image using a pre-trained neural network model to obtain a first organ blood vessel segmentation result; a second segmentation module, configured to use each pixel point of the initially segmented organ blood vessels as a foreground pixel seed point, dilate the foreground pixel seed point by a preset number of pixels to truly represent the organ blood vessel morphology; subtract the dilation result from the target organ segmentation result obtained based on the target organ image to obtain background pixel seed points; perform graph segmentation based on the foreground pixel seed points and the background pixel seed points, remove the graph segmentation result outside the target organ region, and obtain a second organ blood vessel segmentation result; The result generation module is configured to obtain a union of the first organ and blood vessel segmentation result and the second organ and blood vessel segmentation result, find and remove small connected domains, and generate a final organ and blood vessel segmentation result.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: The method comprises a memory and one or more processors, wherein a computer program is stored in the memory, and when the computer program is executed by the one or more processors, the method according to any one of claims 1 to 6 is implemented.

Images