Weak supervision blood vessel segmentation method based on tree structure key points

Abstract

The invention discloses a weak supervision blood vessel segmentation method based on tree structure key points, and relates to the technical field of tree structure segmentation of similar blood vessels in medical images. Firstly, a loss function based on key points (Key-points) is proposed, and various X loss functions are deformed by using a Key-points graph and are suitable for a weak supervised learning scene. The deformed loss functions enhance the shape and topological structure preserving capability of the neural network in blood vessel segmentation. In addition, a method for correcting weak annotations through iterative training is established, and higher segmentation accuracy is brought. According to the method, shape preserving and topology of the target can be realized in a weak supervised learning scene, a more accurate, more effective and more robust segmentation result can be obtained, the workload of manual annotation can be greatly reduced, and the method has a relatively high clinical application value.

Description

Technical Field

[0001] This invention belongs to the field of medical image blood vessel segmentation technology, specifically relating to a weakly supervised blood vessel segmentation method based on tree-structured key points. Background Technology

[0002] Accurate automatic segmentation of medical images is a crucial step in performing accurate quantitative analysis and visualizing tubular and tree-like structures such as bronchi, nerve fibers, and blood vessels, aiding physicians in lesion localization and diagnosis. However, manual annotation of medical tree structures requires a high level of expertise, making it costly and difficult to collect. It is also a time-consuming and labor-intensive process. To overcome these challenges, researchers have proposed numerous techniques, including fully supervised, weakly supervised, and unsupervised methods. However, accurate segmentation of tree structures still faces three major challenges: (1) low contrast between the target and the background; (2) complex geometry and topology of the target; and (3) delicate, localized structures susceptible to various noises. These problems often lead to structural errors in the segmentation results, such as broken blood vessels and missing branches.

[0003] In recent years, many studies have proposed incorporating geometric information to better guide models in perceiving the unique features of tree-like structures, focusing on maintaining the integrity of the global morphology and improving segmentation accuracy. Topological and shape information represent two key categories within geometric information. The first category is topological deep learning. These methods compute topological features (such as Betti numbers and persistent homology groups) or deform segmentation results based on algebraic topology theory to increase constraints during training. However, these methods emphasize the global characteristics of the topology, and errors in just a few key voxels can easily alter the global topological properties. The second category aims to integrate shape structures, such as skeletons, centerlines, and graph representations, to repair segmentation results and enhance vascular connectivity, reducing vascular breakage. However, shape-based models are limited by inaccuracies and offsets, continuously impairing the effectiveness of constraints. Furthermore, none of these methods can simultaneously preserve both local key shapes and global topological structure. In fact, according to homology theory, local shape structures contain redundant information to distinguish topological changes. Only a few key shape structures are needed to support downstream tasks. For example, the focus when drawing a rectangle is on identifying the four vertices rather than all points within the rectangle. For tree-like structures, the key point is that they are important geometric structures that characterize the topological and shape information of the target, such as... Figure 4 However, existing methods lack specific designs to preserve these key geometries. Summary of the Invention

[0004] Objective: To address the problem that existing methods lack specific design to preserve the key geometric structure of the target object, this invention proposes a novel keypoint-based segmentation method to maintain the shape and topology of the target object. This invention can more effectively preserve shape and topology in weakly supervised learning scenarios, resulting in more accurate, efficient, and robust segmentation results.

[0005] Technical solution: This invention proposes a novel weakly supervised vessel segmentation method based on key points, comprising the following two aspects:

[0006] (a) A loss function based on key-points is proposed, which uses key-point graphs to transform various X loss functions, making them suitable for weakly supervised learning scenarios.

[0007] (b) A key-points graph is proposed to correct incomplete annotations and is used in the iterative training process to further improve the segmentation performance under weak supervision.

[0008] Furthermore, in step (a), under the weakly supervised learning setting, the following loss functions are proposed: KP-Warp partial cross-entropy loss and KP-Warp Mumford Shah loss, as follows:

[0009] KP-Warp Partial Cross-Entropy Loss: For tasks such as incomplete vessel annotation, based on the prior knowledge that the neighborhood of a vessel's keypoint is more likely to be an unannotated vessel, the KP-Warp Partial Cross-Entropy Loss (KP-pCE) is proposed. kp-pCE This loss only relaxes the background class constraint within a local region, as shown below:

[0010]

[0011] in It is the annotation value of the i-th voxel in the 0th channel of the one-hot encoded data. It is the predicted probability value of the i-th voxel in the 0th channel of the one-hot encoding. It is the annotation value of the i-th voxel in the c-th channel of the one-hot encoded data. ω is the predicted probability value of the i-th voxel in the c-th channel of one-hot encoding. bg It is the background pixel set, ω kp It is the set of non-zero pixels in the keypoint map, ω g It is a set of labeled pixels, |ω g | represents the size of the labeled pixel set.

[0012] KP-Warp Mumford-Shah Loss: Active contour models are unsupervised segmentation methods that evolve contours to segment target objects in an image by minimizing an energy functional. Level set methods implicitly represent contours using level set functions. The Mumford-Shah (MS) functional is one well-known model. Inspired by this, the KP-Warp Mumford-Shah (KP-MS) loss L is proposed. kp-MS As shown below:

[0013]

[0014] in

[0015]

[0016] α is the weighting parameter, I i It is the intensity value of pixel i. It is the keypoint intensity value of the i-th voxel in the c-th channel of the one-hot encoded data. Let be the predicted probability value of the i-th voxel in the c-th channel of the one-hot encoding, where C and N are the total number of predicted classes and the total number of voxels, respectively. By minimizing the KP-MS loss, the intensity variance within the keypoint structural region is reduced. This deformation loss improves the topology-preserving ability of the MS loss under weak supervision.

[0017] Furthermore, in step (b), under the weakly supervised learning setting, a keypoint map is proposed to correct incomplete annotations, which are then used in the iterative training process, as follows:

[0018] Iterative Correction of Weak Labels Using KP-Map: Since labels are incomplete, erroneous information can hinder neural networks from achieving optimal performance during training based on weakly supervised loss, often leading to oversegmentation. Therefore, iteratively correcting weak labels based on segmentation results is crucial for improving model performance. This can be effectively implemented using keypoint maps, with each iteration consisting of three steps:

[0019] 1) Train the segmentation network with incomplete annotations under weak supervision loss to obtain the prediction mask.

[0020] 2) Extract key point maps from weak annotations and binarize them to generate key point masks.

[0021] 3) Derive the revised annotation using the following equation.

[0022]

[0023] in These represent the masks obtained from the keypoints and the probability map, respectively.

[0024] The beneficial effects of this invention are: This invention can achieve shape and topology preservation of the target in weakly supervised learning scenarios, resulting in more accurate, more efficient and more robust blood vessel segmentation results. It can also greatly reduce the workload of manual annotation and has high clinical application value. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the overall process of the keypoint-based weakly supervised learning segmentation method proposed in this invention. First, the method is trained using weakly labeled data and the partially supervised loss function proposed in this invention. Then, the weak labels are corrected using keypoint maps to form a dataset with more complete labels. The training is then iteratively repeated to gradually improve segmentation performance.

[0026] Figure 2 This diagram illustrates how the segmentation network, guided by the partially supervised loss function proposed in this invention, continuously performs self-correction in local regions of key points during the training process.

[0027] Figure 3 This is a comparison chart of the segmentation results of various loss functions on the hepatic vein dataset under weakly supervised learning.

[0028] Figure 4 This is a schematic diagram of key structural points on the vascular skeleton. Detailed Implementation

[0029] The objectives, technical solutions, and advantages of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] To address the problem that existing methods lack specific design to preserve the key geometric structure of the target object, this invention proposes a novel keypoint-based segmentation method that maintains the shape and topology of the target object. This invention can more effectively preserve shape and topology in weakly supervised learning scenarios, resulting in more accurate, efficient, and robust segmentation results. The overall flowchart of this invention is shown below. Figure 1 As shown, it specifically includes the following content.

[0031] (a) KP-Warp Partial Cross-Entropy Loss: For tasks such as incomplete vessel annotation, based on the prior knowledge that the neighborhood of a vessel's keypoint is more likely to be an unannotated vessel, the KP-Warp Partial Cross-Entropy (KP-pCE) loss function is proposed. kp-pCE This loss only relaxes the background class constraint within a local region, as shown below:

[0032]

[0033] in It is the annotation value of the i-th voxel in the 0th channel of the one-hot encoded data. It is the predicted probability value of the i-th voxel in the 0th channel of the one-hot encoding. It is the annotation value of the i-th voxel in the c-th channel of the one-hot encoded data. ω is the predicted probability value of the i-th voxel in the c-th channel of one-hot encoding. bg It is the background pixel set, ω kp It is the set of non-zero pixels in the keypoint map, ω g It is a set of labeled pixels, |ω g | represents the size of the labeled pixel set.

[0034] KP-Warp Mumford-Shah Loss: Active contour models are unsupervised segmentation methods that evolve contours to segment target objects in an image by minimizing an energy functional. Level set methods implicitly represent contours using level set functions. The Mumford-Shah (MS) functional is one well-known model. Inspired by this, the KP-Warp Mumford-Shah (KP-MS) loss L is proposed. kp-MS As shown below:

[0035]

[0036] in

[0037]

[0038] α is the weighting parameter, I i It is the intensity value of pixel i. It is the keypoint intensity value of the i-th voxel in the c-th channel of the one-hot encoded data. Let be the predicted probability value of the i-th voxel in the c-th channel of the one-hot encoding, where C and N are the total number of predicted classes and the total number of voxels, respectively. By minimizing the KP-MS loss, the intensity variance within the keypoint structural region is reduced. This deformation loss improves the topology-preserving ability of the MS loss under weak supervision.

[0039] (b) Iterative Correction of Weak Labels Using KP-Map: Since labels are incomplete, erroneous information can hinder the neural network from achieving optimal performance during training based on weakly supervised loss, often leading to oversegmentation. Therefore, iteratively correcting weak labels based on segmentation results is crucial for improving model performance. This can be effectively achieved using keypoint maps, with each iteration consisting of three steps:

[0040] 1) Train the segmentation network with incomplete annotations under weak supervision loss to obtain the prediction mask.

[0041] 2) Extract key point maps from weak annotations and binarize them to generate key point masks.

[0042] 3) Derive the revised annotation using the following equation.

[0043]

[0044] in These represent the masks obtained from the keypoints and the probability map, respectively.

[0045] Table 1. Quantitative experimental results of the hepatic vein dataset under weakly supervised learning. Values ​​in bold represent the best results achieved using a non-iterative training strategy with p-values ​​less than 0.05.

[0046]

[0047] To illustrate the superiority of this invention over other weakly supervised medical image segmentation methods, a numerical comparison of the hepatic vein segmentation performance of this invention with other cutting-edge weakly supervised medical image segmentation methods is presented here. This experiment compared five different loss functions: pCE, pCE+EM, pCE+MS, KP-pCE, and pCE+KP-MS loss. Table 1 shows the quantitative evaluation results of the four loss functions on the test set. The pCE loss was used as the baseline for comparison, with Dice and clDice of 0.2479 and 0.3824, respectively. Next, EM, MS, and KP-MS were used as regularization terms. They all significantly improved various metrics. Among them, KP-MS was the most effective, improving Dice and clDice by approximately 0.11 and 0.24, respectively, compared to pCE. However, only KP-pCE effectively suppressed the oversegmentation problem and repaired the breaks present in the weak annotations. The network retrained based on the corrected annotations (KP-pCE-i2) further enhanced the segmentation of small vessels and achieved the best performance. Figure 3 The segmentation results of six methods were visualized. This invention uses weakly labeled regions containing many unlabeled vascular areas, which are fragmented compared to fully labeled regions. The segmentation results of pCE and EM losses are sufficient to identify the main parts of blood vessel management, but a large number of vessels are incorrectly connected. In contrast, the KP-MS loss is more accurate at vessel endpoints and branches because it enhances the ability to focus on key shape structures. However, all four pCE-based loss functions are inferior to KP-pCE in all metrics.

[0048] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A weakly supervised blood vessel segmentation method based on tree-structured key points, characterized in that, Includes the following steps: (a) A loss function based on key-points is proposed, which uses the key-points graph to transform various X loss functions, making them suitable for weakly supervised learning scenarios. (b) A key-points graph is proposed to correct incomplete annotations and is used in the iterative training process to further improve the segmentation performance under weak supervision.

2. The weakly supervised vessel segmentation method based on tree-structured key points according to claim 1, characterized in that, In step (a), under the weakly supervised learning setting, the following loss functions are proposed: KP-Warp partial cross-entropy loss and KP-Warp Mumford Shah loss, as follows: KP-Warp Partial Cross-Entropy Loss: For tasks with incomplete vessel annotation, based on the prior knowledge that the neighborhood of a vessel's keypoint is more likely to be an unannotated vessel, a KP-Warp partial cross-entropy loss function is proposed. kp-pCE This loss only relaxes the background class constraint within a local region, as shown below: in It is the annotation value of the i-th voxel in the 0th channel of the one-hot encoded data. It is the predicted probability value of the i-th voxel in the 0th channel of the one-hot encoding. It is the annotation value of the i-th voxel in the c-th channel of the one-hot encoded data. ω is the predicted probability value of the i-th voxel in the c-th channel of one-hot encoding. bg It is the background pixel set, ω kp It is the set of non-zero pixels in the keypoint map, ω g It is a set of labeled pixels, |ω g | represents the size of the labeled pixel set; KP-Warp Mumford-Shah Loss: The active contour model is an unsupervised segmentation method that evolves contours to segment target objects in an image by minimizing an energy functional; the level set method implicitly represents the contours using level set functions; and the proposed KP-Warp Mumford-Shah loss L... kp-MS As shown below: in α is the weighting parameter, I i It is the intensity value of pixel i. It is the keypoint intensity value of the i-th voxel in the c-th channel of the one-hot encoded data. is the predicted probability value of the i-th voxel in the c-th channel of the one-hot encoding, where C and N are the total number of predicted categories and the total number of voxels, respectively. By minimizing the KP-MS loss, the intensity variance within the key point structural region is reduced. This deformation loss improves the ability of the MS loss to preserve the topology under weak supervision.

3. The weakly supervised vessel segmentation method based on tree-structured key points according to claim 2, characterized in that, In step (b), under the weakly supervised learning setting, a keypoint map is proposed to correct incomplete annotations, which is then used in the iterative training process, as follows: Iterative correction of weak annotations using KP-Map: Since the annotations are incomplete, erroneous information can hinder the neural network from achieving optimal performance during training based on weakly supervised loss, leading to oversegmentation. Therefore, weak annotations are iteratively corrected based on the segmentation results, and this is implemented using a keypoint map. Each iteration is completed in three steps: 1) Train the segmentation network with incomplete annotations under weakly supervised loss to obtain the prediction mask; 2) Extract keypoint maps from weak annotations and binarize them to generate keypoint masks; 3) Derive the revised annotation using the following equation. in These represent the masks obtained from the keypoints and the probability map, respectively.

Images