Weakly supervised segmentation method for medical images with tubular structure based on two-point labeling

Abstract

This invention discloses a weakly supervised segmentation method for tubular structures in medical images based on two-point annotation, relating to the field of segmentation technology for CT images with tubular structures. The first stage uses a "point-line-voxel" strategy to generate pseudo-labels for the tubular structures. First, a point is randomly selected at each end of the tubular structure. The geodesic distance connecting the two points is calculated using an anisotropic 3D Riemann metric to obtain preliminary pseudo-labels, which are continuously updated during training. The second stage employs a self-cross-supervised learning method to repair the segmentation results from the first stage. This softens the pseudo-labels to improve robustness to noise and uses the clDice loss function to preserve the topological structure of the tubular target. This invention not only yields more accurate, efficient, and robust segmentation results for elongated tubular structures but also significantly reduces the workload of manual annotation, possessing high clinical application value.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation technology, specifically relating to a weakly supervised segmentation method for medical images of tubular structures based on two-point annotation. Background Technology

[0002] Tubular structures are ubiquitous in the human body, such as blood vessels, pancreatic ducts, bile ducts, and the colon and rectum. These tubular structures are usually located in specific environments at the interface between liquids, solids, or air and surrounding tissues, playing a vital role in maintaining human physiological functions. There are also some tissues or organs in the human body that, while not tubular structures, resemble them in shape, exhibiting an elongated form; these can be called tubular-like structures. For example, although the pancreas is not a tubular structure, it is a long and narrow gland, divided into four parts: the head, neck, body, and tail. It is typically 17-20 cm long, 3-5 cm wide, and 1.5-2.5 cm thick, thus it can be considered a tubular-like structure. Another example is the hippocampus in the human brain, located between the thalamus and the medial temporal lobe, which arcs around the midbrain and is divided into anterior and posterior regions. It is 4.0-4.5 cm long, and its width and thickness are both approximately 1 cm. Its shape is more like a banana or a bent little finger, and it can also be considered a tubular-like structure. Therefore, tubular structures are a broader category of tubular structures. This invention will explore a weakly supervised segmentation method for tubular targets based on anatomical priors, taking into account the unique elongated anatomical characteristics of tubular structures and drawing upon methods for segmenting tubular structures.

[0003] Segmentation of tubular structures has always been a research hotspot in medical image segmentation. Existing methods can be summarized into two categories:

[0004] (1) Geometric-based methods; (2) Learning-based methods. Geometric-based methods utilize the geometric characteristics of tubular structures to construct deformable energy functionals. For example, tubular structures can be approximated as cylinders to reconstruct tubular regions, and the tubular outline can be further evolved using level set methods. Another example is to extract the centerline of the tubular structure to facilitate the segmentation of the tubular region. However, due to the lack of a powerful learning model, these methods cannot automatically extract features and cannot handle tubular structure segmentation tasks with interference such as poor contrast, noise, and complex backgrounds. Learning-based methods detect tubular structures by learning a model that classifies each pixel. The research progress of this type of method has largely benefited from deep learning, especially fully convolutional networks. Fully convolutional networks and their variants have become the main models for segmenting tubular organs or tissues and have improved segmentation accuracy. However, these networks only attempt to learn a class label for each voxel, which inevitably ignores the geometric characteristics of voxels in tubular structures, and therefore still cannot guarantee that the obtained segmentation has the correct shape. In addition, due to the slender anatomical characteristics of tubular structures, drawing manual annotations is more time-consuming and laborious, which is also one of the challenges of fully supervised learning methods.

[0005] Compared to common tubular structures in the human body, tubular-like structures are relatively larger in width and thickness, and exhibit a wider range of gray-level unevenness. Furthermore, the interiors of tubular structures are generally filled with gas or liquid, while tubular-like structures are typically solid organs or tissues, thus having a density similar to surrounding organs and tissues, resulting in relatively lower gray-level contrast between tubular-like structures and the background on CT images. In addition, tubular-like structures face challenges due to their variable size, shape, and location. Based on these facts, this invention proposes a two-stage, geometrically prior, two-point annotation-based weakly supervised segmentation method for tubular-like structures. This invention only requires taking one point each at the head and tail of the tubular region as weak annotations, without needing to take more points outside or at the boundary of the tubular region. This provides both initial localization and geometrical prior information, and offers a weakly supervised segmentation method that reduces manual annotation, possessing high clinical application value. Summary of the Invention

[0006] Objective: To address the challenges posed by the elongated anatomical features of tubular structures, including uneven grayscale, variable position, and shape, this invention proposes a two-stage weakly supervised segmentation algorithm incorporating geometric priors. In the first stage, an anisotropic 3D Riemannian metric is constructed to extract voxel information from the tubular structures. A "point-line-voxel" strategy is used to generate pseudo-labels for the tubular structures to alleviate the problem of class label imbalance. In the second stage, a self-cross-supervised learning method is employed to mitigate noise in the pseudo-labels. The self-trained loss function uses the clDice loss function to preserve the topological structure of the tubular targets. This invention explores a novel method for generating pseudo-labels, reducing the workload of manual annotation and demonstrating high clinical application value in the segmentation of tubular structures in medical images.

[0007] Technical Solution: This invention proposes a weakly supervised segmentation method for medical images of tubular structures based on two-point annotation, comprising the following four steps:

[0008] (a) Take a point at the head and tail of the internal region of the tubular structure CT image as a weak annotation point and construct an anisotropic 3D Riemann metric.

[0009] (b) Based on the newly constructed 3D Riemann metric, the geodesic distance connecting two labeled points is calculated to obtain the geodesic line connecting the two labeled points. Then, the geodesic distance from all pixels on the CT image of the tubular structure to the points on the geodesic line is calculated to obtain the preliminary pseudo-labels of the tubular structure. That is, the pseudo-labels of the tubular structure are generated by the "point-line-voxel" strategy.

[0010] (c) Use preliminary pseudo-labels for supervised learning, and combine partial supervised loss function and CV model loss function to continuously update 3D Riemann metric during training, thereby updating pseudo-labels to obtain more accurate results;

[0011] (d) The updated result from step (c) is used as a pseudo-label. A self-supervised learning method that preserves the topology of tubular targets is employed to mitigate noise in the pseudo-labels and improve model performance. On the one hand, the pseudo-labels are softened to improve robustness to noise through cross-knowledge distillation. On the other hand, a teacher and student model is adopted, in which the clDice loss function used for training can preserve the topology of tubular targets.

[0012] Furthermore, in step (a), the construction of the anisotropic 3D Riemannian metric takes into account regional information and local geometric information.

[0013] First, we introduce a quantity to characterize the unevenness of image gray levels. Specifically, first, select points p and q at the head and tail of the tubular region, respectively. Then, calculate the union of the two neighborhoods of all pixels x in the image with the head point p and the tail point q. The difference in the average grayscale value of the pixels, i.e.

[0014]

[0015] Where I(x) and I(y) represent the gray values ​​at pixel points x and y, respectively;

[0016] Secondly, according to the optimal directed flux method, the flux function f(x) can be equivalently written as

[0017] f(x) = v T F(x)v,

[0018] in S is the Hessian matrix of the smoothed image. r It is a small ball with radius r. It is S r The characteristic function is f(x); v represents direction information; and the problem of minimizing flux f(x) is equivalent to performing eigenvalue decomposition on matrix F(x). The eigenvector corresponding to the smallest eigenvalue of F(x) is the optimal direction to be found, i.e., F(x) = λ1(x)v1(x)v1(x). T +λ2(x)v2(x)v2(x) T +λ3(x)v3(x)v3(x) T Here, λ1(x), λ2(x), and λ3(x) are the eigenvalues ​​of F(x), which characterize the magnitude of the gray-level curvature along their respective eigenvectors, while v1(x), v2(x), and v3(x) are the corresponding eigenvectors, which characterize the three mutually perpendicular directions of the tubular structure. However, the Riemann metric matrix must be a positive definite matrix, so the Riemann metric is constructed by modifying the eigenvalues ​​of F(x).

[0019]

[0020] in These are the exponential function values ​​taken with respect to the eigenvalues ​​λ1(x), λ2(x), and λ3(x), respectively.

[0021] Finally, the amount of grayscale unevenness will be... Multiplied by the conformity factor Preliminary 3D Riemannian metric was obtained.

[0022]

[0023] Furthermore, in step (b), the 3D Riemann metric newly constructed in step (a) is used. Calculate the geodesic distance connecting two marked points to obtain the geodesic line connecting the two marked points; specifically, use the newly constructed 3D Riemann metric. Substituting into the following equation, we obtain the geodetic distance connecting the two marked points. This yields the geodesic line connecting the two marked points;

[0024]

[0025] geodetic distance Ω→R + It is the minimum energy of the path integral along the path connecting the fixed point p and any point x in Ω, the potential energy function. γ(t) is a smooth curve, and γ′(t) is the tangent line to the curve γ(t). It is the set of all paths connecting points x and p.

[0026] Furthermore, in step (c), preliminary pseudo-labels are used for supervised learning, and the 3D Riemann metric is continuously updated during training by combining a partially supervised loss function and a CV model loss function. Specifically, to ensure that all geodesics are within the target region, the probability of the background region is used to penalize geodesics that cross the background region during training; let f(θ) be the foreground probability value of each pixel predicted by a convolutional neural network parameterized by weights θ; therefore, the Riemann metric is updated as follows:

[0027]

[0028] Repeat the above steps to update the pseudo-label of the tubular structure, denoted as Ω. M This is the initial tubular domain; let Ω B =Ω out ∪(Ω-Ω M ) as the initial background region, then Ω R =Ω B ∪Ω M This is a pseudo-label area that has already been preliminarily marked;

[0029] Let Q be the binary function of the pseudo-label, and let Ω be the value of the pseudo-label. M The value at the pixel in Ω is 1, while in Ω B The value at each pixel in the array is 0; therefore, Q trains a segmentation network using a partially supervised loss.

[0030]

[0031] Where P is the predicted probability; and These are partial cross-entropy loss and partial Dice loss:

[0032]

[0033] Where p i q is the foreground probability value at pixel i in P. i This is the corresponding value in Q. Cross-entropy loss has a larger gradient for each pixel, while Dice loss can better alleviate the class imbalance problem in segmentation tasks;

[0034] Inspired by unsupervised learning, a CV variational model loss function was added during training to generate more pseudo-labels;

[0035]

[0036] Among them I i c1 is the gray value at pixel i in the image, and c2 refers to the average gray values ​​of the foreground and background regions, respectively.

[0037]

[0038] The overall loss function for training is as follows:

[0039]

[0040] Where α is a hyperparameter that controls the weights of the level set loss function.

[0041] Furthermore, in step (d), a self-cross-supervised learning method that preserves the topology of tubular targets is used to alleviate noise in pseudo-labels and improve model performance. Specifically, on the one hand, pseudo-labels are softened to improve robustness to noise through cross-knowledge distillation, and on the other hand, a teacher and student model is used, in which the clDice loss function used for training can preserve the topology of tubular targets.

[0042]

[0043] Where p i It represents the probability that pixel i belongs to the foreground region. Pixel i is a soft label indicating that it belongs to the foreground region, and T is a temperature parameter; when T>1, there is... Soften the original labels to make them smoother, improve the model's generalization ability, and reduce the impact of noise;

[0044] Let p t and p s Y represents the foreground probabilities predicted by the teacher and student models, respectively. t and Y s These represent the pseudo-labels for the teacher and student models, respectively. and These represent the soft labels for the foreground regions predicted by the two models, one for teachers and one for students.

[0045]

[0046] in

[0047]

[0048] Represents cross-knowledge distillation terms. This indicates that pseudo-label Y is used respectively. t and Y s The self-training terms of the two networks are supervised, where λ is a hyperparameter balancing the contributions of KD loss and self-training loss; a combination of cross-entropy loss, Dice loss, and clDice loss is used; let V Q V represents a binary image of pseudo-labels. P A binary graph representing the prediction results, S Q and S P They represent from V Q and V P The skeleton extracted from it, Indicates topological precision. To represent topological sensitivity, then

[0049] The beneficial effects of this invention are as follows: This invention proposes a weakly supervised segmentation algorithm for tubular structures based on two-point annotation and incorporating the anatomical features of narrow and elongated structures. It uses a "point-line-voxel" strategy to generate pseudo-labels and employs a self-cross-supervised learning method to alleviate noise in the pseudo-labels, thus reducing the workload of manual annotation. This invention achieves higher segmentation accuracy than other latest weakly supervised methods based on weak annotation, enabling the detection, localization, and segmentation of medical images of tubular structures, and has high clinical application value. Attached Figure Description

[0050] Figure 1 This is a structural diagram of the weakly supervised segmentation method for tubular structures in medical images based on two-point annotation proposed in this invention.

[0051] Figure 2 This is a visualization of the extraction of directional information of tubular structures proposed in this invention.

[0052] Figure 3 This is a comparative ablation experiment diagram of the anisotropic 3D Riemann metric constructed in this invention; the first and third rows represent the geodesics of the pancreas and abdominal aorta, respectively, and the second and fourth rows represent the segmentation results of the pancreas and abdominal aorta, respectively. (a): Baseline model (no orientation information or background probability), (b): Only background probability, (c): Only orientation information, (d): Both orientation information and background probability are present;

[0053] Figure 4The image shows a comparison of the results of two stages of the weakly supervised segmentation method for tubular structures in medical images based on two-point annotation proposed in this invention; (a): original image, (b): segmentation result of the first stage, (c): segmentation result of the second stage, (d): gold standard; green indicates oversegmentation, blue indicates undersegmentation, and red indicates correct segmentation.

[0054] Figure 5 The figures show a comparison of the results of this invention with different weakly supervised segmentation methods on three datasets; (a): original image, (b): gold standard, (c): SL, (d): DTP, (e): PASeg, (f): InExIS, (g): MedSAM, (h): tubular structure segmentation results of this invention.

[0055] Figure 6 This is a diagram illustrating the robustness of the two-point annotation method proposed in this invention regarding location; columns 1, 3, 5, and 7 represent the different locations of the annotation points in the tubular region; columns 2, 4, 6, and 8 represent the corresponding segmentation results; and the last column represents the gold standard for the pancreas, spleen, abdominal aorta, and hippocampus. Detailed Implementation

[0056] The objectives, technical solutions, and advantages of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] This invention proposes a weakly supervised segmentation algorithm for tubular structures based on the slender anatomical features of two annotation points. It uses a "point-line-voxel" strategy to generate pseudo-labels and employs a self-cross-supervised learning method to alleviate noise in the pseudo-labels, thereby reducing the workload of manual annotation.

[0058] The overall flowchart of the present invention is as follows: Figure 1 As shown, the specific steps include:

[0059] (a) The construction of the anisotropic 3D Riemannian metric, which simultaneously considers regional information and local geometric information;

[0060] First, we introduce a quantity to characterize the unevenness of image gray levels. Specifically, first, select points p and q at the head and tail of the tubular region, respectively. Then, calculate the union of the two neighborhoods of all pixels x in the image with the head point p and the tail point q. The difference in the average grayscale value of the pixels, i.e.

[0061]

[0062] Where I(x) and I(y) represent the gray values ​​at pixel points x and y, respectively;

[0063] Secondly, construct an anisotropic 3D Riemannian metric. Based on the optimal directional flux method, an optimal direction parallel to the tubular structure is found such that the flux flowing out along the optimal direction, by projecting the gradient vector of the image grayscale onto this direction, is minimized. By the divergence theorem, the flux function f(x) is equivalently written as:

[0064] f(x) = v T F(x)v,

[0065] in S is the Hessian matrix of the smoothed image I. r It is a small ball with radius r. It is S r The characteristic function is f(x), where v is the direction information; therefore, finding the minimum flux f(x) is equivalent to performing eigenvalue decomposition on matrix F(x). The eigenvector corresponding to the minimum eigenvalue of F(x) is the optimal direction to be found. See Figure 2 ;

[0066] The eigenvalues ​​λ1(x), λ2(x), and λ3(x) of F(x) characterize the magnitude of the gray-level curvature along their respective eigenvectors; while the eigenvectors v1(x), v2(x), and v3(x) characterize the three mutually perpendicular directions of the tubular structure; since F(x) is a real symmetric matrix, its orthogonal decomposition is F(x) = λ1(x)v1(x)v1(x) T +λ2(x)v2(x)v2(x) T +λ3(x)v3(x)v3(x) T However, the Riemann metric matrix must be a positive definite matrix, so the Riemann metric is constructed by modifying the eigenvalues ​​of F(x).

[0067]

[0068] in These are the exponential function values ​​taken with respect to the eigenvalues ​​λ1(x), λ2(x), and λ3(x), respectively.

[0069] Finally, the amount of grayscale unevenness will be... Multiplied by the conformity factor Preliminary 3D Riemannian metric was obtained.

[0070]

[0071] (b) is based on the newly constructed 3D Riemannian metric in step (a). Calculate the geodesic distance connecting the two marked points to obtain the geodesic line connecting them. Specifically, first, manually select one point at the head and one at the tail of the tubular structure region as the two endpoints of the subsequent geodesic line. To utilize the grayscale information of the tubular structure, the two marked points must be located inside the target region near the head and tail, not outside the region. Next, use these two points to generate a 3D bounding box, and to ensure that the target region is completely contained within the bounding box, further expand the length, width, and height of the bounding box outward by 10 pixels each. Let Ω represent the entire CT image, and denote the region outside the bounding box as Ω. out The pixels within this area can be identified as belonging to the background region;

[0072] The newly constructed 3D Riemannian metric Substituting into the following equation, we obtain the geodetic distance connecting the two marked points.

[0073]

[0074] geodetic distance Ω→R + It is the minimum energy of the path integral along the path connecting the fixed point p and any point x in Ω, the potential energy function. γ(t) is a smooth curve, and γ′(t) is the tangent line to the curve γ(t). It is the set of all paths connecting points x and p; furthermore, the distance function... The solution uses a fast traversal algorithm and performs gradient descent search on the distance function from q to p in reverse. The Runge-Kutta algorithm is used to obtain the geodesics of the two marked points.

[0075] (c) utilizes preliminary pseudo-labels for supervised learning, while continuously updating the 3D Riemann metric during training by combining a partially supervised loss function and a CV model loss function. Specifically, to ensure that all geodesics are within the target region, the probability of the background region is used to penalize geodesics that cross the background region during training. Let f(θ) be the foreground probability value of each pixel predicted by a convolutional neural network parameterized by weights θ. Therefore, the Riemann metric is updated as follows:

[0076]

[0077] Repeat the above steps to update the pseudo-label of the tubular structure, denoted as Ω. M This is the initial tubular domain. Let Ω B =Ω out ∪(Ω-Ω M ) as the initial background region, then Ω R =Ω B ∪Ω MThis is a pseudo-label area that has already been preliminarily marked;

[0078] Let Q be the binary function of the pseudo-label, and let Ω be the value of the pseudo-label. M The value at the pixel in Ω is 1, while in Ω B The value at each pixel in the array is 0; therefore, Q trains a segmentation network using a partially supervised loss.

[0079]

[0080] Where P is the predicted probability; and These are partial cross-entropy loss and partial Dice loss:

[0081]

[0082] Where p i q is the foreground probability value at pixel i in P. i This is the corresponding value in Q; cross-entropy loss has a larger gradient for each pixel, and Dice loss can better alleviate the class imbalance problem in segmentation tasks.

[0083] Inspired by unsupervised learning, a CV variational model loss function was added during training to generate more pseudo-labels;

[0084]

[0085] Among them I i c1 is the gray value at pixel i in the image, and c2 refers to the average gray values ​​of the foreground and background regions, respectively.

[0086]

[0087] The overall loss function for training is as follows:

[0088]

[0089] Where α is a hyperparameter that controls the weights of the level set loss function;

[0090] (d) employs a self-cross-supervised learning method that preserves the topology of tubular targets to mitigate noise in pseudo-labels and improve model performance. Specifically, on the one hand, pseudo-labels are softened to improve robustness to noise through cross-knowledge distillation, and on the other hand, a teacher and student model is adopted, in which the clDice loss function used for training can preserve the topology of tubular targets.

[0091]

[0092] Where p iIt represents the probability that pixel i belongs to the foreground region. Pixel i is a soft label indicating that it belongs to the foreground region, and T is a temperature parameter; when T>1, there is... Soften the original labels to make them smoother, improve the model's generalization ability, and reduce the impact of noise;

[0093] Let p t and p s Y represents the foreground probabilities predicted by the teacher and student models, respectively. t and Y s These represent the pseudo-labels for the teacher and student models, respectively. and These represent the soft labels for the foreground regions predicted by the two models, one for teachers and one for students.

[0094] in

[0095]

[0096] Represents cross-knowledge distillation terms. This indicates that pseudo-label Y is used respectively. t and Y s Supervise the self-training terms of both networks. λ is a hyperparameter balancing the contributions of KD loss and self-training loss; a combination of cross-entropy loss, Dice loss, and clDice loss is used, where V... Q V represents a binary image of pseudo-labels. P A binary graph representing the prediction results, S Q and S P They represent from V Q and V P The skeleton extracted from it, Indicates topological precision. To represent topological sensitivity, then

[0097] To more clearly illustrate the feasibility and superiority of this invention, the performance of the method in ablation experiments is shown below. To analyze the respective contributions of orientation information and background region probability in the Riemann metric, only one type of information is used in the Riemann metric to segment the pancreas and abdominal aorta. Figure 3Examples of these segmentations are shown, and Table 1 provides a quantitative analysis of the segmentation results. (1) Effectiveness of orientation information: Without embedded orientation information, geodesics will run outside the target area, resulting in the generation of incorrect pseudo-labels. (2) Effectiveness of background probability: Without embedded background region probability, geodesics will still run outside the target area, but the deviation will be small, so the accuracy of the generated pseudo-labels is relatively high. (3) Effectiveness of background probability and orientation information: If neither background probability nor orientation information is embedded in the Riemann metric, the baseline model performs the worst experimentally, indicating that both orientation information and background probability are important for improving segmentation performance.

[0098] Table 1. Comparison of numerical results from ablation experiments using a weakly supervised segmentation method for tubular structures in medical images based on two-point annotation.

[0099]

[0100] To illustrate the results of each stage of the two-stage weakly supervised segmentation of tubular structures proposed in this invention, experiments were conducted on four datasets: pancreas, spleen, abdominal aorta, and hippocampus. The experimental results are shown below. Figure 4 As can be seen, after the second-stage noise robustness learning repair, the first-stage two-point-based weakly supervised segmentation results were slightly improved. Specifically, from... Figure 4 As shown in the second column, the pancreas and hippocampus segmentation results in the first stage exhibited both undersegmentation and oversegmentation at the boundaries, which were improved to some extent after the second stage repair. Table 2 shows the quantitative indicators of the two stages in this invention. The four evaluation indicators of the segmentation results in the first stage all improved by approximately 1.5% after the noise robustness learning repair in the second stage, verifying the effectiveness of the self-crossing supervised learning method proposed in the second stage that preserves the topology of tubular targets.

[0101] Table 2 Comparison of results at each stage of the two-stage weakly supervised segmentation of tubular structures

[0102]

[0103] To illustrate the superiority of this invention over other weakly supervised medical image segmentation methods, a comparison of the segmentation performance and numerical experiments of this invention with other cutting-edge weakly supervised medical image segmentation algorithms is presented here. Figure 5The segmentation performance of this invention and other weakly supervised segmentation algorithms based on weak annotation is shown on four datasets. Table 3 quantitatively compares the segmentation performance of this invention and other weakly supervised segmentation algorithms on medical images of tubular structures. Although DTP and SL perform well, they show some obvious errors at the boundaries of the segmentation results. This is because they only consider the prior knowledge of the volume and topology of the target region, without considering features such as grayscale and geometric information of the foreground and background regions. The method of this invention has the highest accuracy, with few false positives and false negatives, which also demonstrates the effectiveness and robustness of the method.

[0104] Table 3 compares the numerical results with other weakly supervised segmentation algorithms on four datasets.

[0105]

[0106]

[0107] To demonstrate the robustness of the single-point annotation proposed in this invention with respect to position, different locations were selected within a tubular structure region to verify the robustness with respect to the positions of two annotation points. (See...) Figure 6 As can be observed from Table 4, as long as the two points are marked within the tubular structure region, the corresponding segmentation results show little difference. However, if the two points are marked on the boundary of the tubular structure region, the segmentation results will be affected by the surrounding tissue. Therefore, it can be concluded that the two points can be marked at any location within the tubular structure region, but cannot be marked outside the tubular structure region or on its boundary, which is consistent with the theoretical analysis of this invention.

[0108] Table 4 compares the Dice values ​​of segmentation results when two-point annotations are taken at four different locations in the target region.

[0109] Dataset Result 1 Result 2 Result 3 Result 4 pancreas 74.88±5.89 74.59±6.64 73.54±8.21 69.02±7.90 spleen 81.71±7.97 81.33±8.75 80.69±9.19 76.63±10.15 Abdominal aorta 78.55±9.58 77.88±9.43 77.26±10.87 71.47±11.25 hippocampus 75.89±9.21 76.22±10.86 75.05±9.74 68.72±12.41

Claims

1. A weakly supervised segmentation method for medical images of tubular structures based on two-point annotation, characterized in that, Includes the following steps: (a) Take a point at the head and tail of the internal region of the tubular structure CT image as a weak annotation point and construct an anisotropic 3D Riemann metric; (b) Based on the newly constructed 3D Riemann metric, the geodesic distance connecting two labeled points is calculated to obtain the geodesic line connecting the two labeled points. Then, the geodesic distance from all pixels on the CT image of the tubular structure to the points on the geodesic line is calculated to obtain the preliminary pseudo-labels of the tubular structure. That is, the pseudo-labels of the tubular structure are generated by the "point-line-voxel" strategy. (c) Use preliminary pseudo-labels for supervised learning, and combine partial supervised loss function and CV model loss function to continuously update 3D Riemann metric during training, thereby updating pseudo-labels; (d) The updated result of step (c) is used as a pseudo label. A self-cross-supervised learning method that preserves the topology of tubular targets is adopted to alleviate the noise in the pseudo label. On the one hand, the pseudo label is softened to improve the robustness to noise through cross-knowledge distillation. On the other hand, a teacher and student model is adopted, in which the clDice loss function used for training can preserve the topology of tubular targets.

2. The weakly supervised segmentation method for tubular structure medical images based on two-point annotation according to claim 1, characterized in that: The construction of the anisotropic 3D Riemannian metric in step (a) considers both region information and local geometric information; specifically: First, we introduce a quantity to characterize the unevenness of image gray levels. Specifically, first select one point at the head and one point at the tail of the tubular region. , Then calculate all pixels in the image. With head point Tail point Union of two neighborhoods The difference in the average grayscale value of the pixels, i.e. in, and These represent pixels. , The grayscale value at that location; Secondly, according to the optimal directed flux method, the flux function Equivalently written as in , It is the Hessian matrix of the smoothed image. It is a radius of The small ball, yes Indicator functions; It provides directional information; while calculating flux... Minimal problem is equivalent to matrix Perform feature decomposition. The eigenvector corresponding to the smallest eigenvalue is the optimal direction to be found. ,in The eigenvalues ​​respectively characterize the magnitude of the gray-level curvature along the direction of their respective eigenvectors, while The three mutually perpendicular directions of the tubular structure were depicted respectively; through modification Constructing Riemannian metrics using eigenvalues in , , These are the eigenvalues Take the value of the exponential function; Finally, the amount of grayscale unevenness will be... Multiplied by the conformity factor Preliminary 3D Riemann metric was obtained. 。 3. The weakly supervised segmentation method for tubular structure medical images based on two-point annotation according to claim 2, characterized in that: In step (b), based on the newly constructed 3D Riemannian metric from step (a), the geodesic distance connecting the two marked points is calculated, resulting in the geodesic line connecting the two marked points; specifically: The newly constructed 3D Riemannian metric Substituting into the following equation, we obtain the geodetic distance connecting the two marked points. Thus, the geodesic line connecting the two marked points is obtained; geodetic distance It is along the connection fixed point and any point in the middle The minimum energy of the path integral, the potential energy function , It is a smooth curve. It is a curve tangent, It is a connection point and The set of all paths.

4. The weakly supervised segmentation method for tubular structure medical images based on two-point annotation according to claim 3, characterized in that: Step (c) utilizes preliminary pseudo-labels for supervised learning, while simultaneously updating the 3D Riemann metric during training by combining a portion of the supervised loss function and the CV model loss function; specifically: To ensure that all geodesics are within the target area, we consider using the probability of the background region during training to penalize geodesics that cross the background region; let... It is a weight The parameterized convolutional neural network predicts the foreground probability value for each pixel; therefore, the Riemann metric is updated to... Repeat the above steps to update the pseudo-label of the tubular structure, denoted as This is the initial tubular domain; let As a preliminary background area, This is a pseudo-label area that has already been preliminarily marked; The binary function of the pseudo-label is denoted as... ,exist The value at the pixel in the image is 1, while... The value at each pixel in the image is 0; therefore... Train a segmentation network using partially supervised loss: in It is the predicted probability; and These are partial cross-entropy loss and partial Dice loss: in yes medium pixel Foreground probability value at location, yes The corresponding value in; Inspired by unsupervised learning, a CV variational model loss function was added during training to generate more pseudo-labels; in pixels in an image grayscale value at that location and These refer to the average grayscale values ​​of the foreground and background regions, respectively. , The overall loss function for training is as follows: in It is a hyperparameter that controls the weights of the level set loss function.

5. The weakly supervised segmentation method for tubular structure medical images based on two-point annotation according to claim 4, characterized in that: In step (d), a self-cross-supervised learning method that preserves the topology of tubular targets is used to alleviate noise in pseudo-labels. On the one hand, pseudo-labels are softened to improve robustness to noise through cross-knowledge distillation. On the other hand, a teacher and student model is used, in which the clDice loss function used for training can preserve the topology of tubular targets. in It is a pixel. The probability of belonging to the foreground region. It is a pixel. Soft tags belonging to the foreground area, It is a temperature parameter; when Sometimes, Soften the original labels; make and These represent the foreground probabilities predicted by the teacher and student models, respectively. and These represent the pseudo-labels for the teacher and student models, respectively. and These represent the soft labels for the foreground regions predicted by the two models, one for teachers and one for students. in Represents cross-knowledge distillation terms. This indicates the use of pseudo-tags respectively. and Supervise the self-training terms of the two networks. It is balance Hyperparameters contributing to the loss and self-training loss; employing a combination of cross-entropy loss, Dice loss, and clDice loss; let A binary image representing pseudo-labels. A binary graph representing the prediction results. and They represent from and The skeleton extracted from it, Indicates topological precision. To represent topological sensitivity, then .

Images