Weakly supervised variational segmentation method for medical images based on single point annotation

Abstract

The present invention discloses a weakly supervised variational segmentation method for medical images based on single-point annotation, which relates to the technical field of tumor segmentation in medical images. First, a point is randomly taken in the target tumor area, and a new anisotropic Riemannian metric is constructed based on the texture direction information of the entire CT image, the local direction information of the grayscale at the boundary, and the heterogeneity of the grayscale. Secondly, two distances from all points in the entire CT image to the points taken in the tumor area are calculated: the geodesic distance and the Euclidean distance based on the new Riemannian metric. Subsequently, these two point-by-point distance information are embedded in the Chan-Vese model. Finally, the existence and uniqueness of the model solution are proved, and the Douglas-Rachford algorithm is used to solve it to obtain the final tumor segmentation result. The present invention can obtain more accurate, more effective, and more robust segmentation results, which can be used for tumor detection, positioning and segmentation. It can also greatly reduce the workload of manual labeling and has high clinical application value.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and in particular relates to a medical image weakly supervised variational segmentation method based on single-point annotation. Background Art

[0002] There are currently two mainstream approaches in the field of image segmentation: data-driven and model-driven. Data-driven approaches, primarily those based on deep learning, have witnessed rapid progress in semantic segmentation over the past decade in diverse fields, such as medical diagnosis, aerial imaging, and autonomous driving. However, high-performing segmentation models typically require extensive, high-quality manual annotations. It is well known that annotating images drawn by doctors is a time-consuming and labor-intensive task. Model-driven approaches, primarily those based on traditional variational models, while not requiring extensive manual annotation, often prove insufficient for segmenting complex medical images and often require additional prior knowledge to aid the segmentation task. Therefore, to alleviate these challenges, weakly supervised segmentation techniques have garnered increasing attention and research. By providing simple, weakly supervised information, such as point annotations, bounding box annotations, line annotations, and image-level annotations, information related to the target region can be extracted from the image, alleviating the difficulty of obtaining manual annotations while also providing valuable prior information.

[0003] Weakly supervised segmentation based on optimization models uses graph theory or variational models to construct energy functionals and iteratively minimizes the energy functional to obtain the final segmentation result, such as graph cuts and selective segmentation methods based on variational models. Weakly supervised segmentation based on deep learning typically uses weak annotations in different semantic contexts. For example, weak annotations are used to obtain pseudo-labels before network training to mimic fully supervised learning, weak annotations are incorporated into the network architecture to improve or guide network predictions, and weak annotation information is used in the post-processing stage after inference to repair prediction results. In fact, both weakly supervised segmentation methods based on optimization models and deep learning involve a key step: extracting useful segmentation information through weak annotations to guide subsequent segmentation. In addition, without additional prior knowledge, traditional variational models that only contain simple data terms and regularization terms often perform poorly when segmenting complex images, often mistaking regions that are close or far from the target region and have similar grayscale as the target region. Based on these insights, the present invention proposes a weakly supervised variational segmentation method for medical images based on single-point annotation. This method extracts useful tumor segmentation information through single-point annotation, achieving higher segmentation accuracy than other weakly supervised methods based on weak annotation. This method can not only be used for tumor detection, localization, and segmentation, but also significantly reduces the workload of manual annotation, thus possessing high clinical application value. Summary of the Invention

[0004] Purpose of the invention: In view of the current situation that fully supervised learning requires a large amount of labeled training data, the present invention proposes a weakly supervised variational segmentation method for medical images based on single-point annotation. By constructing an anisotropic Riemannian metric, the tumor grayscale variation, texture direction and local edge direction information are embedded, and the two point-by-point distance information are simultaneously embedded in the variational model to guide tumor segmentation, reducing the workload of manual annotation.

[0005] Technical solution: This paper proposes a weakly supervised variational segmentation method for medical images based only on single-point annotation, which includes the following four steps:

[0006] (a) A point is randomly selected within the target tumor region, and the structural tensor field is decomposed to characterize the texture direction of the entire CT image and the direction of the grayscale at the boundary. The local grayscale variation is also calculated to characterize the grayscale heterogeneity of the tumor.

[0007] (b) A new anisotropic Riemannian metric is constructed using the texture direction information of the entire CT image, the local direction information of the grayscale at the boundary, and the heterogeneity of the grayscale;

[0008] (c) Calculate two distances from all points in the entire CT image to points in the tumor region: geodesic distance and Euclidean distance based on the new Riemannian metric, and obtain two types of point-by-point distance information;

[0009] (d) These two point-by-point distance information are embedded into the Chan-Vese model and solved using the Douglas-Rachford algorithm to obtain the final tumor segmentation result.

[0010] Furthermore, the calculation of the structure tensor field in step (a) utilizes a corner detection operator based on the image local autocorrelation function analysis algorithm. The local autocorrelation function represents the local grayscale change when the local image window is slightly translated in different directions, and is calculated as follows:

[0011]

[0012] Where E(Δx1, Δx2) represents the local autocorrelation function, Δx1 and Δx2 represent the horizontal and vertical translation changes, respectively;

[0013]

[0014] Where f(x) represents the grayscale value at the pixel point x=(x1,x2), is a Gaussian function, * represents convolution operation, f x1 , f x2Represents the horizontal and vertical gradients of the image grayscale respectively. Q(x) can be orthogonally decomposed as: Q(x) = λ1(x)v1(x)v1(x) T +λ2(x)v2(x)v2(x) T , where λ1(x), λ2(x) are the eigenvalues ​​of Q(x), v1(x), v2(x) are the corresponding eigenvectors that can describe the texture direction of each pixel on the image and the direction information of the grayscale at the tumor boundary, and Q(x) is the structure tensor field;

[0015] Furthermore, in step (b), a new anisotropic Riemannian metric is constructed, specifically by first introducing a quantity that describes the grayscale non-uniformity of the image. The point p is a point randomly taken in the target area. is the neighborhood centered at point p, I(x) represents the grayscale at pixel x, and I(y) represents the grayscale at pixel y. Secondly, construct an anisotropic Riemannian metric Among them, λ1(x), λ2(x) are the eigenvalues ​​of the structure tensor field Q(x), v1(x), v2(x) are the corresponding eigenvectors that can describe the texture direction of each pixel on the image and the direction information of the grayscale at the tumor boundary. Finally, the grayscale inhomogeneity is calculated. As a conformal factor multiplied by Get a new Riemannian metric that embeds grayscale variation, local edge information, and texture information

[0016] Furthermore, in step (c), two distances from all points in the entire CT image to the point p in the tumor area are calculated. Specifically, the newly constructed Riemann metric Substitute into the following equation

[0017]

[0018] The geodesic distance is the minimum energy integrated along the path connecting the fixed point p and any point x in Ω, the potential energy function γ(t) is a smooth curve, γ′(t) is the tangent line of the curve γ(t), is the set of all paths connecting point x and point p. In addition, the Euclidean distance The potential energy function Specifically, the Chan-Vese model in step (d) is to transform two distances, namely the geodesic distance and Euclidean distance At the same time, it is embedded into the following Chan-Vese model to guide subsequent tumor segmentation.

[0019]

[0020] Where u(x) represents the probability function that the pixel point x belongs to the target tumor area Ω, c1 and c2 represent the grayscale mean inside and outside the target tumor area Ω, respectively.

[0021] The beneficial effects of the present invention are as follows: the present invention embeds the tumor grayscale variation, texture direction and local edge direction information by constructing an anisotropic Riemannian metric, and simultaneously embeds the two point-by-point distance information into the variational model to guide tumor segmentation, thereby reducing the workload of manual labeling; the present invention obtains higher segmentation accuracy than other latest weak-supervised medical image methods based on weak labeling, and can also be used for tumor detection, positioning and segmentation, with high clinical application value.

[0022] Contents of attached figure

[0023] Figure 1 This is a structural diagram of the weakly supervised variational segmentation method for medical images based on single-point annotation proposed in the present invention;

[0024] Figure 2 This is a visualization diagram of the structure tensor field Q(x) proposed in the present invention;

[0025] Figure 3 Comparison of ablation experiments using the two distance-embedded variational models proposed in this invention; (a): original image, (b): tumor segmentation results of the baseline model (without geodesic distance and Euclidean distance), (c): tumor segmentation results using only the Euclidean distance model, (d): tumor segmentation results using only the geodesic distance model, (e): tumor segmentation results using both distance-embedded models, (f): tumor gold standard.

[0026] Figure 4 This figure shows the robustness of the single-point annotation proposed in this paper. The first, third, fifth, and seventh columns represent the different locations of the annotation points in the target tumor region. The second, fourth, sixth, and eighth columns show the corresponding tumor segmentation results. The last column shows the gold standards for pancreatic cysts, kidney tumors, and lung tumors, respectively.

[0027] Figure 5 The comparison chart of the results of the present invention with different weakly supervised segmentation methods on three data sets; (a): original image, (b): tumor gold standard, (c): Graph-cut, (d): Grab-cut, (e): Selective-seg, (f): PA-seg, (g): DTP, (h): SAM, (i): tumor segmentation result of the present invention. DETAILED DESCRIPTION

[0028] The objectives, technical solutions and advantages of the present invention are described in detail below in conjunction with the accompanying drawings and specific embodiments of the present invention.

[0029] The present invention proposes a weakly supervised variational segmentation method for medical images based on single-point annotation. By constructing an anisotropic Riemannian metric, it embeds the tumor grayscale variation, texture direction, and local edge direction information, and simultaneously embeds the Euclidean distance and geodesic distance into the variational model to obtain the final tumor segmentation result, reducing the workload of manual annotation. The overall flow chart of the present invention is shown in the figure below. Figure 1 As shown, the specific steps include the following steps.

[0030] (a) A point is randomly selected within the target tumor region, and a corner detection operator based on the image local autocorrelation function analysis algorithm is used to extract the texture information of the entire CT image and the directional information of the grayscale at the boundary. Note that the local autocorrelation function represents the local grayscale change when the local image window is slightly translated in different directions. It is calculated as follows:

[0031]

[0032] Where E(Δx1, Δx2) represents the local autocorrelation function, Δx1 and Δx2 represent the horizontal and vertical translation changes, respectively;

[0033]

[0034] Here f(x) represents the grayscale value at the pixel point x=(x1,x2), is a Gaussian function, * represents convolution operation, Represents the horizontal and vertical gradients of the image grayscale respectively. Q(x) can be orthogonally decomposed as: Q(x) = λ1(x)v1(x)v1(x) T +λ2(x)v2(x)v2(x) T , where λ1(x), λ2(x) are the eigenvalues ​​of Q(x), v1(x), v2(x) are the corresponding eigenvectors that can describe the texture direction of each pixel on the image and the direction of the grayscale at the boundary. Q(x) is the structure tensor field. The structure tensor field is decomposed to describe the texture direction and the direction of the grayscale at the boundary in the entire CT image. At the same time, the local grayscale variation is calculated to describe the heterogeneity of the grayscale.

[0035] (b) Using the texture direction information and the local direction information of the grayscale at the boundary extracted by the single point annotation in step (a), a quantity that describes the uneven grayscale of the image is introduced. The point p is a point randomly taken in the target tumor area. is the neighborhood centered at point p, and I(x) represents the grayscale at pixel x. Secondly, construct an anisotropic Riemannian metric Among them, λ1(x), λ2(x) are the eigenvalues ​​of the structure tensor field Q(x), v1(x), v2(x) are the corresponding eigenvectors that can describe the texture direction of each pixel on the image and the direction information of the grayscale at the boundary. Finally, the grayscale unevenness is As a conformal factor multiplied by Get a new Riemannian metric that embeds grayscale variation, local edge information, and texture information Constructing a new anisotropic Riemannian metric

[0036] (c) Calculate the two distances from all points in the entire CT image to the points in the tumor area: Based on the geodesic distance and Euclidean distance of the Riemannian metric constructed in step (b), the newly constructed Riemannian metric Substitute into the following equation

[0037]

[0038] The geodesic distance is the minimum energy integrated along the path connecting the fixed point p and any point x in Ω, the potential energy function γ(t) is a smooth curve, γ′(t) is the tangent line of the curve γ(t), is the set of all paths connecting point x and point p. In addition, the Euclidean distance The potential energy function The fast marching algorithm is used to solve the eikonal equation and obtain two point-by-point distance information;

[0039] (d) The two point-by-point distance information in step (c), i.e., the geodesic distance and Euclidean distance At the same time, it is embedded into the following variational model to guide subsequent tumor segmentation.

[0040]

[0041] Where u(x) represents the probability function that pixel x belongs to the target tumor region Ω, and c1 and c2 represent the grayscale mean inside and outside the target tumor region Ω, respectively. The Douglas-Rachford algorithm is used to solve it and obtain the final tumor segmentation result.

[0042] To more clearly illustrate the feasibility and superiority of the present invention, the following demonstrates the performance of the present invention in ablation experiments. To analyze the respective contributions of geodesic distance and Euclidean distance in the variational model, only one distance was used in the variational model to segment pancreatic cysts, kidney tumors, and lung tumors. Figure 3 Examples of these segmentations are shown. (1) Validity of geodesic distance: If there is no embedded geodesic distance Then the pixels close to the target area and with similar grayscale cannot be excluded, see Figure 3 (2) Validity of Euclidean distance: If there is no embedded Euclidean distance Then the pixels that are far away from the target area and have similar grayscale cannot be excluded, see Figure 3 (3) Effectiveness of geodesic distance and Euclidean distance: If neither geodesic distance nor Euclidean distance is embedded in the model, the experimental effect of the baseline model is the worst, which shows the importance of the simultaneous effect of the two distances, see Figure 3 The second to fifth columns.

[0043] Table 1 Comparison of numerical results of ablation experiments on weakly supervised variational segmentation methods for medical images based on single-point annotation

[0044]

[0045]

[0046] In order to illustrate the robustness of the single-point annotation proposed in the present invention with respect to position, the present invention selects different positions in the target tumor area to verify the robustness of the annotation point position, see Figure 4 As can be observed in Table 2, as long as the annotation points are within the target tumor region, the corresponding tumor segmentation results are not significantly different. However, if the annotation points are located on the boundary of the target tumor region, the segmentation results will be affected by the surrounding tissue. Therefore, it can be concluded that the annotation points can be located anywhere within the target tumor region, but cannot be located outside the target tumor region or on the boundary, which is consistent with the theoretical analysis of the present invention.

[0047] Table 2 Comparison of Dice values ​​of segmentation results of single point annotation at four different locations in the target area

[0048] Dataset Result 1 Result 2 Result 3 Result 4 Pancreatic cyst 72.51±10.04 72.83±9.46 72.60±11.62 71.29±10.77 Kidney tumor 81.71±8.86 80.67±9.09 80.84±9.72 79.50±10.39 Lung tumors 77.29±7.85 76.34±5.85 76.23±8.83 74.76±7.07

[0049] In order to illustrate the superiority of the present invention compared with other weakly supervised segmentation methods for medical images, the tumor segmentation effects and numerical experimental comparisons of the present invention method and other cutting-edge weakly supervised segmentation algorithms for medical images are given here. Figure 5 The segmentation performance of the present invention and other weakly supervised segmentation algorithms on three datasets is demonstrated. Tables 3-5 quantitatively compare the tumor segmentation performance of the present invention and other weakly supervised segmentation algorithms. The numerical results demonstrate that the segmentation results proposed by the present invention achieve the highest accuracy while utilizing only a single label, making the labels more accessible.

[0050] Table 3 Comparison of numerical results with other weakly supervised segmentation algorithms on pancreatic cyst dataset

[0051]

[0052] Table 4 Comparison of numerical results with other weakly supervised segmentation algorithms on the kidney tumor dataset

[0053]

[0054]

[0055] Table 5 Comparison of numerical results with other weakly supervised segmentation algorithms on lung tumor dataset

[0056]

Claims

1. A weakly supervised variational segmentation method for medical images based on single-point annotation, characterized by: The steps include: (a) A point is randomly selected within the target tumor region, and the structural tensor field is decomposed to characterize the texture direction of the entire CT image and the direction of the grayscale at the boundary. The local grayscale variation is also calculated to characterize the grayscale heterogeneity of the tumor. (b) A new anisotropic Riemannian metric is constructed by utilizing the texture direction information of the entire CT image, the local direction information of the grayscale at the boundary, and the heterogeneity of the grayscale; (c) Calculate two distances from all points in the entire CT image to the points in the tumor region: geodesic distance and Euclidean distance based on the new Riemannian metric, and obtain two types of point-by-point distance information; (d) These two point-by-point distance information are embedded into the Chan-Vese model and solved using the Douglas-Rachford algorithm to obtain the final tumor segmentation result; The calculation of the structure tensor field in step (a) utilizes a corner detection operator based on the image local autocorrelation function analysis algorithm; the local autocorrelation function represents the local grayscale change when the local image window is translated in different directions. It is calculated as follows: ； in, represents the local autocorrelation function, Represents the translation changes in the horizontal and vertical directions respectively; ； Where, Represents pixel points The gray value at is a Gaussian function, * represents the convolution operation, Represents the horizontal and vertical gradients of the image grayscale respectively; Do orthogonal decomposition: ,in yes The eigenvalues ​​of is the corresponding eigenvector, which can describe the texture direction of each pixel point on the CT image and the direction information of the grayscale at the boundary, called is the structure tensor field; In step (b), a new anisotropic Riemannian metric is constructed, specifically by first introducing a quantity that describes the uneven grayscale of the image. , where the point is a point randomly taken in the target tumor area, It is a point The neighborhood of the center, Represents pixel points The grayscale at Represents pixel points Grayscale at Secondly, construct an anisotropic Riemannian metric ,in is the structure tensor field The eigenvalues ​​of is the corresponding eigenvector, which can describe the texture direction of each pixel on the image and the direction information of the grayscale at the boundary; finally, the grayscale unevenness As a conformal factor multiplied by , we get a new Riemannian metric that embeds grayscale variation, local edge information, and texture information ; The Chan-Vese model in step (d) is specifically to transform two distances, namely the geodesic distance and Euclidean distance At the same time, it is embedded into the following Chan-Vese model to guide subsequent tumor segmentation. ； in Represents pixel points Belong to the target tumor area The probability function of Represents the target tumor area The mean grayscale value inside and outside.

2. The weakly supervised variational segmentation method for medical images based on single-point annotation according to claim 1, characterized in that: In step (c), the distance from all points in the entire CT image to the points within the tumor area is calculated. Two distances, specifically the newly constructed Riemann metric Substitute into the following equation ； The geodesic distance Along the connection fixed point and Any point in The energy minimum of the path integral, the potential energy function , It is a smooth curve. It is a curve The tangent line, It is a connection point and point The set of all paths; In addition, the Euclidean distance The potential energy function .

Images