Hybrid level set image segmentation method based on box-level prior and weight self-constraint

Abstract

The present invention proposes a hybrid level set image segmentation method based on box-level prior and weight self-constraint, which includes the following steps: step 1, training the target detection network to obtain box-level prior information and initialize the level set function; step 2, constructing a weighted edge detection energy term, using the box-level prior information obtained in step 1 to define the weight matrix, and constraining the edge detection range to the vicinity of the target area to avoid interference from the background area; step 3, constructing a weighted self-constraint energy term, introducing a variable coefficient to constrain the relationship between the level set function and the box-level prior information, so that the evolution of the level set function changes dynamically with the degree of similarity between the two; step 4, constructing a local area fitting energy term and a distance regularization term for processing grayscale unevenness and eliminating reinitialization; step 5, combining the energy terms constructed in step 2, step 3, and step 4 to obtain a total energy functional, and minimizing the energy functional to obtain a segmentation result.

Description

Technical Field

[0001] The present invention relates to hybrid level set image segmentation using box-level priors and weight self-constraints. Background Art

[0002] Image segmentation is a key technology in computer vision, playing an important role in image analysis, pattern recognition, and computer-aided diagnosis. Currently, existing segmentation methods can be roughly divided into two categories: traditional segmentation methods and learning-based segmentation methods. Traditional segmentation methods mainly include threshold segmentation, region growing segmentation, edge detection segmentation, clustering segmentation, and active contour segmentation.

[0003] Threshold segmentation divides an image into target and background regions by setting one or more optimal grayscale thresholds. Common threshold segmentation methods include the Ostu method, the histogram bimodal method, and the maximum entropy method. However, these methods only utilize grayscale information and are sensitive to noise. The segmentation effect depends on the selected threshold, making it difficult to quickly and effectively select the optimal threshold for complex images.

[0004] Region growing segmentation uses a similarity criterion to gradually merge pixels to form the desired region. First, a seed point is selected. Pixels in the seed point's neighborhood with similar properties are merged into the region containing the seed point, forming a new seed. Merging continues until the region stops growing. This method is suitable for segmenting connected regions with the same characteristics, but it is prone to holes and over-segmentation in images with uneven grayscale and noise.

[0005] Edge detection segmentation relies on detecting significant changes in grayscale values and texture at the boundaries of different regions. Common edge detection methods include the Roberts gradient operator, the Prewitt operator, and the Canny operator. However, edge detection is susceptible to noise and cannot guarantee edge continuity and closure. It is difficult to segment images with complex content and unclear gradients.

[0006] Cluster segmentation is to cluster pixels with similar features into the same category, converting image segmentation into cluster analysis. Common clustering methods include K-means and fuzzy C-means, but this method requires manual setting of the number of clusters. The segmentation results are highly sensitive to the initial parameters and are easily interfered by noise and abnormal data.

[0007] The active contour method transforms segmentation into an energy functional minimization problem. Image information is used to construct an energy functional about a closed curve. When the energy function is minimized, the closed curve is exactly the target contour to be segmented. Depending on how the closed curve is represented, active contour models are divided into parametric active contours and geometric active contours. Parametric active contour models represent curves in a parametric form, such as the Snack model proposed by Kass et al. However, the parametric representation cannot handle the splitting and merging of contours. To address this problem, researchers introduced level set theory and proposed a geometric active contour model, commonly known as the level set method. The closed curve is implicitly represented by the zero level set of the level set function, and segmentation is achieved by continuously evolving and updating the level set function.

[0008] Existing level set segmentation models are mainly divided into edge-based and region-based models. Edge-based models use gradient information to construct energy functions, and typical models include the GAC model and the DRLSE model. Region-based models, such as the CV model and the LBF model, use image statistics to construct energy functions. Level set methods are numerically stable and have unique advantages in handling complex topological transformations. However, they are sensitive to initial contours, prone to falling into local minima, and have poor segmentation performance for images with weak boundaries and uneven grayscale.

[0009] The learning-based method is to perform end-to-end learning on manually labeled data and train the network model to achieve segmentation, which can improve the applicability of the algorithm. However, learning-based segmentation is data-driven, has high requirements on computing power, lacks interpretability, and requires massive training data with pixel-level labels. These labels need to be manually marked, which is time-consuming and labor-intensive.

[0010] Traditional segmentation methods are knowledge-driven and stable, but they lack generalizability and struggle to address complex segmentation scenarios. Learning-based methods are data-driven and highly automated, but they require massive amounts of data and suffer from poor interpretability. Both traditional and learning-based methods have their own strengths and weaknesses. Currently proposed image segmentation algorithms all have limitations and are not applicable to all image segmentation scenarios.

[0011] In summary, we combine traditional segmentation with learning-based methods to propose a hybrid level set image segmentation method based on box-level priors and weighted self-constraints. First, we use an object detection network to extract box-level prior information. Then, we define a weighted edge detection energy term and a weighted self-constraint energy term. This term is combined with a local region fitting energy term and a distance regularization term to derive a total segmentation energy functional. Finally, we minimize this energy functional using a variational method and gradient descent to obtain the segmentation result. Summary of the Invention

[0012] To address the limitations of existing image segmentation technologies, such as limited scope of application, poor segmentation effect for images with uneven grayscale and weak boundaries, large amount of manual pixel-level labeling, and low degree of automation, the present invention proposes a hybrid level set image segmentation method based on box-level prior and weighted self-constraint. The invention mainly includes: using the target detection network to automatically extract box-level prior information, improving the edge detection operator to define the weighted edge detection energy term, constructing the weighted self-constrained energy term to dynamically adjust the evolution of the level set function, and combining the region and edge information to construct an energy functional for segmentation.

[0013] A hybrid level set image segmentation method based on box-level prior and weight self-constraint is characterized by comprising at least the following steps:

[0014] Step 1: Train the object detection network to extract box-level prior information Ψ and initialize the level set function Φ. First, obtain an image dataset and build a target detection network model. Input the dataset into the network model for training to obtain a trained target detection network. Then, input the test image to be segmented into the trained target detection network and define the box-level prior information Ψ based on the output detection results. Finally, initialize the level set function Φ with an arbitrary constant.

[0015] Step 2: Construct a weighted edge detection energy term. Using the box-level prior information obtained in step 1, define a weight matrix and improve the traditional edge detection operator. This constrains the edge detection range to the vicinity of the target area, avoiding interference from background areas and achieving more accurate edge detection results. The improved edge detection operator is used to construct a weighted edge detection energy term with respect to the level set function. This energy term is minimized when the zero level set is the target contour.

[0016] Step 3: Construct a weighted self-constrained energy term. A variable coefficient is introduced to constrain the relationship between the level set function and the box-level prior information, so that the evolution of the level set function changes dynamically with the degree of similarity between the two. First, the variable coefficient is calculated based on the degree of similarity between the level set function and the box-level prior information. Then, a similarity measure is defined between the level set function and the box-level prior information, and the energy term is obtained by combining it with the variable coefficient. When the zero level set is far away from the target contour, the coefficient takes a larger value, and the prior information is used as the main factor to accelerate the evolution of the level set function towards the target. When the zero level set is close to the target contour, the coefficient takes a smaller value, and the information of the image itself is used as the main factor to achieve fine segmentation of local details in the target area.

[0017] Step 4: Construct the local region fitting energy term and distance regularization term. The local region fitting energy term can extract local information of the image and process grayscale unevenness, while the distance regularization term is used to eliminate reinitialization and maintain stable evolution of the level set function.

[0018] Step 5: Combine the weighted edge detection energy term, the weighted self-constraint energy term, the local region fitting energy term, and the distance regularization term to obtain the total energy functional. Compute the variational derivative of the energy functional and iteratively minimize it using gradient descent. Take the zero level set of the corresponding level set function to obtain the final segmentation contour to complete the image segmentation.

[0019] Beneficial effects:

[0020] Compared with the prior art, the design scheme of the present invention can achieve the following technical effects:

[0021] 1. We propose to use box-level priors, which can significantly reduce the amount of manual annotation and improve efficiency compared with the existing pixel-level prior methods obtained by semantic segmentation.

[0022] 2. Using box-level prior information to construct a weighted edge detection energy term can effectively limit the edge detection range and obtain more accurate detection results.

[0023] 3. Defining a weighted self-constrained energy term can dynamically adjust the curve evolution and reduce the sensitivity to the initial contour, which is more effective and robust than using a constant coefficient.

[0024] 4. Combine edge and region information to better utilize image information, obtain more accurate segmentation results, and expand the scope of application of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 .Method framework flow chart

[0026] Figure 2 .Method calculation flow chart DETAILED DESCRIPTION

[0027] Step 1: Train the target detection network to obtain box-level prior information Ψ and initialize the level set function Φ.

[0028] Prior information plays a guiding role, which can drive the level set function to evolve correctly towards the target area, effectively improving the reliability of the algorithm. The box-level prior information proposed in this invention is obtained by using the target detection network, without the need for manual intervention, which effectively improves the automation level of prior information acquisition. Compared with the existing pixel-level prior information obtained by the segmentation network, model training only requires box-level labels, without pixel-level annotation, and the amount of manual annotation required is significantly reduced, which is more efficient. The specific process is as follows:

[0029] (1) Obtain image data, mark the target box, and obtain box-level labels to complete the dataset construction. Build the target detection network. Here, the YOLOF target detection algorithm is used. YOLOF is a one-stage detection algorithm that only needs one feature layer to achieve similar results as using multiple feature layers (feature pyramid FPN), and has faster inference speed. The network model consists of three parts: Backbone, Encoder, and Decoder. Backbone uses ResNet and ResNeXt to output C5 features with 2048 channels. The Encoder takes the C5 features output by Backbone as input and first uses 1x1 convolution and 3x3 convolution to reduce the number of feature channels to 512. Then, it uses 4 consecutive residual modules (in each residual module, 1x1 convolution is first used to reduce the number of feature channels to 1 / 4 of the original number, then 3x3 dilated convolution is used to increase the receptive field, and finally 1x1 convolution is used to expand the number of feature channels by 4 times) to extract features. The Decoder performs regression and classification and decodes the output feature layer into the detection result. We write each part of the network structure according to the YOLOF algorithm to complete the network model construction. After writing, the constructed data set is input into the model for training to obtain the YOLOF target detection network.

[0030] (2) Input the test image I to be segmented into the trained YOLOF target detection network and output the detection result [x, y, w, h], which represents the center point coordinates, width and height of the predicted bounding box respectively. According to the bounding box coordinates [x, y, w, h], a box-level box with point (x, y) as the center and w, h as the width and height is obtained. The internal area of the box-level box is represented by B, the image domain where the test image I is located is represented by Ω, c0 is a positive constant, and the box-level prior information Ψ is defined as follows:

[0031]

[0032] (3) Use Ω0 to represent the subset of the image domain Ω, represents all points on the boundary of Ω0, c0 is a positive constant, and the level set function Φ is initialized as follows;

[0033]

[0034] Step 2: Construct the weighted edge detection energy term.

[0035] This is a widely used normalized edge detection operator. Its value approaches 1 in uniform areas of the image and 0 at edges. Because the edge detection operator g acts on the entire image, it is susceptible to noise and grayscale variations in background areas, causing the detection of redundant false edges to interfere with the evolution of the level set function. We have improved this edge detection operator by introducing a weight matrix that limits detection to the vicinity of the target area, eliminating the influence of background areas and achieving more accurate edge detection results. We use this improved edge detection operator to define an energy term. When the energy term is minimized, the zero level set represents the target contour edge, effectively improving segmentation accuracy.

[0036] First, two weight matrices are defined based on the box-level prior information obtained in step 1 to limit the edge detection range. Then, the weight matrices are used to linearly transform the traditional edge detection operator g(x) to obtain a new edge detection operator, which is used to detect edges near the target area and eliminate interference from other background areas. Finally, the new edge detection operator is combined with the level set function to construct the energy term. The specific implementation is as follows:

[0037] (1) m and n represent the height and width of the test image I to be segmented. w and The weight matrix is m*n in size. The weight matrix is defined according to the box-level prior information as follows:

[0038]

[0039]

[0040] (2) The original edge detection operator g(x) easily detects noise and complex background as edges, which interferes with the segmentation results. For more effective detection, we use the box-level prior information to define the weight matrix to constrain the detection range, so that only edge detection is performed near the target area, and the rest of the area is regarded as a uniform background area. Combining the weight matrix and linear transformation, a new edge detection operator e is defined as:

[0041]

[0042] (3) Use the edge detection operator e to construct the edge detection energy term E edg (Φ):

[0043]

[0044] Where H represents the step function defined as:

[0045]

[0046] Step 3: Construct weighted self-constrained energy term

[0047] The similarity measure and the variable coefficient F(s) are used to construct a weighted self-constrained energy term to dynamically adjust the relationship between the level set function Φ and the prior information Ψ. When the zero level set is far away from the target contour, the prior information plays a dominant role, guiding the level set function to evolve toward the target area. When the zero level set is close to the target contour, the role of the prior information is weakened, and the image information itself plays a dominant role, completing further fine segmentation of the local area.

[0048] The variable coefficient changes dynamically with the degree of similarity between the level set function and the prior information. The similarity is measured by area. The area of the internal area of the zero level set and the area of the intersection of the internal area of the zero level set and the internal area of the box-level frame are calculated respectively. The value obtained by subtracting the former from the latter represents the degree of similarity. The larger the value, the lower the similarity. The function is further used to normalize the value to obtain the variable coefficient, which satisfies the requirement that the value of the variable coefficient increases when the similarity decreases and decreases when the similarity increases. Finally, the squared Euclidean distance between the level set function and the box-level prior information is calculated and multiplied by the variable coefficient to obtain the weighted self-constraint energy term. The specific implementation is as follows:

[0049] (1) The widely used constant coefficient constraint energy term has a constant effect during the evolution process, which will weaken the effect of other energy terms and disrupt the segmentation boundary. Therefore, we propose to use a variable coefficient. First, we use the area value to measure the similarity between the segmentation contour and the prior information, and then use this value to define the variable coefficient. When the segmentation contour is closer to the prior information, the effect of the constraint energy term is weaker, and when it is farther away from the prior information, the effect is stronger, effectively avoiding interference from irrelevant areas near the target to be segmented. The variable coefficient F(s) is calculated as follows:

[0050] s=Area(Φ<0)-Area((Φ<0)∩(Ψ<0))

[0051]

[0052] Where Area(Z) represents the area of the calculation region Z.

[0053] (2) Using the variable coefficient and similarity measure, the weighted self-constrained energy term is constructed as follows:

[0054]

[0055] This item makes the evolution speed increase and approach the target faster when the zero level set is far away from the target contour; when it is close to the target contour, the evolution speed decreases and gradually approaches the target contour to prevent over-segmentation.

[0056] Step 4: Construct the local area fitting energy term and distance regularization term to extract the local area information of the image, eliminate reinitialization, and make the level set function evolve stably.

[0057] Considering that the image to be segmented is usually accompanied by weak boundaries and uneven grayscale, in addition to using the edge information in step 2, the local area information of the image is further introduced to obtain a more accurate segmentation result. Define C to represent the closed contour curve formed by the level set function Φ(x)=0, Φ(x)>0 constitutes the outer area of the curve C, represented by Ω1, and Φ(x)<0 constitutes the inner area of the curve C, represented by Ω2. Introduce the kernel function K σ And use its local attributes to extract the grayscale information of each pixel point x in the local neighborhood, and construct the local area fitting energy term E reg (Φ) is as follows:

[0058]

[0059] in

[0060]

[0061] p1(x) and p2(x) represent the weighted average grayscale values of pixels within regions Ω1 and Ω2, respectively, within a neighborhood centered on each pixel x. M1(Φ) = H(Φ) and M2(Φ) = 1-H(Φ), respectively, indicating the outer and inner regions of the curve.

[0062] In order to ensure the stability of the curve evolution, it is necessary to make the level set function maintain the signed distance property in the area near its zero level set, that is, to satisfy The distance regularization term is introduced as follows:

[0063]

[0064] in

[0065]

[0066] Step 5: Combine the weighted edge detection energy term, the weighted self-constraint energy term, the local region fitting energy term, and the distance regularization term to obtain the total energy functional. Calculate the variation of the energy functional, iteratively minimize the energy functional using gradient descent flow, and take the zero level set of the corresponding level set function to obtain the final segmentation contour to complete the image segmentation. The specific calculation is as follows:

[0067] (1) Total energy functional:

[0068]

[0069] (2) Calculate the variation of the energy functional E(Φ) and obtain the gradient descent flow equation

[0070]

[0071] in

[0072] ei (x)=∫ Ω K σ (yx)|I(x)-p i (y)| 2 dy,i=1,2

[0073]

[0074]

[0075]

[0076] (3) Update the level set iteration formula

[0077]

[0078] (4) Determine whether the iteration stop condition |Φ is met n+1 -Φ n |<ν, if not satisfied, return to step (3) and continue iterating, if satisfied, output Φ n+1 The zero level set of is used to complete the image segmentation.

Claims

1. A hybrid level set image segmentation method based on box-level prior and weight self-constraint, characterized by: At least the following steps are included: Step 1: Define box-level prior information and initialize the level set function. First, obtain the image to build a dataset, establish a network structure model, input the dataset into the network model for training, and obtain a trained target detection network. Then, input the test image to be segmented into the trained target detection network, and define the box-level prior information based on the output detection results. Finally, take any constant to initialize the level set function. Step 2: Construct a weighted edge detection energy term. Use the box-level prior information obtained in step 1 to define a weight matrix. Improve the traditional edge detection operator, constrain the edge detection range to the vicinity of the target area, avoid interference from the background area, and obtain more accurate edge detection results. Use the improved edge detection operator to construct a weighted edge detection energy term about the level set function. When the zero level set is the target contour, the energy term is minimized. Step 3: Construct a weighted self-constrained energy term and introduce a variable coefficient to constrain the relationship between the level set function and the box-level prior information, so that the evolution of the level set function changes dynamically with the degree of similarity between the two. First, calculate the variable coefficient based on the degree of similarity between the level set function and the box-level prior information. Then, define a similarity measure between the level set function and the box-level prior information, and combine it with the variable coefficient to obtain the energy term. When the zero level set is far away from the target contour, the coefficient takes a larger value, and the prior information is used as the main factor to accelerate the evolution of the level set function toward the target. When the zero level set is close to the target contour, the coefficient takes a smaller value, and the information of the image itself is used as the main factor to achieve fine segmentation of local details of the target area. Step 4: Construct a local region fitting energy term and a distance regularization term. The local region fitting energy term can extract local information of the image and process grayscale unevenness. The distance regularization term is used to eliminate reinitialization and maintain stable evolution of the level set function. Step 5. Combine the weighted edge detection energy term, weighted self-constraint energy term, local area fitting energy term and distance regularization term constructed in steps 2, 3 and 4 to obtain the total energy functional; calculate the variation of the energy functional, use the gradient descent flow to iteratively minimize the energy functional, take the zero level set of the corresponding level set function to obtain the final segmentation contour to complete the image segmentation.

2. The hybrid level set image segmentation method based on box-level prior and weight self-constraint according to claim 1, characterized in that: In step 2, a weighted edge detection energy term is constructed. First, two weight matrices are defined to limit the edge detection range based on the box-level prior information obtained in step 1. Then, the weight matrix is used to perform a linear transformation on the traditional edge detection operator to obtain a new edge detection operator for edge detection near the target area. Finally, the new edge detection operator is combined with the level set function to obtain a weighted edge detection energy term. When the energy term is minimized, the zero level set is the target contour edge, which effectively improves the segmentation accuracy.

3. The hybrid level set image segmentation method based on box-level prior and weight self-constraint according to claim 1, characterized in that: In step 3, the weighted self-constrained energy term is constructed. First, the variable coefficient is defined according to the similarity between the level set function and the box-level prior information. The similarity is measured by the area change. First, the area of the zero level set internal area and the area of the intersection of the zero level internal area and the box-level box internal area are calculated respectively. The value obtained by subtracting the former from the latter represents the similarity. Then, the value is normalized by a function to obtain a variable coefficient, which satisfies the requirement that the value of the variable coefficient increases when the similarity decreases and decreases when the similarity increases. Finally, the squared Euclidean distance between the level set function and the box-level prior information is calculated and multiplied by the variable coefficient to obtain the weighted self-constraint energy term.

Images